Essays about: "switchtransformatorer"

Found 1 essay containing the word switchtransformatorer.

  1. 1. DistillaBSE: Task-agnostic  distillation of multilingual sentence  embeddings : Exploring deep self-attention distillation with switch transformers

    University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

    Author : Boris Bubla; [2021]
    Keywords : Transformers; Knowledge Distillation; Language Agnostic BERT Sentence Embeddings; Natural Language Processing; Switch Transformers; Transformatorer; kunskapsdestillation; språkagnostisk inbäddning av BERT- mening; naturlig bearbetning av språk; switchtransformatorer;

    Abstract : The recent development of massive multilingual transformer networks has resulted in drastic improvements in model performance. These models, however, are so large they suffer from large inference latency and consume vast computing resources. Such features hinder widespread adoption of the models in industry and some academic settings. READ MORE