Essays about: "Multilingual BERT"

Showing result 11 - 15 of 22 essays containing the words Multilingual BERT.

  1. 11. What the BERT? : Fine-tuning KB-BERT for Question Classification

    University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

    Author : Jonatan Cervall; [2021]
    Keywords : BERT; KB-BERT; Question Classification; TREC; Li and Roth Taxonomy; BERT; KB-BERT; Frågeklassificering; TREC; Li och Roth Taxonomi;

    Abstract : This work explores the capabilities of KB-BERT on the downstream task of Question Classification. The TREC data set for Question Classification with the Li and Roth taxonomy was translated to Swedish, by manually correcting the output of Google’s Neural Machine Translation. 500 new data points were added. READ MORE

  3. 12. Extractive Text Summarization of Norwegian News Articles Using BERT

    University essay from Linköpings universitet/Medie- och Informationsteknik; Linköpings universitet/Tekniska fakulteten

    Author : Thomas Indrias Biniam; Adam Morén; [2021]
    Keywords : extractive text summarization; NLP; deep learning; BERT; BERTSum; Multilingual BERT; Norwegian BERT; transformer; Norwegian; news articles;

    Abstract : Extractive text summarization has over the years been an important research area in Natural Language Processing. Numerous methods have been proposed for extracting information from text documents. Recent works have shown great success for English summarization tasks by fine-tuning the language model BERT using large summarization datasets. READ MORE

  4. 13. Large-Context Question Answering with Cross-Lingual Transfer

    University essay from Uppsala universitet/Institutionen för informationsteknologi

    Author : Markus Sagen; [2021]
    Keywords : Long-Context Multilingual Model; Longformer XLM-R; Longformer; Long-term Context; Extending Context; Extend Context; Large-Context; Long-Context; Large Context; Long Context; Cross-Lingual; Multi-Lingual; Cross Lingual; Multi Lingual; QA; Question-Answering; Question Answering; Transformer model; Machine Learning; Transfer Learning; SQuAD; Memory; Transfer Learning; Long-Context; Long Context; Efficient; Monolingual; Multilingual; QA model; Language Model; Huggingface; BERT; RoBERTa; XLM-R; mBERT; Multilingual BERT; Efficient Transformers; Reformer; Linformer; Performer; Transformer-XL; Wikitext-103; TriviaQA; HotpotQA; WikiHopQA; VINNOVA; Peltarion; AI; LM; MLM; Deep Learning; Natural Language Processing; NLP; Attention; Transformers; Transfer Learning; Datasets;

    Abstract : Models based around the transformer architecture have become one of the most prominent for solving a multitude of natural language processing (NLP)tasks since its introduction in 2017. However, much research related to the transformer model has focused primarily on achieving high performance and many problems remain unsolved. READ MORE

  5. 14. QPLaBSE: Quantized and Pruned Language-Agnostic BERT Sentence Embedding Model : Production-ready compression for multilingual transformers

    University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

    Author : Sarthak Langde; [2021]
    Keywords : Transformers; LaBSE; Quantization; Pruning; PyTorch; TensorFlow; ONNX; Transformatorer; LaBSE; Kvantisering; Beskärning; PyTorch; TensorFlow; ONNX;

    Abstract : Transformer models perform well on Natural Language Processing and Natural Language Understanding tasks. Training and fine-tuning of these models consume a large amount of data and computing resources. Fast inference also requires high-end hardware for user-facing products. READ MORE

  6. 15. Multilingual Zero-Shot and Few-Shot Causality Detection

    University essay from Uppsala universitet/Institutionen för lingvistik och filologi

    Author : Sebastian Michael Reimann; [2021]
    Keywords : classification; causality; causal relation; multilingual; cross-lingual; zero-shot; few-shot; bert; xlm-r; laser;

    Abstract : Relations that hold between causes and their effects are fundamental for a wide range of different sectors. Automatically finding sentences that express such relations may for example be of great interest for the economy or political institutions. READ MORE