Essays about: "Byte Pair Encoding"
Found 3 essays containing the words Byte Pair Encoding.
-
1. Incremental Re-tokenization in BPE-trained SentencePiece Models
University essay from Umeå universitet/Institutionen för datavetenskapAbstract : This bachelor's thesis in Computer Science explores the efficiency of an incremental re-tokenization algorithm in the context of BPE-trained SentencePiece models used in natural language processing. The thesis begins by underscoring the critical role of tokenization in NLP, particularly highlighting the complexities introduced by modifications in tokenized text. READ MORE
-
2. Question answering on introductory Java programming concepts using the Transformer
University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)Abstract : AI applications for education could help students learn in their introductory programming courses. Many applications for education try to simulate a humantutoring session that engages the student in a dialogue. During the session, they can ask questions and have them answered while working throughan exercise. READ MORE
-
3. Bidirectional LSTM-CNNs-CRF Models for POS Tagging
University essay from Uppsala universitet/Institutionen för lingvistik och filologiAbstract : In order to achieve state-of-the-art performance for part-of-speech(POS) tagging, the traditional systems require a significant amount of hand-crafted features and data pre-processing. In this thesis, we present a discriminative word embedding, character embedding and byte pair encoding (BPE) hybrid neural network architecture to implement a true end-to-end system without feature engineering and data pre-processing. READ MORE
