Essays about: "Jae Eun Hong"

Found 1 essay containing the words Jae Eun Hong.

  1. 1. Regularized Fine-tuning Strategies for Neural Language Models : Application of entropy regularization on GPT-2

    University essay from Uppsala universitet/Institutionen för lingvistik och filologi

    Author : Jae Eun Hong; [2022]
    Keywords : text generation; entropy regularization; neural language model; gpt-2;

    Abstract : Deep neural language models like GPT-2 is undoubtedly strong at text generation, but often requires special decoding strategies to prevent producing degenerate output - namely repetition. The use of maximum likelihood training objective results in a peaked probability distribution, leading to the over-confidence of neural networks. READ MORE