Essays about: "Mohammad Al-Jaff"

Found 1 essay containing the words Mohammad Al-Jaff.

  1. 1. Messing With The Gap: On The Modality Gap Phenomenon In Multimodal Contrastive Representation Learning

    University essay from Uppsala universitet/Industriell teknik

    Author : Mohammad Al-Jaff; [2023]
    Keywords : Multimodal machine learning; Representation learning; Self-supervised learning; contrastive learning; computer vision; computational biology; bioinformatics;

    Abstract : In machine learning, a sub-field of computer science, a two-tower architecture model is a specialised type of neural network model that encodes paired data from different modalities (like text and images, sound and video, or proteomics and gene expression profiles) into a shared latent representation space. However, when training these models using a specific contrastive loss function, known as the multimodalinfoNCE loss, seems to often lead to a unique geometric phenomenon known as the modality gap. READ MORE