We propose to use ideas from predictive coding theory to augment BERT-style language models with a mechanism that allows them to learn suitable discourse-level ...
Sep 10, 2021 · We propose to use ideas from predictive coding theory to augment BERT-style language models with a mechanism that allows them to learn suitable discourse-level ...
This work proposes to use ideas from predictive coding theory to augment BERT-style language models with a mechanism that allows them to learn suitable ...
Sep 11, 2024 · By experimenting with benchmarks designed to evaluate discourse-related knowledge using pre-trained sentence representations, we demonstrate ...
Current language models are usually trained using a self-supervised scheme, where the main focus is learning representations at the word or sentence level.
Apr 27, 2024 · This paper is an extension of work originally presented at the Conference on Empirical Methods in Natural Language Processing 2021 as a ...
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations. Author: Araujo, Vladimir. Villa, Andres ; Mendoza, Marcelo ...
On-demand video platform giving you access to lectures from conferences worldwide.
Comunidad Cenia How To · CENIA · twitter · youtube · En · CENIA. Inicio · Nosotros; ¿Qué hacemos? Investigación Transferencia Tecnológica Vinculación.
2023. Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations. V Araujo, A Villa, M Mendoza, MF Moens, A Soto.