In this work, we show that it is possible to reduce interference by instead identifying and pruning language-specific attention heads.

Oct 11, 2022 · In this work, we show that it is possible to reduce interference by instead identifying and pruning language-specific parameters.

May 2, 2023 · Our pruning procedure works with any signed im- portance metric. Specifically, we test the utility of the Shapley Values metric for removing ...

Oct 11, 2022 · This work uses Shapley Values, a credit allocation metric from coalitional game theory, to identify attention heads that introduce ...

Sep 29, 2024 · In this work, we show that it is possible to reduce interference by instead identifying and pruning language-specific parameters. First, we use ...

Oct 11, 2022 · By using Shapley Values as the basis of structured pruning for multilingual tasks, we measure their practical ability to remove interference ...

Co-authors ; Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers. W Held, D Yang. Proceedings of the 17th Conference of the ...

Language Modelling +2. 23. Paper · Code · Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers · no code implementations • ...

Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers. William Held, Diyi Yang. EACL, 2023. [pdf]. Bounding the Capabilities ...

https://dblp.org/rec/conf/eacl/HeldY23. William Held, Diyi Yang: Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers.