IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set

Steinunn Rut Friðriksdóttir; Hjalti Daníelsson; Steinþór Steingrímsson; Einar Freyr Sigurðsson

IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set

Steinunn Rut Friðriksdóttir, Hjalti Daníelsson, Steinþór Steingrímsson, Einar Freyr Sigurðsson

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Word embedding models have become commonplace in a wide range of NLP applications. In order to train and use the best possible models, accurate evaluation is needed. For extrinsic evaluation of word embedding models, analogy evaluation sets have been shown to be a good quality estimator. We introduce an Icelandic adaptation of a large analogy dataset, BATS, evaluate it on three different word embedding models and show that our evaluation set is apt at measuring the capabilities of such models.

Original language	English
Title of host publication	2022 Language Resources and Evaluation Conference, LREC 2022
Editors	Nicoletta Calzolari, Frederic Bechet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Jan Odijk, Stelios Piperidis
Publisher	European Language Resources Association (ELRA)
Pages	4227-4234
Number of pages	8
ISBN (Electronic)	9791095546726
Publication status	Published - 2022
Event	13th International Conference on Language Resources and Evaluation Conference, LREC 2022 - Marseille, France Duration: 20 Jun 2022 → 25 Jun 2022

Publication series

Name	2022 Language Resources and Evaluation Conference, LREC 2022

Conference

Conference	13th International Conference on Language Resources and Evaluation Conference, LREC 2022
Country/Territory	France
City	Marseille
Period	20/06/22 → 25/06/22

Bibliographical note

Publisher Copyright:
© European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.

Other keywords

analogy test set
BATS
Icelandic
word embeddings

Access to Document

https://aclanthology.org/2022.lrec-1.449/Licence: CC BY

Cite this

Friðriksdóttir, S. R., Daníelsson, H., Steingrímsson, S., & Sigurðsson, E. F. (2022). IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set. In N. Calzolari, F. Bechet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, J. Odijk, & S. Piperidis (Eds.), 2022 Language Resources and Evaluation Conference, LREC 2022 (pp. 4227-4234). (2022 Language Resources and Evaluation Conference, LREC 2022). European Language Resources Association (ELRA). https://aclanthology.org/2022.lrec-1.449/

Friðriksdóttir, Steinunn Rut ; Daníelsson, Hjalti ; Steingrímsson, Steinþór et al. / IceBATS : An Icelandic Adaptation of the Bigger Analogy Test Set. 2022 Language Resources and Evaluation Conference, LREC 2022. editor / Nicoletta Calzolari ; Frederic Bechet ; Philippe Blache ; Khalid Choukri ; Christopher Cieri ; Thierry Declerck ; Sara Goggi ; Hitoshi Isahara ; Bente Maegaard ; Joseph Mariani ; Helene Mazo ; Jan Odijk ; Stelios Piperidis. European Language Resources Association (ELRA), 2022. pp. 4227-4234 (2022 Language Resources and Evaluation Conference, LREC 2022).

@inproceedings{8346c15e09be4d6f8e3492a4a9253e78,

title = "IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set",

abstract = "Word embedding models have become commonplace in a wide range of NLP applications. In order to train and use the best possible models, accurate evaluation is needed. For extrinsic evaluation of word embedding models, analogy evaluation sets have been shown to be a good quality estimator. We introduce an Icelandic adaptation of a large analogy dataset, BATS, evaluate it on three different word embedding models and show that our evaluation set is apt at measuring the capabilities of such models.",

keywords = "analogy test set, BATS, Icelandic, word embeddings",

author = "Fri{\dh}riksd{\'o}ttir, {Steinunn Rut} and Hjalti Dan{\'i}elsson and Stein{\th}{\'o}r Steingr{\'i}msson and Sigur{\dh}sson, {Einar Freyr}",

note = "Publisher Copyright: {\textcopyright} European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.; 13th International Conference on Language Resources and Evaluation Conference, LREC 2022 ; Conference date: 20-06-2022 Through 25-06-2022",

year = "2022",

language = "English",

series = "2022 Language Resources and Evaluation Conference, LREC 2022",

publisher = "European Language Resources Association (ELRA)",

pages = "4227--4234",

editor = "Nicoletta Calzolari and Frederic Bechet and Philippe Blache and Khalid Choukri and Christopher Cieri and Thierry Declerck and Sara Goggi and Hitoshi Isahara and Bente Maegaard and Joseph Mariani and Helene Mazo and Jan Odijk and Stelios Piperidis",

booktitle = "2022 Language Resources and Evaluation Conference, LREC 2022",

}

Friðriksdóttir, SR, Daníelsson, H, Steingrímsson, S & Sigurðsson, EF 2022, IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set. in N Calzolari, F Bechet, P Blache, K Choukri, C Cieri, T Declerck, S Goggi, H Isahara, B Maegaard, J Mariani, H Mazo, J Odijk & S Piperidis (eds), 2022 Language Resources and Evaluation Conference, LREC 2022. 2022 Language Resources and Evaluation Conference, LREC 2022, European Language Resources Association (ELRA), pp. 4227-4234, 13th International Conference on Language Resources and Evaluation Conference, LREC 2022, Marseille, France, 20/06/22. <https://aclanthology.org/2022.lrec-1.449/>

IceBATS : An Icelandic Adaptation of the Bigger Analogy Test Set. / Friðriksdóttir, Steinunn Rut; Daníelsson, Hjalti; Steingrímsson, Steinþór et al.

2022 Language Resources and Evaluation Conference, LREC 2022. ed. / Nicoletta Calzolari; Frederic Bechet; Philippe Blache; Khalid Choukri; Christopher Cieri; Thierry Declerck; Sara Goggi; Hitoshi Isahara; Bente Maegaard; Joseph Mariani; Helene Mazo; Jan Odijk; Stelios Piperidis. European Language Resources Association (ELRA), 2022. p. 4227-4234 (2022 Language Resources and Evaluation Conference, LREC 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - IceBATS

T2 - 13th International Conference on Language Resources and Evaluation Conference, LREC 2022

AU - Friðriksdóttir, Steinunn Rut

AU - Daníelsson, Hjalti

AU - Steingrímsson, Steinþór

AU - Sigurðsson, Einar Freyr

N1 - Publisher Copyright: © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.

PY - 2022

Y1 - 2022

N2 - Word embedding models have become commonplace in a wide range of NLP applications. In order to train and use the best possible models, accurate evaluation is needed. For extrinsic evaluation of word embedding models, analogy evaluation sets have been shown to be a good quality estimator. We introduce an Icelandic adaptation of a large analogy dataset, BATS, evaluate it on three different word embedding models and show that our evaluation set is apt at measuring the capabilities of such models.

AB - Word embedding models have become commonplace in a wide range of NLP applications. In order to train and use the best possible models, accurate evaluation is needed. For extrinsic evaluation of word embedding models, analogy evaluation sets have been shown to be a good quality estimator. We introduce an Icelandic adaptation of a large analogy dataset, BATS, evaluate it on three different word embedding models and show that our evaluation set is apt at measuring the capabilities of such models.

KW - analogy test set

KW - BATS

KW - Icelandic

KW - word embeddings

UR - http://www.scopus.com/inward/record.url?scp=85144417829&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85144417829

T3 - 2022 Language Resources and Evaluation Conference, LREC 2022

SP - 4227

EP - 4234

BT - 2022 Language Resources and Evaluation Conference, LREC 2022

A2 - Calzolari, Nicoletta

A2 - Bechet, Frederic

A2 - Blache, Philippe

A2 - Choukri, Khalid

A2 - Cieri, Christopher

A2 - Declerck, Thierry

A2 - Goggi, Sara

A2 - Isahara, Hitoshi

A2 - Maegaard, Bente

A2 - Mariani, Joseph

A2 - Mazo, Helene

A2 - Odijk, Jan

A2 - Piperidis, Stelios

PB - European Language Resources Association (ELRA)

Y2 - 20 June 2022 through 25 June 2022

ER -

Friðriksdóttir SR, Daníelsson H, Steingrímsson S , Sigurðsson EF. IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set. In Calzolari N, Bechet F, Blache P, Choukri K, Cieri C, Declerck T, Goggi S, Isahara H, Maegaard B, Mariani J, Mazo H, Odijk J, Piperidis S, editors, 2022 Language Resources and Evaluation Conference, LREC 2022. European Language Resources Association (ELRA). 2022. p. 4227-4234. (2022 Language Resources and Evaluation Conference, LREC 2022).

IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set

Abstract

Publication series

Conference

Bibliographical note

Other keywords

Access to Document

Fingerprint

Cite this