TY  - JOUR
A1  - Kohl, Philipp
A1  - Krämer, Yoka
A1  - Fohry, Claudia
A1  - Kraft, Bodo
ED  - Fred, Ana
ED  - Hadjali, Allel
ED  - Gusikhin, Oleg
ED  - Sansone, Carlo
T1  - Scoping review of active learning strategies and their evaluation environments for entity recognition tasks
JF  - Deep learning theory and applications
N2  - We conducted a scoping review for active learning in the domain of natural language processing (NLP), which we summarize in accordance with the PRISMA-ScR guidelines as follows:

Objective: Identify active learning strategies that were proposed for entity recognition and their evaluation environments (datasets, metrics, hardware, execution time).

Design: We used Scopus and ACM as our search engines. We compared the results with two literature surveys to assess the search quality. We included peer-reviewed English publications introducing or comparing active learning strategies for entity recognition.

Results: We analyzed 62 relevant papers and identified 106 active learning strategies. We grouped them into three categories: exploitation-based (60x), exploration-based (14x), and hybrid strategies (32x). We found that all studies used the F1-score as an evaluation metric. Information about hardware (6x) and execution time (13x) was only occasionally included. The 62 papers used 57 different datasets to evaluate their respective strategies. Most datasets contained newspaper articles or biomedical/medical data. Our analysis revealed that 26 out of 57 datasets are publicly accessible.

Conclusion: Numerous active learning strategies have been identified, along with significant open questions that still need to be addressed. Researchers and practitioners face difficulties when making data-driven decisions about which active learning strategy to adopt. Conducting comprehensive empirical comparisons using the evaluation environment proposed in this study could help establish best practices in the domain.
Y1  - 2024
SN  - 978-3-031-66694-0 (online ISBN)
SN  - 978-3-031-66693-3 (print ISBN)
U6  - https://doi.org/10.1007/978-3-031-66694-0_6
SP  - 84
EP  - 106
PB  - Springer
CY  - Cham
ER  -