Refine
Year of publication
- 2023 (28) (remove)
Institute
- Fachbereich Medizintechnik und Technomathematik (28) (remove)
Language
- English (28) (remove)
Document Type
- Article (22)
- Conference Proceeding (4)
- Habilitation (1)
- Preprint (1)
Keywords
- Information extraction (3)
- Natural language processing (3)
- ultrasound (2)
- Active learning (1)
- Agent-based simulation (1)
- Architectural design (1)
- Asymptotic relative efficiency (1)
- Bacillus atrophaeus spores (1)
- Bacterial cellulose (1)
- Bioabsorbable (1)
- Capacitive field-effect sensor (1)
- Carbon sources (1)
- Cellulose nanostructure (1)
- Clustering (1)
- Competitiveness (1)
- Conductive Boundary Condition (1)
- Cost-effectiveness (1)
- Cramér-von-Mises test (1)
- Cross border adjustment mechanism (1)
- Culture media (1)
- DPA (dipicolinic acid) (1)
- Deep learning (1)
- E-Mobility (1)
- Endothelial dysfunction (1)
- Energy market design (1)
- Energy-intensive industry (1)
- Enterprise information systems (1)
- Fault approximation (1)
- Fault detection (1)
- Floor prices (1)
- Geriatric (1)
- Gold nanoparticles (1)
- Hip fractures (1)
- Incomplete data (1)
- Inverse Scattering (1)
- Inverse scattering problem (1)
- Label-free detection (1)
- LbL films (1)
- Long COVID (1)
- MCDA (1)
- Marginal homogeneity (1)
- Market modeling (1)
- Medusomyces gisevi (1)
- Mobility transition (1)
- Model-driven software engineering (1)
- Multi-criteria decision analysis (1)
- Multicell (1)
- Multiplexing (1)
- Natural language understanding (1)
- Paired sample (1)
- Polylactide acid (1)
- Post-COVID-19 syndrome (1)
- Preference assessment (1)
- Prevention (1)
- Profile extraction (1)
- Prophylaxis (1)
- Query learning (1)
- Raman spectroscopy (1)
- Regionalization (1)
- Relation classification (1)
- Reproducible research (1)
- Resistive temperature detector (1)
- Silk fibroin (1)
- Sn₃O₄ (1)
- Software and systems modeling (1)
- Steel industry (1)
- Text mining (1)
- Transmission Eigenvalues (1)
- Trustworthy artificial intelligence (1)
- Volumes of confidence regions (1)
- allocation (1)
- amperometric biosensors (1)
- biocompatible (1)
- biodegradabl (1)
- biomechanics (1)
- biosensor (1)
- central symmetry test (1)
- conditional excess distribution (1)
- conditional expectation principle (1)
- confidence interval (1)
- connective tissue (1)
- covariance principle (1)
- electromyography (1)
- encapsulation materials (1)
- enzyme cascade (1)
- exchangeability test (1)
- fibroin (1)
- field-effect sensor (1)
- forecast (1)
- glucose oxidase (GOx) (1)
- goodness-of-fit test (1)
- heavy metals (1)
- horseradish peroxidase (HRP) (1)
- independence test (1)
- locomotion (1)
- nanobelts (1)
- not identically distributed (1)
- optical sensor setup (1)
- optical trapping (1)
- overload (1)
- physiology (1)
- portfolio risk (1)
- random effects (1)
- retinal microvasculature (1)
- sterilization (1)
- stretch-shortening cycle (1)
- tobacco mosaic virus (TMV) (1)
- turnip vein clearing virus (TVCV) (1)
Supervised machine learning and deep learning require a large amount of labeled data, which data scientists obtain in a manual, and time-consuming annotation process. To mitigate this challenge, Active Learning (AL) proposes promising data points to annotators they annotate next instead of a subsequent or random sample. This method is supposed to save annotation effort while maintaining model performance.
However, practitioners face many AL strategies for different tasks and need an empirical basis to choose between them. Surveys categorize AL strategies into taxonomies without performance indications. Presentations of novel AL strategies compare the performance to a small subset of strategies. Our contribution addresses the empirical basis by introducing a reproducible active learning evaluation (ALE) framework for the comparative evaluation of AL strategies in NLP.
The framework allows the implementation of AL strategies with low effort and a fair data-driven comparison through defining and tracking experiment parameters (e.g., initial dataset size, number of data points per query step, and the budget). ALE helps practitioners to make more informed decisions, and researchers can focus on developing new, effective AL strategies and deriving best practices for specific use cases. With best practices, practitioners can lower their annotation costs. We present a case study to illustrate how to use the framework.