My research and practical experience cover the full AI data-preparation cycle: annotation-scheme design, guidelines, annotator training, gold data, agreement metrics, and multi-stage validation.
A separate focus is annotation automation with LLMs and quality comparisons with experts and crowdworkers.