Infectious risk events and their novelty in event-based surveillance: new definitions and annotated corpus: Infectious risk events and their novelty in event-based surveillance: F. Delon et al.
Event-based surveillance (EBS) requires the analysis of an ever-increasing volume of documents, requiring automated processing to support human analysts. Few annotated corpora are available for the evaluation of information extraction tools in the EBS domain. The main objective of this work was to b...
| Published in: | Language Resources & Evaluation Vol. 59; no. 1; pp. 277 - 296 |
|---|---|
| Main Authors: | , , , , , , , |
| Format: | Article |
| Published: |
Springer Nature
Mar2025
|
| Subjects: | |
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=183750674&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 183750674 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Mar2025 vid: 59 iid: 1 pid: 237 pub: Springer Nature artinfo: ui: 183750674 10.1007/s10579-024-09728-w ppf: 277 ppct: 19 formats: fmt: – @attributes: type: T – @attributes: type: P size: 1.1MB tig: atl: Infectious risk events and their novelty in event-based surveillance: new definitions and annotated corpus: Infectious risk events and their novelty in event-based surveillance: F. Delon et al. aug: au: Delon, François Bédubourg, Gabriel Bouscarrat, Léo Meynard, Jean-Baptiste Valois, Aude Queyriaux, Benjamin Ramisch, Carlos Tanti, Marc affil: https://ror.org/0508wny29 Aix Marseille Univ, Inserm, IRD, SESSTIM, Sciences Economiques & Sociales de la Santé & Traitement de l'Information Médicale, ISSPAM, Marseille, France https://ror.org/0103yxp25 French Defense Health Service, Paris, France EURA NOVA, Marseille, France https://ror.org/0257sgk90 CNRS, LIS, Université de Toulon, Aix-Marseille Université, Marseille, France https://ror.org/02r084d93 Centre Hospitalier de Cayenne, Cayenne, French Guiana HIPS Agency GmbH, Munich, Germany su: Natural language processing Text mining Artificial intelligence Data mining Image processing sug: subj: Natural language processing Text mining Artificial intelligence Data mining Image processing keyword: Corpus Event-based surveillance Information and Computing Sciences Artificial Intelligence and Image Processing Semantics ab: Event-based surveillance (EBS) requires the analysis of an ever-increasing volume of documents, requiring automated processing to support human analysts. Few annotated corpora are available for the evaluation of information extraction tools in the EBS domain. The main objective of this work was to build a corpus containing documents which are representative of those collected in the current EBS information systems, and to annotate them with events and their novelty. We proposed new definitions of infectious events and their novelty suited to the background work of analysts working in the EBS domain, and we compiled a corpus of 305 documents describing 283 infectious events. There were 36 included documents in French, representing a total of 11 events, with the remainder in English. We annotated novelty for the 110 most recent documents in the corpus, resulting in 101 events. The inter-annotator agreement was 0.74 for event identification (F1-Score) and 0.69 [95% CI: 0.51; 0.88] (Kappa) for novelty annotation. The overall agreement for entity annotation was lower, with a significant variation according to the type of entities considered (range 0.30–0.68). This corpus is a useful tool for creating and evaluating algorithms and methods submitted by EBS research teams for event detection and annotation of their novelties, aiming at the operational improvement of document flow processing. The small size of this corpus makes it less suitable for training natural language processing models, although this limitation tends to fade away when using few-shots learning methods. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2025. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2025 holdings: @attributes: islocal: N |
|---|