Cover Image
QR Code

An unsupervised and customizable misspelling generator for mining noisy health-related text sources.

Background: Data collection and extraction from noisy text sources such as social media typically rely on keyword-based searching/listening. However, health-related terms are often misspelled in such noisy text sources due to their complex morphology, resulting in the exclusion of relevant data for...

Full description

Published in:Journal of Biomedical Informatics Vol. 88; pp. 98 - 108
Main Authors: Sarker, Abeed, Gonzalez-Hernandez, Graciela
Format: research tables/charts Journal Article
Published: Academic Press Inc. Dec2018
Online Access:View this record in EBSCOhost
fields @attributes:
  recordID: 1
pdfLink:
plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=133557515&site=ehost-live
header:
  @attributes:
    shortDbName: ccm
    uiTerm: 133557515
    longDbName: CINAHL Complete
    uiTag: AN
  controlInfo:
    bkinfo:
    dissinfo:
    jinfo:
      jid:
        15320464
        OMB
      jtl: Journal of Biomedical Informatics
      issn: 15320464
      maglogo: N
    pubinfo:
      dt: Dec2018
      vid: 88
      pid: 735
      pub: Academic Press Inc.
      place: Burlington, Massachusetts
    artinfo:
      ui:
        133557515
        133557515
        NLM30445220
        133557515
        10.1016/j.jbi.2018.11.007
        NLM30445220
        133557515
      ppf: 98
      ppct: 10
      formats:
      tig:
        atl: An unsupervised and customizable misspelling generator for mining noisy health-related text sources.
      aug:
        au:
          Sarker, Abeed
          Gonzalez-Hernandez, Graciela
        affil: Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, United States
      sug:
        subj:
          Social Media
          Data Mining Methods
          Medical Informatics Methods
          Natural Language Processing
          Data Collection
          Logic
          Information Science
          Algorithms
          Reproducibility of Results
          Scales
          Funding Source
      ab: Background: Data collection and extraction from noisy text sources such as social media typically rely on keyword-based searching/listening. However, health-related terms are often misspelled in such noisy text sources due to their complex morphology, resulting in the exclusion of relevant data for studies. In this paper, we present a customizable data-centric system that automatically generates common misspellings for complex health-related terms, which can improve the data collection process from noisy text sources.Materials and Methods: The spelling variant generator relies on a dense vector model learned from large, unlabeled text, which is used to find semantically close terms to the original/seed keyword, followed by the filtering of terms that are lexically dissimilar beyond a given threshold. The process is executed recursively, converging when no new terms similar (lexically and semantically) to the seed keyword are found. The weighting of intra-word character sequence similarities allows further problem-specific customization of the system.Results: On a dataset prepared for this study, our system outperforms the current state-of-the-art medication name variant generator with best F1-score of 0.69 and F14-score of 0.78. Extrinsic evaluation of the system on a set of cancer-related terms demonstrated an increase of over 67% in retrieval rate from Twitter posts when the generated variants are included.Discussion: Our proposed spelling variant generator has several advantages over past spelling variant generators-(i) it is capable of filtering out lexically similar but semantically dissimilar terms, (ii) the number of variants generated is low, as many low-frequency and ambiguous misspellings are filtered out, and (iii) the system is fully automatic, customizable and easily executable. While the base system is fully unsupervised, we show how supervision may be employed to adjust weights for task-specific customizations.Conclusion: The performance and relative simplicity of our proposed approach make it a much-needed spelling variant generation resource for health-related text mining from noisy sources. The source code for the system has been made publicly available for research.
      pubtype: Academic Journal
      doctype:
        research
        tables/charts
        Journal Article
      ougenre: Article
    language: English
    refInfo:
    holdings:
      @attributes:
        islocal: N