VIIDA and InViDe: computational approaches for generating and evaluating inclusive image paragraphs for the visually impaired.
Background: Existing image description methods when used as Assistive Technologies often fall short in meeting the needs of blind or low vision (BLV) individuals. They tend to either compress all visual elements into brief captions, create disjointed sentences for each image region, or provide exten...
| Publicado en: | Disability & Rehabilitation: Assistive Technology Vol. 20; no. 5; pp. 1470 - 1496 |
|---|---|
| Autores principales: | , , , |
| Formato: | algorithm equations & formulas pictorial research tables/charts Journal Article |
| Publicado: |
Taylor & Francis Ltd
Jul2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=186130655&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 186130655 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 17483107 1X04 jtl: Disability & Rehabilitation: Assistive Technology issn: 17483107 maglogo: Y pubinfo: dt: Jul2025 vid: 20 iid: 5 pid: 377 pub: Taylor & Francis Ltd place: Philadelphia, Pennsylvania artinfo: ui: 186130655 181564377 186130655 186130655 10.1080/17483107.2024.2437567 186130655 ppf: 1470 ppct: 26 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: VIIDA and InViDe: computational approaches for generating and evaluating inclusive image paragraphs for the visually impaired. aug: au: Fernandes, Daniel L. Ribeiro, Marcos H. F. Silva, Michel M. Cerqueira, Fabio R. affil: Department of Informatics, Universidade Federal de Viçosa - UFV, Viçosa, Brazil sug: subj: Persons with Visual Disabilities Webinars Assistive Technology Mathematics Artificial Intelligence, Generative Human Funding Source Needs Assessment Natural Language Processing Algorithms Linguistics Word Lists Content Analysis Qualitative Studies Photography Communication Aids for Persons with Disabilities ab: Background: Existing image description methods when used as Assistive Technologies often fall short in meeting the needs of blind or low vision (BLV) individuals. They tend to either compress all visual elements into brief captions, create disjointed sentences for each image region, or provide extensive descriptions. Purpose: To address these limitations, we introduce VIIDA, a procedure aimed at the Visually Impaired which implements an Image Description Approach, focusing on webinar scenes. We also propose InViDe, an Inclusive Visual Description metric, a novel approach for evaluating image descriptions targeting BLV people. Methods: We reviewed existing methods and developed VIIDA by integrating a multimodal Visual Question Answering model with Natural Language Processing (NLP) filters. A scene graph-based algorithm was then applied to structure final paragraphs. By employing NLP tools, InViDe conducts a multicriteria analysis based on accessibility standards and guidelines. Results: Experiments statistically demonstrate that VIIDA generates descriptions closely aligned with image content as well as human-written linguistic features, and that suit BLV needs. InViDe offers valuable insights into the behaviour of the compared methods – among them, state-of-the-art methods based on Large Language Models – across diverse criteria. Conclusion: VIIDA and InViDe emerge as efficient Assistive Technologies, combining Artificial Intelligence models and computational/mathematical techniques to generate and evaluate image descriptions for the visually impaired with low computational costs. This work is anticipated to inspire further research and application development in the domain of Assistive Technologies. Our codes are publicly available at: . IMPLICATIONS FOR REHABILITATION: Development of low-cost computational approaches for generating and automatically evaluating image descriptions based on accessibility standards and rules for people with visual impairments as Assistive Technology, thus increasing inclusion and reducing accessibility limitations. Extraction of semantic visual information and modelling of textual descriptions of images using current Computer Vision and Natural Language Processing models and techniques. The synthetic images generated from the paragraphs produced by our approaches closely resemble the original images in terms of semantic similarity and statistical distribution of features. As this work is one of the few studies in the area and is characterised by flexibility and interpretability, researchers can use the approaches presented here to produce new or improve existing Assistive Technologies for the visually impaired. pubtype: Academic Journal doctype: algorithm equations & formulas pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|