ChatGPT for complex text evaluation tasks.
ChatGPT and other large language models (LLMs) have been successful at natural and computer language processing tasks with varying degrees of complexity. This brief communication summarizes the lessons learned from a series of investigations into its use for the complex text analysis task of researc...
| Publicado en: | Journal of the Association for Information Science & Technology Vol. 76; no. 4; pp. 645 - 649 |
|---|---|
| Autor principal: | |
| Formato: | Journal Article |
| Publicado: |
Wiley-Blackwell
Apr2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=183690122&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 183690122 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 23301635 H6JN jtl: Journal of the Association for Information Science & Technology issn: 23301635 maglogo: N pubinfo: dt: Apr2025 vid: 76 iid: 4 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 183690122 180818837 10.1002/asi.24966 183690122 ppf: 645 ppct: 4 formats: tig: atl: ChatGPT for complex text evaluation tasks. aug: au: Thelwall, Mike affil: Information School, University of Sheffield, Sheffield, UK sug: subj: Artificial Intelligence, Generative Natural Language Processing Reading Task Performance and Analysis Grammar Research, Medical Data Analysis, Computer Assisted ab: ChatGPT and other large language models (LLMs) have been successful at natural and computer language processing tasks with varying degrees of complexity. This brief communication summarizes the lessons learned from a series of investigations into its use for the complex text analysis task of research quality evaluation. In summary, ChatGPT is very good at understanding and carrying out complex text processing tasks in the sense of producing plausible responses with minimum input from the researcher. Nevertheless, its outputs require systematic testing to assess their value because they can be misleading. In contrast to simple tasks, the outputs from complex tasks are highly varied and better results can be obtained by repeating the prompts multiple times in different sessions and averaging the ChatGPT outputs. Varying ChatGPT's configuration parameters from their defaults does not seem to be useful, except for the length of the output requested. pubtype: Academic Journal doctype: Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|