Human or Machine? A Comparative Analysis of Artificial Intelligence–Generated Writing Detection in Personal Statements.
Supplemental Digital Content is Available in the Text. Introduction.: This study examines the ability of human readers, recurrence quantification analysis (RQA), and an online artificial intelligence (AI) detection tool (GPTZero) to distinguish between AI-generated and human-written personal stateme...
| Publicado en: | Journal of Physical Therapy Education (Lippincott Williams & Wilkins) Vol. 39; no. 4; pp. 329 - 339 |
|---|---|
| Autores principales: | , , , |
| Formato: | research tables/charts Journal Article |
| Publicado: |
Lippincott Williams & Wilkins
Dec2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=189289533&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 189289533 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 08991855 LETN jtl: Journal of Physical Therapy Education (Lippincott Williams & Wilkins) issn: 08991855 maglogo: N pubinfo: dt: Dec2025 vid: 39 iid: 4 pid: 5086 pub: Lippincott Williams & Wilkins place: Baltimore, Maryland artinfo: ui: 189289533 189289533 189289533 10.1097/JTE.0000000000000396 189289533 ppf: 329 ppct: 10 formats: tig: atl: Human or Machine? A Comparative Analysis of Artificial Intelligence–Generated Writing Detection in Personal Statements. aug: au: Goodman, Margaret A. Lee, Anthony M. Schreck, Zachary Hollman, John H. affil: Program in Physical Therapy in the Mayo Clinic School of Health Sciences at the Mayo Clinic College of Medicine and Science and in the Department of Physical Medicine and Rehabilitation at the Mayo Clinic. sug: subj: Writing Reading Linguistics Evaluation Artificial Intelligence, Generative Education, Physical Therapy Student Selection Human Comparative Studies Authorship Random Sample Descriptive Statistics Data Analysis Software Accountability T-Tests Male Female Adult Confidence Intervals Sensitivity and Specificity Adult: 19-44 years Male Female ab: Supplemental Digital Content is Available in the Text. Introduction.: This study examines the ability of human readers, recurrence quantification analysis (RQA), and an online artificial intelligence (AI) detection tool (GPTZero) to distinguish between AI-generated and human-written personal statements in physical therapist education program applications. Review of Literature.: The emergence of large language models such as ChatGPT and Google Gemini has raised concerns about the authenticity of personal statements. Previous studies have reported varying degrees of success in detecting AI-generated text. Subjects.: Data were collected from 50 randomly selected nonmatriculated individuals who applied to the Mayo Clinic School of Health Sciences Doctor of Physical Therapy Program during the 2021–2022 application cycle. Methods.: Fifty personal statements from applicants were pooled with 50 Google Gemini–generated statements, then analyzed by 2 individuals, RQA, and GPTZero. RQA provided quantitative measures of lexical sophistication, whereas GPTZero used advanced machine learning algorithms to quantify AI-specific text characteristics. Results.: Human raters demonstrated high agreement (κ = 0.92) and accuracy (97% and 99%). RQA parameters, particularly recurrence and max line, differentiated human- from AI-generated statements (areas under receiver operating characteristic [ROC] curve = 0.768 and 0.859, respectively). GPTZero parameters including simplicity, perplexity, and readability also differentiated human- from AI-generated statements (areas under ROC curve > 0.875). Discussion and Conclusion.: The study reveals that human raters, RQA, and GPTZero offer varying levels of accuracy in differentiating human-written from AI-generated personal statements. The findings could have important implications in academic admissions processes, where distinguishing between human- and AI-generated submissions is becoming increasingly important. Future research should explore integrating these methods to enhance the robustness and reliability of personal statement content evaluation across various domains. Three strategies for managing AI's role in applications—for applicants, governing organizations, and academic institutions—are provided to promote integrity and accountability in admission processes. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|