A Comparison of DeepSeek and other LLMs.
Recently, DeepSeek has been the focus of attention in and beyond the AI community. An interesting problem is how DeepSeek compares to other large language models (LLMs). There are many tasks an LLM can do, and in this article, we use the task of predicting an outcome using a short text for compariso...
| Publicado en: | American Statistician Vol. 80; no. 1; pp. 164 - 177 |
|---|---|
| Autores principales: | , , , |
| Formato: | Artículo |
| Publicado: |
Taylor & Francis Ltd
Feb2026
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=191630194&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 191630194 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00031305 STT jtl: American Statistician issn: 00031305 maglogo: Y pubinfo: dt: Feb2026 vid: 80 iid: 1 pid: 377 pub: Taylor & Francis Ltd artinfo: ui: 191630194 10.1080/00031305.2025.2611010 ppf: 164 ppct: 13 formats: tig: atl: A Comparison of DeepSeek and other LLMs. aug: au: Gao, Tianchen Jin, Jiashun Ke, Zheng Tracy Moryoussef, Gabriel affil: Beijing International Center for Mathematical Research, Peking University, Beijing, China Department of Statistics & Data Science, Carnegie Mellon University, Pittsburgh, PA Department of Statistics, Harvard University, Cambridge, MA su: Electronic data processing Language models Classification Attribution of authorship Benchmark problems (Computer science) sug: subj: Electronic data processing Data Processing, Hosting, and Related Services Language models Classification Attribution of authorship Benchmark problems (Computer science) keyword: AI-generated text detection Citation classification MADStat Prompt Text analysis Textual content AI-generated text detection Citation classification MADStat Prompt Text analysis Textual content ab: Recently, DeepSeek has been the focus of attention in and beyond the AI community. An interesting problem is how DeepSeek compares to other large language models (LLMs). There are many tasks an LLM can do, and in this article, we use the task of predicting an outcome using a short text for comparison. We consider two settings, an authorship classification setting and a citation classification setting. In the first one, the goal is to determine whether a short text is written by human or AI. In the second one, the goal is to classify a citation into one of four types using the textual content. For each experiment, we compare DeepSeek with four popular LLMs: Claude, Gemini, GPT, and Llama. We find that, in terms of classification accuracy, DeepSeek outperforms Gemini, GPT, and Llama in most cases, but underperforms Claude. We also find that DeepSeek is comparably slower than others but with a low cost to use, while Claude is much more expensive than all the others. Finally, we find that in terms of similarity, the output of DeepSeek is most similar to those of Gemini and Claude (and among all five LLMs, Claude and Gemini have the most similar outputs). In this article, we also present a fully-labeled dataset collected by ourselves, and propose a recipe where we can use the LLMs and a recent dataset, MADStat, to generate new datasets. The datasets in our article can be used as benchmarks for future study on LLMs. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|