PolitePEER: does peer review hurt? A dataset to gauge politeness intensity in the peer reviews.
Even though peer review is a central aspect of scientific communication, research shows that the process reveals a power imbalance. The position of the reviewer allows them to be harsh and intentionally offensive without being held accountable. It casts doubt on the integrity of the peer-review proc...
| Publicado en: | Language Resources & Evaluation Vol. 58; no. 4; pp. 1291 - 1314 |
|---|---|
| Autores principales: | , , , |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Dec2024
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=180627300&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 180627300 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 1574020X 179V jtl: Language Resources & Evaluation issn: 1574020X maglogo: N pubinfo: dt: Dec2024 vid: 58 iid: 4 pid: 237 pub: Springer Nature artinfo: ui: 180627300 10.1007/s10579-023-09662-3 ppf: 1291 ppct: 23 formats: fmt: – @attributes: type: T – @attributes: type: P size: 1.1MB tig: atl: PolitePEER: does peer review hurt? A dataset to gauge politeness intensity in the peer reviews. aug: au: Bharti, Prabhat Kumar Navlakha, Meith Agarwal, Mayank Ekbal, Asif affil: Department of Computer Science and Engineering, Indian Institute of Technology, Bihta, 801106, Patna, Bihar, India https://ror.org/04d4hxn32 Department of Computer Engineering, Dwarkadas J. Sanghvi College of Engineering, 400056, Mumbai, Maharashtra, India su: Scientific communication Courtesy Computer performance Offensive behavior Professionalism Digital libraries sug: subj: Scientific communication Courtesy Computer performance Offensive behavior Professionalism Digital libraries keyword: Dataset applications Politeness scaling Review comments ab: Even though peer review is a central aspect of scientific communication, research shows that the process reveals a power imbalance. The position of the reviewer allows them to be harsh and intentionally offensive without being held accountable. It casts doubt on the integrity of the peer-review process and transforms it into an unpleasant and traumatic experience for authors. Accordingly, more effort should be given to provide critical and constructive feedback. Hence, it is necessary to remedy the growing rudeness and lack of professionalism in the review system, by analyzing the tone of review comments and creating a classification on the level of politeness in a review comment. To this end, we develop the first annotated PolitePEER dataset encompassing five levels of politeness: (1) highly impolite, (2) impolite, (3) neutral, (4) polite, and (5) highly polite. The review sentences accrued from multiple venues, viz., ICLR, NeurIPS, Publons and ShitMyReviewersSay. We have formulated our annotation guidelines and conducted a thorough analysis of the PolitePEER dataset, ensuring the dataset quality with an inter-annotation agreement of 93%. Additionally, we have benchmarked PolitePEER for multiclass classification and provided an extensive analysis of the proposed baseline. As a result, the proposed PolitePEER can aid in developing a politeness indicator to notify the reviewer and the editors to amend and formalize the review accordingly. Our dataset and codes are available at https://github.com/PrabhatkrBharti/PolitePEER.git for the community to explore further. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Language Resources & Evaluation is a copyright of Springer, 2024. All Rights Reserved. item: Language Resources & Evaluation holder: Springer Nature dt: @attributes: year: 2024 holdings: @attributes: islocal: N |
|---|