Cover Image
QR Code

DETECTION OF MACHINE-TRANSLATED CROATIAN– ENGLISH PHRASES: DO TRANSLATORS AGREE IN THEIR ASSESSMENT?

This study examines the extent to which machine-translated and human-translated Croatian–English phrases can be distinguished by expert evaluators by analysing inter-rater agreement. For this purpose, a parallel corpus was compiled from four Croatian research articles in the fields of Education and...

Full description

Published in:Croatica et Slavica Iadertina Vol. 22; no. 1; pp. 11 - 30
Main Authors: BORUCINSKY, MIRJANA, ČANŽAR, MARGARETA
Format: Article
Published: University of Zadar, Department of Croatian & Slavic Studies 2026
Subjects:
Inter-observer reliability
Translators
Machine translating
Croatian language
Online Access:View this record in EBSCOhost
fields @attributes:
  recordID: 1
pdfLink:
plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=196043116&site=ehost-live
header:
  @attributes:
    shortDbName: hlh
    uiTerm: 196043116
    longDbName: Humanities International Complete
    uiTag: AN
  controlInfo:
    bkinfo:
    jinfo:
      jid:
        18456839
        IQO6
      jtl: Croatica et Slavica Iadertina
      issn: 18456839
      maglogo: N
    pubinfo:
      dt: 2026
      vid: 22
      iid: 1
      pid: 97013
      pub: University of Zadar, Department of Croatian & Slavic Studies
    artinfo:
      ui:
        196043116
        10.15291/csi.5069
      ppf: 11
      ppct: 19
      formats:
        fmt:
          @attributes:
            type: P
            size: 1.1MB
      tig:
        atl: DETECTION OF MACHINE-TRANSLATED CROATIAN– ENGLISH PHRASES: DO TRANSLATORS AGREE IN THEIR ASSESSMENT?
      aug:
        au:
          BORUCINSKY, MIRJANA
          ČANŽAR, MARGARETA
        affil:
          Faculty of Maritime Studies, University of Rijeka.
          Faculty of Humanities and Social Sciences, University of Rijeka.
      su:
        Inter-observer reliability
        Translators
        Machine translating
        Croatian language
      sug:
        subj:
          Inter-observer reliability
          Translators
          Machine translating
          Croatian language
      keyword:
        Croatian–English
        human evaluation
        inter-rater agreement
        machine translation
        hrvatsko-engleski prijevod
        prepoznavanje strojnog prijevoda
        procjena
        suglasnost među ocjenjivačima
      ab:
        This study examines the extent to which machine-translated and human-translated Croatian–English phrases can be distinguished by expert evaluators by analysing inter-rater agreement. For this purpose, a parallel corpus was compiled from four Croatian research articles in the fields of Education and Psychology, together with their human translations and machine translations produced using the online tool onlinedoctranslator.com. Two professional translators, native speakers of Croatian with extensive translation experience, were asked to classify selected English translations of Croatian phrases as either machine-translated or human-translated. The analysis shows a raw inter-rater agreement of 75 % for both machine-translated and human-translated phrase sets, indicating a moderate level of agreement between the raters. These results suggest that even expert translators do not consistently reach the same conclusions when judging the origin of short, decontextualised translated phrases. The findings support the view that, at the phrase level, machine-translated output is increasingly difficult to distinguish from human translation on the basis of surface linguistic features alone.
        U ovom se istraživanju ispituje je li moguće razlikovati strojni prijevod od prijevoda koji je izradio čovjek na primjeru prijevoda hrvatskih izraza na engleski jezik. Analizira se u kojoj mjeri se stručni ocjenjivači slažu u procjeni prijevoda kao strojnoga prijevoda ili prijevoda koji je izradio čovjek. U tu svrhu sastavljen je korpus znanstvenih radova iz područja obrazovanja i psihologije, koji se sastoji od triju potkorpusa: 1. korpus izvornih tekstova na hrvatskome, 2. korpus prijevoda izvornih tekstova na engleski jezik koje je preveo čovjek, 3. korpus strojno prevedenih hrvatskih tekstova na engleski. U zadatku prepoznavanja dva profesionalna prevoditelja, izvorni govornici hrvatskoga, razvrstavala su engleske prijevode odabranih hrvatskih izraza kao strojne ili one koje je preveo čovjek. Slaganje među ocjenjivačima iznosilo je 75 % uz umjerenu podudarnost. Rezultati pokazuju da se prevoditelji ne slažu u potpunosti pri procjeni, što podupire stav da postaje teže razlikovati strojno prevedene izraze od prijevoda koje je izradio čovjek.
      pubtype: Academic Journal
      doctype: Article
      src: R
    language: English
    refInfo:
    copyright:
      @attributes:
        flag: Y
      custom: Copyright of Croatica et Slavica Iadertina is the property of University of Zadar, Department of Croatian & Slavic Studies and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use.
      item: Croatica et Slavica Iadertina
      holder: University of Zadar, Department of Croatian & Slavic Studies
      dt:
        @attributes:
          year: 2026
    holdings:
      @attributes:
        islocal: N