Ranking, relevance judgment, and precision of information retrieval on children's queries: Evaluation of Google, Yahoo!, Bing, Yahoo! Kids, and ask Kids.
This study employed benchmarking and intellectual relevance judgment in evaluating Google, Yahoo!, Bing, Yahoo! Kids, and Ask Kids on 30 queries that children formulated to find information for specific tasks. Retrieved hits on given queries were benchmarked to Google's and Yahoo! Kids' top-five ran...
| Published in: | Journal of the American Society for Information Science & Technology Vol. 63; no. 9; pp. 1879 - 1897 |
|---|---|
| Main Author: | |
| Format: | research tables/charts Journal Article |
| Published: |
Wiley-Blackwell
Sep2012
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=104495007&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 104495007 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 15322882 IGD jtl: Journal of the American Society for Information Science & Technology issn: 15322882 maglogo: Y pubinfo: dt: Sep2012 vid: 63 iid: 9 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 104495007 79122273 10.1002/asi.22675 104495007 ppf: 1879 ppct: 18 formats: tig: atl: Ranking, relevance judgment, and precision of information retrieval on children's queries: Evaluation of Google, Yahoo!, Bing, Yahoo! Kids, and ask Kids. aug: au: Bilal, Dania affil: School of Information Sciences, University of Tennessee sug: subj: Internet Information Retrieval Information Seeking Behavior In Infancy and Childhood Web Search Engines Utilization Human Child Algorithms Conceptual Framework Benchmarking Child: 6-12 years ab: This study employed benchmarking and intellectual relevance judgment in evaluating Google, Yahoo!, Bing, Yahoo! Kids, and Ask Kids on 30 queries that children formulated to find information for specific tasks. Retrieved hits on given queries were benchmarked to Google's and Yahoo! Kids' top-five ranked hits retrieved. Relevancy of hits was judged on a graded scale; precision was calculated using the precision-at-ten metric ( P@10). Yahoo! and Bing produced a similar percentage in hit overlap with Google (nearly 30%), but differed in the ranking of hits. Ask Kids retrieved 11% in hit overlap with Google versus 3% by Yahoo! Kids. The engines retrieved 26 hits across query clusters that overlapped with Yahoo! Kids' top-five ranked hits. Precision ( P) that the engines produced across the queries was P = 0.48 for relevant hits, and P = 0.28 for partially relevant hits. Precision by Ask Kids was P = 0.44 for relevant hits versus P = 0.21 by Yahoo! Kids. Bing produced the highest total precision ( TP) of relevant hits ( TP = 0.86) across the queries, and Yahoo! Kids yielded the lowest ( TP = 0.47). Average precision ( AP) of relevant hits was AP = 0.56 by leading engines versus AP = 0.29 by small engines. In contrast, average precision of partially relevant hits was AP = 0.83 by small engines versus AP = 0.33 by leading engines. Average precision of relevant hits across the engines was highest on two-word queries and lowest on one-word queries. Google performed best on natural language queries; Bing did the same ( P = 0.69) on two-word queries. The findings have implications for search engine ranking algorithms, relevance theory, search engine design, research design, and information literacy. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|