'I'm pickin' up good regressions': the governance of generalisability analyses.
Context Investigators applying generalisability theory to educational research and evaluation have sometimes done so poorly. The main difficulties have related to: inadequate or non-random sampling of effects, dealing with naturalistic data, and interpreting and presenting variance components. Metho...
| Publicado en: | Medical Education Vol. 41; no. 10; pp. 926 - 935 |
|---|---|
| Autores principales: | , , , , , , |
| Formato: | research Journal Article |
| Publicado: |
Wiley-Blackwell
Oct2007
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=106010842&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 106010842 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 03080110 ESF jtl: Medical Education issn: 03080110 maglogo: Y pubinfo: dt: Oct2007 vid: 41 iid: 10 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 106010842 106010842 2009687983 10.1111/j.1365-2923.2007.02843.x NLM17908111 106010842 ppf: 926 ppct: 9 formats: fmt: @attributes: type: P tig: atl: 'I'm pickin' up good regressions': the governance of generalisability analyses. aug: au: Crossley J Russell J Jolly B Ricketts C Roberts C Schuwirth L Norcini J sug: subj: Education, Medical Standards Research, Medical Analysis of Variance Data Collection Study Design Human ab: Context Investigators applying generalisability theory to educational research and evaluation have sometimes done so poorly. The main difficulties have related to: inadequate or non-random sampling of effects, dealing with naturalistic data, and interpreting and presenting variance components. Methods This paper addresses these areas of difficulty, and articulates an informal consensus amongst medical educators from Europe, Australia and the USA, who are familiar with generalisability theory. Results We make the following recommendations. * Ensure that all relevant factors are sampled, and that the sampling meets the theory's assumption that the conditions represent a random and representative sample of the factor's 'universe'. Research evaluations will require large samples of each factor if they are to generalise adequately. * Where feasible, conduct 2 separate studies (pilot and evaluation, or Generalisability and Decision studies). * For unbalanced data, use either urgenova, or 1 of the procedures minimum norm quadratic unbiased estimator, (minque), maximum likelihood (ml) or restricted maximum likelihood (reml) in spss or sas if the data are too complex. * State which mathematical procedure was used and the degrees of freedom (d.f.) of the effect estimates. If the procedure does not report d.f., re-analyse with type III sum of squares anova (anova ss III) and report these d.f. * Describe and justify the regression model used. * Present the raw variance components. Describe the effects that they represent in plain, non-statistical language. * If standard error of measurement (SEM) or Reliability coefficients are presented, give the equations used to calculate them. * Make sure that the method of reporting reliability (precision or discrimination) is appropriate to the purpose of the assessment. This will usually demand a precision indicator such as SEM. Consider a graphical presentation to combine precision and discrimination. pubtype: Academic Journal doctype: research Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|