Inter-rater reliability

Discussion in 'tMP's .... "On the Spot" interview area' started by DublinBass, Feb 23, 2006.

  1. DublinBass

    DublinBass Supporting Member

    I have read your articles in the BB about criteria-based adjudication and (as a researcher in science education) I see one glaring part missing from having accurate, precise and reliable results...that being inter-rater reliability.

    Now it seems your presentation about a month ago at the ABBA AGM may have began to touch on this issue when all the adujidicators separately assessed a performance by Gawthorpe Brass '85 and then shared their results, but what mechanisms do you think should be in place to make sure adjudicators are consistent from rater to rater and then remain contistent within their own assessments over time.
  2. Ankanala

    Ankanala Member

    My articles in the band press which were based on my presentation to ABBA and related a whole series of mechanisms to try and standardise assessment.

    From the initial application requirements, through interview and acceptance. Then training and standardisation to in-service moderation and back up professional service to solve problems before the final resort of expulsion or enforced retirement.

    Much too long to repeat here in detail but it is all there in the articles and it has provoked a substantial response so far from several quarters.