A.B.B.A Criteria

Discussion in 'The Rehearsal Room' started by Columbo, Aug 7, 2009.

  1. Columbo

    Columbo Member

    Not a whinge at all this. Just want your thoughts as to the criteria for adjudicators written up in 2006 by Alan Morisson and endorsed by Derek Broadbent, where he states discretion is allowed (contradiction?). Personally, I agree with it as, in theory, it should help bands, although it does state the obvious.

    I also feel that adjudicators should have add an element of discretion so as to take away robotic and parrot fashion performances, however my concern is the points system (when required). It advises parameters then completes by stating discretion again, therefore, what is the point of it.

    One contest springs to mind where a ridiculous points system was used by the adjudicator. Preston 2008, 4th Section. Fair result, however a certain Miss Nicholson split every band by 5 points from top to bottom. How would you come to that conclusion? It made a mockery of the criteria set. Incidentally, and for another time maybe, she slated the bands that played the Area Test Piece in preparation (her opinion is fine), however it then became blatantly obvious, looking at the incipid remarks, that she had not bothered to adjudicate the 3 bands in question, who all incidentally, finished in the bottom 3.

    I also know of bands that have followed the criteria to the letter and basically come away with nothing after reasonable performances. This includes following tempo's to the letter also. Bands who had not worried so much about it have had results at the very same contests.

    My concern is merely that , why have the guidelines when it is blatantly apparent they are not being adhered to by even some of the best of adjudicators? Have they taken them on board or do we ignore them and carry on regardless?

    I would be interested to hear your comments. Like I said, debate only, not a whinge.
    Last edited: Aug 7, 2009
  2. Accidental

    Accidental Supporting Member

    As far as I'm aware they're only guidelines, not set criteria or rules.
    Unless contest organisers state they must be adhered to, the adjudicators are free to choose whatever method of judging and scoring they like, and personally I'd rather see it stay that way.
  3. Columbo

    Columbo Member

    I agree entirely with you and it begs the question why have it. Any conductor worth their salt already knows the criteria for a good performance. Worth noting Richard Evans comments on 4BR about conductors which ties in well with this point.
  4. mikelyons

    mikelyons Supporting Member

    I think it's OK to standardise criteria for marking for something like the areas and other national contests. This at least ensures that the different regions are on a more-or-less level playing field. I also don't think 'discretion' should be allowed to go too far on these contests. There should be, IMO a range of marks that can be awarded for each criterion and discretion is limited to a range of marks for each of those criteria. This is how exam marking works and there are meetings to check that everyone is 'singing from the same hymn sheet'. This is called moderation and is an essential part of the process. How else can we be sure that marking is fair and accurate?
  5. TheMusicMan

    TheMusicMan tMP Founder Staff Member

    Agree 100% with what has been said here by Mike.

    Moderation serves two purposes;

    1 - to ensure consistent marking is applied across the entire range of submissions
    2 - to deal with examiners/markers (by identifying training needs, by close monitoring, and ultimately by not asking them to come back as an examiner etc) who consistently apply poor marking
  6. mikelyons

    mikelyons Supporting Member

    You wouldn't be thiking of one person in particular here, John, by any chance? ;)
  7. TheMusicMan

    TheMusicMan tMP Founder Staff Member

    Hehe... no, not anyone in particular Mike. It's just that there is no such moderating process in Adjudicating, that's all.
  8. Columbo

    Columbo Member

    In light of my previous observations in regards to the ABBA assessment criteria and also recent rants, threads and comments in regards to the Open, adjudicators etc, does anyone agree that the 2006 criteria should be reviewed on a regular basis to suit particular choices of test pieces for the soming season? We have agreed previously, that some adjudicators have not followed the criteria to the letter (Not a critisism on my part), therefore, does it need to be reiterated or indeed updated? In my opinion, it either needs to be enforced or re published to make it clear what we (MD's) are to expect from established adjudicators and a reminder to new adjudicators in order for us to be clear on the amount of discretion expected. Having spoken to many MD's recently on this subject, it would certainly be a welcomed response from the ABBA to recent complaints and rants.
  9. floppymute

    floppymute Member

    Let's be practical here.
    We can discuss marking schemes and moderation till the cows come home, but to equate adjudicating with examination marking is a false comparison. Having done both personally, I can state quite clearly that they are two very different beasts. Why?
    The all encompassing difference is simple - time!
    Marking an exam paper you can take as much time as you like to come to a decision. In the box it is very different. You have to come to your conclusions very quickly. You have the time-span of the performance itself when you are juggling the need to write coherent and relevant comments with listening closely and fairly (try that for multi-tasking), then a matter of some five minutes or so between one performance and the next, and finally the relatively short span of time at the end of the contest when everyone is expectantly and, more to the point, impatiently, waiting for your results.
    This aspect of adjudication is an art in itself.
  10. RamasII

    RamasII Member

    Yes floppy, it is v difficult...you have to know the music well..and also be able to look at the whole score at once...being able to see errors in pitch and rhythm and praising great playing....personally I love doing it...and I believe I am very fair. Ive never had any complaints only people saying fair result etc..I do think it helps being, one involved with bands and two also doing other musical groups also..keeps the old grey matter working at its best!! haha
  11. DaveEb

    DaveEb Member

    Referring back to the point in the original post

    "she slated the bands that played the Area Test Piece in preparation (her opinion is fine), however it then became blatantly obvious, looking at the incipid remarks, that she had not bothered to adjudicate the 3 bands in question, who all incidentally, finished in the bottom 3."

    Now we all know Preston was 4 weeks before the area, can a fourth section band work up 2 pieces to the highest standard they can after the christmas break and a hectic run up to christmas with lots of concerts caroling etc. My answer would have to be that if a band is a genuine 4th or 3rd section band then one piece would suffer due to putting a priority onto one contest/piece.

    The options are:
    Just go and play your area piece knowing that you wont be considered for a prize(therefore it's not really open choice).
    Go and play a sub-par performance of another piece.
    Or not go at all.

    My opinion would be not go at all!!
    An adjudicator for me in the lower sections has a responsibilty to yes criticise(constructively), but also to help and encourage the bands taking part. As for points we are used to 1,2or sometimes 3 points being the difference in placings so why have a 5 point gap between each, was that reflective in how the adjudicator felt the performances differ.

    I think all we want is more information perhaps from adjudicators instead of what feels a little cloak and dagger at times. A man inside a box doesnt help this opinion either.
    Last edited: Sep 22, 2009
  12. Thirteen Ball

    Thirteen Ball Active Member

    So let's remove the necessity to write then.

    In america, australia and new zealand (And so far as I'm aware other places too) it is common practice for adjudicators to speak their comments and have them recorded along with the performance.

    That gives the adjudicator more chance to focus on listening - and we are designed to be able to speak and listen at the same time. Writing and listening is much harder.
  13. floppymute

    floppymute Member

    I quite agree - it would go some way to providing a solution.
    The main drawback I forsee is that some of the more 'cosy' venues, would find it difficult. It wouldn't help matters if the adjudicator could be heard during the performance making their comments. This would happen in some places nomatter how 'sotto voce' the judges were.
    Now that's OK if we accept different methods at different contests, but doesn't that open a whole new can of worms? Maybe, maybe not.
  14. Columbo

    Columbo Member

    I feel you have missed my point. It was not a critisism. We have to remember, that if its not necessary to write and then also to make a discretionary judgement, which I agree is difficult, then why put the criteria there in the first place. Clarification is all we are asking for. Maybe a percentage of criteria and percentage of discretion. If you read it, it contradicts itself anyway by stating the points required and then giving license to the man in the box, so what is the point of it?
  15. brasscrest

    brasscrest Active Member

    But it is impossible for the adjudicators to refer back to the recorded comments in order to score the bands or place them in order, because you have to play back each performance in its entirety. You still have to have some way for the adjudicator to mark or score the band during the performance so that they can be compared to the other bands. If you have no written notes, then only thing you would have is the adjudicator's memory.
  16. Columbo

    Columbo Member

    So.....Should there just be an update to clarify the Criteria expected and stick to it? There is no change to what the adjudicator expects, just a reassurance that they will follow the criteria we are training our bands to follow at the given contests.
  17. Anno Draconis

    Anno Draconis Well-Known Member

    The discretion part as I understand it mainly refers to the awarding of marks over and above the "maximum" limit:
    http://www.abbadjudicators.org.uk/ (my italics)

    However, I'm afraid I find the whole criteria thing to be incomprehensible. There are 14 PACs (Performance Assessment Criteria), and the titles are fairly self explanatory (although there is one called "Inspiration", which is a bit vague) but no indication of how they are weighted. If marking out of 200, should you allocate 14.28 marks to each PAC? Or do some considerations outweigh others? If so, which ones? Or have I misunderstood, and these are just a list of the things adjudicators should be listening out for, not directly linked to points available? In which case, I could probably have guessed most of them myself and I don't really see the point of the list.

    Also why do virtually all bands at any given contest score in the upper quartile - if you're marking out of 200, surely the "average" bands should be scoring around 100? Technically, if you're scoring to a consistent standard, 4th section bands should be regularly scoring below 40, because they are in the bottom 5th of UK bands overall.

    The simplest thing must surely be to allocate 20 points to each PAC and mark out of 280. Excellence (at the Cory/Dyke/Fodens level) would mean getting 18 or 19 out of 20 in each PAC, so a final score of 250 upwards. Average would mean getting 10-12 for each PAC, so a score of 140-160 ish. A band having a bad day would be getting 5-8 for each PAC so might only score 70-100.

    To avoid the demoralising effect of always giving lower section bands low scores, you could simply say that a PAC score of 10 represents adequately meeting the average expectations for the section - so a band that has an excellent day in the 4th section would still be scoring in excess of 200 because the scores reflect the expectations of playing in that particular section.
    There are, I believe, plans to run a joint ABBA/NABBC workshop on the Area Test Pieces (lower 3 sections) in January, in place of the usual NABBC workshop, although the exact date and venue are to be confirmed. I'm pretty sure it will be announced in the banding press and I'd encourage anyone who seeks greater clarity in the adjudication process to come along. If/when I find out more details I'll post them here.
  18. Columbo

    Columbo Member

    Cheers for that, I for one would be very interested in attending that..... and a constructive post also! Keep me in the loop please on that one.
  19. JR

    JR Member

  20. brassneck

    brassneck Active Member

    Has anyone seen Steven Mead's new blog about adjudication criteria? He's certainly sat down and had a good think about attitudes and I feel he has nailed them in relation to our system on the head. From reading tMPers' comments about change, I get the impression that clarity, consistency and fairness to all are the main objectives. I personally would like to see accountability added to that as well ... there are a few adjudication authorities that allow 360 degree feedback relating to judges used in events.

    To see how and why Steve breaks down his categories please read further at http://www.euphonium.net/home.php?newsID=121.
    Last edited: Sep 27, 2009