Criteria Judging

Discussion in 'The Rehearsal Room' started by timbloke, Sep 8, 2010.

  1. timbloke

    timbloke Member

    I read the following with great interest.

    I'd obviously been asleep for the last few months as it was the first i'd heard of it - probably more because I wasn't that interested in the Open, hence I'd not read any articles or threads on it. However I felt it was probably worth having a separate thread to discuss the wider benefits of this method of judging and assist Steve in improving the system by getting feedback.

    Personally I think it is a brilliant idea and well presented. I've often thought that this form of judging would benefit providing a more objective and transparent adjudication. It would be interesting to see it trialled at most of the major contests (and some of the less major ones) over the next year to see where improvements could be made and to get feed back on the outcomes and the differences between this system and the traditional system. It could also help bands (particularly at the lower end) to understand which aspects of their performances are lacking and therefore where improvements can be made.

    One thing that could really bring brass bands into the 21st century is reporting the scoring live as it happens. You could see who is leading the contest during the day - would certainly add a different edge to competing.

    So now over to everyone else to discuss....
  2. Super Ph

    Super Ph Member

    Judging from his efforts it works no better or worse than the traditional approach - good bands get high marks and bad bands get low marks.

    What this system really lacks is any accounting for the piece played and its unique difficulties. Ultimately at a good contest the best bands are all going to play the right notes at the right time at the right dynamic, in tune, etc. The contest then comes down to (simplified example):
    band A played the loud fast bits faster, louder, and tighter, with a safe slow movement
    band B was more controlled but had a lovely slow movement and a good overall sound
    band C had one or two mistakes in the fast bits but some of it was awesome and powerful - they also had a lovely slow movement and took some interesting liberties with the score

    In this situation, the judging criteria are useless - it comes down to which version the judge enjoyed most.

    If you are going to try this type of system, I think many of the boxes need to be for "how impressive was the opening", "how well did the band play letter J", "did the end of the 2nd movement come off" etc. etc. i.e. specific to the piece.
    One box at the end for "was this band together, in tune, correct to the score, and good-sounding enough to win the Open" with answers yes/no/not even good enough to be here.
  3. timbloke

    timbloke Member

    More details on the findings of the adjudication seminar last week. Interesting to note Stephen's comments regarding the views and opinions of the ABBA members present. Can they not see that there is at least an advantage to considering other options for adjudication? Why must they persist in giving the impression that they will not budge?

    It is frustrating as a player, listner and (hopefully in the future) conductor, that someone somewhere at the top has decided they are happy with the status quo, and will not entertain listening to other people's views.

    Not sure what can be done, but my fear is that nothing will be done, and 10 years down the line, nothing will have changed (other than even more cheesed off bandsmen). You only have to look at the various comments on this forum to see that there is strong feeling for exploring alternative adjudication methods. Whilst it is not necessarily representative of the whole banding world, surely it shows that it is something that matters and should be considered.

    Well done to Mr Mead for starting the ball rolling, hope it doesn't hit a brick wall.
  4. WoodenFlugel

    WoodenFlugel Moderator Staff Member

    I have to say that article is terribly sad, but not terribly surprising. Perhaps ABBA is trying to cut a niche for itself as a modern-day Flat Earth Society? ;)
  5. Euphgit

    Euphgit Member

    Well obviously its going to do that, what criteria marking is supposed to do though, is provide transparency. This means that a band coming 3rd who thought they should have been first can see why they came 3rd. It tells them where they failed to impress the judges and thus tells them where they can improve their playing

    I disagree, any ''difficulties'' in any piece are going to fall under the criteria set. I.e. fast, quiet section would fall under Dynamic range, Ensemble precision, Pitch, balance etc

    Whilst top bands are going to make very few mistakes, there will still be a difference in quality between them (you pointed this out above, ''a few mistakes'' would impact the mark received in the category where the mistakes were made (pitch etc) . With the criteria system it gives the judges the opportunity seperate them, even if it's only one point. There is also a category called musicality which takes into account the adjudicators opinion of the music (worth double the other categories).

    No, again, all these would be covered by the criteria provided. You can't give a mark per bar of music. But you can take a general view of the performance

    I attended the seminar, and criteria based marking may not be the answer. There is however a problem and by far the most disappointing aspect of the seminar by miles was the attitude of the ABBA members present

    They were more interested in arguing about trivial spelling mistakes (in one case, a missing apostrophe, excusable considering the amount of typing that had been done, and it did not detract or confuse in anyway) and drawing the discussion around in circles

    There was a mini trial of the marking system (three quartets played) and the ABBA members refused to even try the new marking system. When confronted with the question ''could a new marking system improve audience attendance'' one member stated ''that's just the way it is, nothing we can do about it'' (paraphrased as I can't remember exactly how it was worded)

    As a relative newcomer to contesting this worries me (Only been contesting for 3.5 years, Been successful for the most part with Eccles though, so I'm not bitter about not winning). Lower section banding is where it begins for most people, and quite frankly the audience attendance is pathetic. And the experience of playing to an empty room is pretty soul destroying. It has to put off people from playing. And yet ABBA seem perfectly happy to flog away and take no action

    And thus endeth the rant :wink:
  6. DublinBass

    DublinBass Supporting Member

    I think the other issue that is trying to be solved is that the bottom half of bands get properly ranked (especially in contests like the areas, where promotion and relegation are based on placing)
  7. Steam Driven Cornet

    Steam Driven Cornet New Member

    You know, you have to love ABBA's attitude, basically clapping their hands over their ears and shouting to themselves, over what anyone else is saying, "because it has always been done our way it will always be done our way".

    Not sure how other's would describe this, but I consider it stubborn, childish, inflexible and, frankly, pathetic. They're doing no-one any favours.
  8. bariwizard

    bariwizard Member

    The real sadness here is that even with "criteria" based judging, the judgement of the criteria themselves will still be subjective, dependant on the view of a fallible human (in or out of a box). You can't make a science out of an art (thankfully), and at every major contest there will still only be one winner, with 19 losers left to judge the adjudication. It beggars belief that some of the great musicians involved in the campaign for reform can't look at the big picture and see that until the infallible judging machine / computer is invented we will have to accept subjective judgement in good faith - just like "proper" musicians outside of banding do when they lose in a competition.
    The real lack of progress here is shown by the so called "professional conductors" who still rail against a "bad" adjudication, insinuating/accusing ABBA members of being bent, deaf or incapable, much like the undignified scenes post contest in the late 18/early 1900's.
    It seems that the truly great and successful bands/MD's in our movement can accept a bad result as a glitch, knowing that in the medium - long term their quality will be rewarded with prizes. In the meantime some great players who just don't perhaps have what it takes to be great MD's continue to dress their childish tantrums, and refusal to accept their own limitations up as a campaign for reform. Still, all the best to Whitburn with their new (successful) MD, and Wire with their future endeavours!
  9. euphalogy

    euphalogy Member

    Great post!! if you cant win within the rules.......change the rules!
  10. WoodenFlugel

    WoodenFlugel Moderator Staff Member

    I've raised this before and its a misnomer to suggest a criteria based system will give a scientifically provable result. But what it does give is transparency as to why you were given the result you were - maybe not against any other band (because that's subjective), but as a stand-alone adjudication. This transparency should (in theory) cut down the level of confusion / whinging after a contest. In any case its a better system than the current one, and I'm yet to see anything else proposed that is equal or better.

    There are plenty of players willing to throw those accusations around as well. Just take a look at some of the contests threads here after the day of the contest - nearly every one has a 'we wuz robbed' comment in it. Actually in some way we're all guilty - isn't sticking the adjudicator in a closed box tantamount to saying to them 'we can't trust you not to put your favourite band first'?

    Oh and it was going so well wasn't it? :rolleyes: Who mentioned Whitburn and Wire? Was this whole post just an excuse to have another dig at messers Mead and Andrews?
  11. Euphgit

    Euphgit Member

    With an adjudicator who is up to the job of marking (Not saying that members of ABBA are incapable), that just isn't true. For example, pitch accuracy is something that can be quantified and there will be a difference from band to band that can be measured. Of course it isn't a science, but when you take music and push it into a contesting scenario you have to introduce a certain methodology into how you mark it, otherwise there is too much of an element of lottery about it, where bands have to guess what the adjudicator likes, more than play an interpretation that they feel is best#

    I would also like to mention that ABBA ALREADY HAVE a set of criteria laid down by which they mark. But the way they use them atm is vague, non-transparent and non-constructive. If you receive data back which tells you what needs to be improved then it would make bad results easier to swallow and would allow bands to work on areas that they didn't do well in

    Many other music competitions outside of banding use a criteria based, transparent marking system, so when they accept a poor result in good faith its because of objective, transparent adjudication

    The two people you are talking about maybe two of the leading members of the 'campaign' but they are not the only people who think that an improvement in the current system needs to be made. Attendance at the seminar proved that.
  12. hayleyb

    hayleyb New Member

    I don't ever find myself having the urge to get involved with anything on here however I do find this an interesting thread...

    Being at the seminar at RNCM on friday it was a really interesting debate with lots of pro's and con's on both sides, which I thought was handled very well.

    However one quote from someone is one that stuck with me..

    "No-one is ever going to be happy with the result unless they are given first place so if you don't like it don't come along, its just the way it is"...

    I found this a pretty shocking statement, and rather worrying to think that people had this attitude.

    Well why do bands even bother then if this is the case? If bands had this attitude there would be a severe lack in numbers entering im sure, and the whole thing would seem pointless. Do we really have to stick with the 'traditional method' just because 'that's the way it is'.

    So many times bands coming 4th,5th,6th,7th.. place are left in the dark as to why they were placed in the middle branch in awards. From experience too, when placed in these, the crit given to the band doesn't even sometimes contain any negative feedback. So should we just think, ohwell thats another contest;blame the adjudicators;comment on band favouritism or find other excuses?

    If ABBA or other individuals do not see the need for change perhaps they should ask the bands involved in contests what they would like, and im sure it would show some interesting feedback? I think is in bands' interest that something does change, lower section bands to build up and work their way through the ranks in time, and for higher top section bands to rid the doubt in adjudicators(blaming favouritism/cheating/choice of adjudicators) and provide a reasoning behind placings.

    As for attendance at the seminar, yes perhaps it was saddening that there were not any more people there to listen and have there say (friday afternoon, people working taken into account), maybe people in the brass band movement have just lost faith and do have the attitude 'thats just the way it is'.

    I am not suggesting one method or the other, as Friday showed there have been a few 'tried and tested' methods. However combining what we have now with a new criteria based judging system.

    I may have just blethered a load of rubbish, so apologies in advance!:)
  13. DublinBass

    DublinBass Supporting Member

    As soon as bands started having contesting that's exactly what they have they might as well do science right (or at least try to do science) the best that they can!!

    Exactly!! And right now they are using very poor methodology.

    "Ignorant" comes to mind.
  14. euphalogy

    euphalogy Member

    Bands shouldnt have to "guess" May i make a terribly radical suggestion, probably will be charged with treason.
    "Why dont the contest organisers ASK the appointed adjudicators to give an indication of what is required from the score" and to communicate this to the respective entrants.
    Tempo in relation to competence etc, time and space in tricky passages, stylistic lyricism and dynamic contrasts, etc.
    The criteria based adjudication would indeed produce a number of performances to slip conveniently into boxes, Gold, Silver, Bronze, perhaps, yet like other posts suggest, it wouldnt transparently produce an order of merit, as bands performances bring so many differing offerings of excellence or otherwise to the competition.

    i agree that ABBA dont necessarily cover themselves in glory by what may be perceived as a negative stance on the issue, however, i know them to be hard working and honest individuals whose passion for banding can not be questioned.
    Involve the adjudicator(s) and organisors and the bands in meaningful dialogue before the competition, as to what the adjudicators views of the score are................this is surely not it?
    We have some excellent adjudicators who would welcome this opportunity i believe. Conversley we have some lazy ones (as they were as MDs) who perhaps wouldnt.

    Tell em what ya going to tell em !! Tell em !! then tell em what you have told em!!
    Result = Understanding.
  15. floppymute

    floppymute Member

    A few people are bringing up this point and it puzzles me.
    Surely good musicianship leading to a good performance will always require all of these specified elements. Why does an adjudicator have to spell it out every time?
    It may well be that a certain piece might well hold a particular difficulty and the adjudicator's focus will likely fall on that as a natural course of events, but to any MD worth their salt that shouldn't be too hard to predict anyway.
  16. hayleyb

    hayleyb New Member

    I agree, but that was my point in asking bands for their opinion. Fair enough most adjudicators naturally do have this built in but I think even if it is just to provide evidence of their conclusions would prove to please quite a few more bands and I cannot see any harm in this, especially if it stops the mumbling that bands always has 'blamin the adjudicator' or the contest itself 'we never do well here' in this case they would know why they had specifically been given their result.
  17. StellaJohnson

    StellaJohnson Active Member

    Adjudicators are human after all. They only get to listen to your performance once and have to write so many points during this time without missing other elements of the permance. Then they have to listen to another 20 performances of the same piece. If you what to be so presise of every element and having to prove why you didn't play it to how he wanted it, then expect to be to at the contest till gone midnight. I personally have better things to do, especially when his decsion will be the same anyway :)
  18. hayleyb

    hayleyb New Member

    Just a question, but do you not think that having the criteria would make it easier for the adjudicator to sit back and listen? After trying Steven Meads method on 3 quartets last week I and so many others im sure found it far easier than what i'd imagine writing a full page of crit would be and you didn't neccessarily have to tick a box during the performance. I am not suggesting just using this as im actually quite keen to combine the two (where there are two adjudicators, one using this and the other writing the crit) but just don't think it should be so quickly ruled out.
  19. euphalogy

    euphalogy Member

    "i dont do predictions, and i never will" Kevin Keegan 1999 i think
    Its a case of subjective likes and dislikes, the TRANSPARENT adjudicators would surely welcome such a move, if only to minimise the "risk" of second guessing.
  20. StellaJohnson

    StellaJohnson Active Member

    I see why you would find it helpful for a quartet, you would need one for each movement I would imagine. Would you find it useful for if you were a band competing in the British open or championship national finals when all the bands were excellent.
    How would the criteria diffentiate fodens winning instead of Black Dyke because I think it would all be in the excellent criteria and they still wouldn't know what they have done to lose the contest?