3 in a box (split from Black Dyke thread)

Thirteen Ball

Active Member
For what it's worth, my two-penneth.

The only reason the results at the masters were so seemingly contradictory was that none of the adjudicators approached each performance from the same perspective. It's all very easy to say that because (if memory serves) one band were placed first, first and thirteenth by the three men in question, that the system "doesn't work" - but shouldn't that very fact be telling us something about the bewildering differences between the processes each adjudicator uses to assess performances? Surely it's not the separate boxes system that "doesn't work" - it's the adjudication system we already have.

I have oft been heard to bemoan the lack of identifiable prerequisites for a good contest performance, although I have long accepted that these are never going to become a part of the adjudication process, because so much of what makes a performance "A Winner" in the view of most of our most-utilised adjudicators appears to be unquantifiable.

Likewise I've previously suggested a division of the points allocated to a band between 'technical merit' and 'artistic interpretation' the same as is done with Ice dance or rhythmic gymnastics, in order to counteract the effect we often experience at contests, where an error-strewn performance which contains precisely the interpretation the adjudicator wants can handsomely beat a clean performance which differs from his/her taste and opinion. Unfortunately I also can't see this taking off as ABBA appear resistant to any predefined criteria being implemented, despite propsals from the likes of Alan Morrison.

I think a large part of what the banding community at large (ie: players, conductors, supporters etc) would like is a greater clarity as regards what constitutes a 'good' performance. Because no criteria are laid down beforehand it's all guesswork - and without seeing everyone else's comments in the same section, it's impossible in the aftermath to build up any picture of what it was prevented any particular performance from being a winner.

The unwillingness of certain adjudicators to clarify, or even to discuss the reasons for their decisions has already been well documented here previously (sometimes resulting in threats of legal actions) so I shall not dwell on it further - suffice to say that it is not all adjudicators. I have personally read some wonderful, helpful and constructive comments from adjudicators, suggesting what could have been done to improve the performance and why we were placed where we were. One or two have also been amenable enough to clarify their opinions in person - and those individuals who refuse to discuss things over a pint afterwards, and become agressive when the reasons for their decisions are questioned would be well advised that they will catch more flies with honey than vinegar....

I don't wish to be uncharitable, but it's eminently possible, under the current adjudication system that an adjudicator could decide to imitate the contrary, wheelchair-bound character from Little Britain, and simply cry 'don't like it,' potentially placing a band in a relagation position for no other reason than that. Ok, because of the way results and comments are handed out, I've no evidence to prove that this has happened - but these same circumstances also make it damnably difficult to prove that it hasn't.

When we live ina world where the very survival of bands can depend on promotion and relegation decisions, is it too much to ask that we at least know the criteria the adjudicator(s) is/are looking for before the pre-results speeches?
 
The Scottish Brass Band Association (SBBA) has announced that the adjudicators at this year's Scottish Open Championships, to be held at Perth Concert Hall on Saturday 20 November, will sit in the open. Thought to be the first time since the demise of the National Mineworkers' Championships in Blackpool that a major test-piece contest in the UK has opted for open adjudication, the event will see a panel of three judges "working separately", with their combined placings determining the final result.

A new Test piece (-- AND WHEN THE RIVER TOLD--) so no recording CD ( Goodoh !) should therefore make intresting listening i.e. interpretation. and the composer one of the adjudicators.

No doubt there will be complaints regarding his remarks i.e. 'What does he know anyway?'

Bravo SBBA for having the guts to try this.
 

yoda

Member
I think three separate adjudicators is a good idea for several reasons.

1) As already mentioned, a strong personality can dominate the cause. If you are going to have three adjudicators get three opinions...if you want just one opinion, only use one adjudicator.
2) Three adjudicators getting three different winners is probably a better representation of actual opinions of the performance. Not everybody in the audience fancies the same band...why should we expect adjudicators to?
3) The transparency will make it easier to identify (and then get rid of) "bad" adjudicators (That is adjudicators that don't conform to the general opinion.
Trouble was, the band that were declared winners that day (and I do not decry nor devalue their win, after all a win is a win) I believe that not of the 3 had them close to winning (a am happy to be corrected if the old grey matter is suffering a little)

however, you make a very valid 3rd point. again the only problem i see is that non of the present crop of adjudicators would be happy to judge under these conditions knowing they could look silly if they didnt just happen to agree with each other and worse never get work again.

a single adjudicator can never be wrong. ;-)

The only reason the results at the masters were so seemingly contradictory was that none of the adjudicators approached each performance from the same perspective. It's all very easy to say that because (if memory serves) one band were placed first, first and thirteenth by the three men in question, that the system "doesn't work" - but shouldn't that very fact be telling us something about the bewildering differences between the processes each adjudicator uses to assess performances? Surely it's not the separate boxes system that "doesn't work" - it's the adjudication system we already have.

I have oft been heard to bemoan the lack of identifiable prerequisites for a good contest performance, although I have long accepted that these are never going to become a part of the adjudication process, because so much of what makes a performance "A Winner" in the view of most of our most-utilised adjudicators appears to be unquantifiable.

Likewise I've previously suggested a division of the points allocated to a band between 'technical merit' and 'artistic interpretation' the same as is done with Ice dance or rhythmic gymnastics, in order to counteract the effect we often experience at contests, where an error-strewn performance which contains precisely the interpretation the adjudicator wants can handsomely beat a clean performance which differs from his/her taste and opinion. Unfortunately I also can't see this taking off as ABBA appear resistant to any predefined criteria being implemented, despite propsals from the likes of Alan Morrison.

I think a large part of what the banding community at large (ie: players, conductors, supporters etc) would like is a greater clarity as regards what constitutes a 'good' performance. Because no criteria are laid down beforehand it's all guesswork - and without seeing everyone else's comments in the same section, it's impossible in the aftermath to build up any picture of what it was prevented any particular performance from being a winner.

The unwillingness of certain adjudicators to clarify, or even to discuss the reasons for their decisions has already been well documented here previously (sometimes resulting in threats of legal actions) so I shall not dwell on it further - suffice to say that it is not all adjudicators. I have personally read some wonderful, helpful and constructive comments from adjudicators, suggesting what could have been done to improve the performance and why we were placed where we were. One or two have also been amenable enough to clarify their opinions in person - and those individuals who refuse to discuss things over a pint afterwards, and become agressive when the reasons for their decisions are questioned would be well advised that they will catch more flies with honey than vinegar....

I don't wish to be uncharitable, but it's eminently possible, under the current adjudication system that an adjudicator could decide to imitate the contrary, wheelchair-bound character from Little Britain, and simply cry 'don't like it,' potentially placing a band in a relagation position for no other reason than that. Ok, because of the way results and comments are handed out, I've no evidence to prove that this has happened - but these same circumstances also make it damnably difficult to prove that it hasn't.

When we live ina world where the very survival of bands can depend on promotion and relegation decisions, is it too much to ask that we at least know the criteria the adjudicator(s) is/are looking for before the pre-results speeches?
An excellent post Andy.

I agree with many of your points, especially the one about knowing what the adjudicator is looking for. Maybe an open letter to all competing bands a month before the contest...? just saying hi and i will be looking for this or that. Some do this at the draw, but i usually find its a little late by then ;-)
 
i guess the cambridge experiment shows the vagueries of adjudication. so what if it showed the results were diverse? bands could say they did well under adjudicator x or y. for the cost of a couple of tents i'd imagine it would be worth another try.

the problem as i would see it, is that adjudicators might be tempted to not score high a brave original performance. it might make things more boring as bands find middle ground.

still, worth a try. no big reason not to try.
 

SMead

Member
I know a mod has asked us to leave this, but i couldnt without saying the following........

There are lots of people with very short memories clearly. This was Tried (at the request of competing bands) at the Cambridge Masters a number of years ago.

The up shot was a total farce with 3 VERY different sets of results. when amalgamated they did indeed produce a winner, but one put one band in the top 3 and another put them in the bottom 3...... :)

I expect then that this "experiment" wont be happening again in a hurry because no one want to look silly. That said, each gave an honest assessment based on what they heard. Just goes to show that it is always a personal opinion system (and there is nothing wrong with that as it is the best we have come up with over many years of mucking about with it). It was very funny tho, and IMHO invalidates somewhat the point of having 2 or more even in the same box, as there will usually be differences of opinion (I know as I have been a part of this). The differences may be negotiated (as has always been the case when i have been involved) or (and I stress this is NOT my experience) the "senior" will make the call.

Lets get back to just one, as any more than one just muddies the waters

:)
So, can we put this one to bed now...the debacle that was the results of the Masters a few years ago, as described above was that one adjudicator, clearly not used to the new system of marking, gave what he considered the best band the most number of points, which was in fact the complete upside result of what was intended, ie the best band gets one point, not 18 !!! so, had this been spotted by the contest controller and queried this myth of a failed system wouldn't have been perpetuated,and the result would have been unanimous. Please trust me on this one ! The truth needs to be revealed, finally.
 

DublinBass

Supporting Member
again the only problem i see is that non of the present crop of adjudicators would be happy to judge under these conditions knowing they could look silly if they didnt just happen to agree with each other and worse never get work again.
I think much of this fear could be fixed by using a more consistent and reliable rubric for adjudicating (Alan Morrison suggested one several years back, and Steven Mead more recently).

I agree with many of your points, especially the one about knowing what the adjudicator is looking for. Maybe an open letter to all competing bands a month before the contest...?
I think the contest should be telling both the adjudicators and bands what they want to have looked for. I've attached (or at least tried to) the judging criteria for the US Open that is sent out to the bands. Additionally, I believe the adjudicators have a meeting to make sure they are on the same page.

Separate scores and their tallies are distributed to the contesting bands (to help with transparency). Did all the adjudicators agree on every placing this year. Of course not. But I certainly accept the result more because of the transparency, their audio comments and their comments on the rubric.
 

Attachments

RDTCBD

Member
I took a youth marching band to the European Concert and Marching Band Championships a few years ago. In the concert element it was similar to brass in concert where the band were required to undertake a 25 minute programme.
They had 7 judges each marking out of 100 and giving feedback comments. The top and bottom marks out of the 7 were then discarded and then an average of the other 5 judges scores gave a final mark out of 100. This stopped any rogue or inconsistant judges affecting a result. They all sat in an open boxed area and to me, seemed a very fair way of acheiving a result.
 

SMead

Member
For what it's worth, my two-penneth.

The only reason the results at the masters were so seemingly contradictory was that none of the adjudicators approached each performance from the same perspective. It's all very easy to say that because (if memory serves) one band were placed first, first and thirteenth by the three men in question, that the system "doesn't work" - but shouldn't that very fact be telling us something about the bewildering differences between the processes each adjudicator uses to assess performances? Surely it's not the separate boxes system that "doesn't work" - it's the adjudication system we already have.

I have oft been heard to bemoan the lack of identifiable prerequisites for a good contest performance, although I have long accepted that these are never going to become a part of the adjudication process, because so much of what makes a performance "A Winner" in the view of most of our most-utilised adjudicators appears to be unquantifiable.

Likewise I've previously suggested a division of the points allocated to a band between 'technical merit' and 'artistic interpretation' the same as is done with Ice dance or rhythmic gymnastics, in order to counteract the effect we often experience at contests, where an error-strewn performance which contains precisely the interpretation the adjudicator wants can handsomely beat a clean performance which differs from his/her taste and opinion. Unfortunately I also can't see this taking off as ABBA appear resistant to any predefined criteria being implemented, despite propsals from the likes of Alan Morrison.

I think a large part of what the banding community at large (ie: players, conductors, supporters etc) would like is a greater clarity as regards what constitutes a 'good' performance. Because no criteria are laid down beforehand it's all guesswork - and without seeing everyone else's comments in the same section, it's impossible in the aftermath to build up any picture of what it was prevented any particular performance from being a winner.

The unwillingness of certain adjudicators to clarify, or even to discuss the reasons for their decisions has already been well documented here previously (sometimes resulting in threats of legal actions) so I shall not dwell on it further - suffice to say that it is not all adjudicators. I have personally read some wonderful, helpful and constructive comments from adjudicators, suggesting what could have been done to improve the performance and why we were placed where we were. One or two have also been amenable enough to clarify their opinions in person - and those individuals who refuse to discuss things over a pint afterwards, and become agressive when the reasons for their decisions are questioned would be well advised that they will catch more flies with honey than vinegar....

I don't wish to be uncharitable, but it's eminently possible, under the current adjudication system that an adjudicator could decide to imitate the contrary, wheelchair-bound character from Little Britain, and simply cry 'don't like it,' potentially placing a band in a relagation position for no other reason than that. Ok, because of the way results and comments are handed out, I've no evidence to prove that this has happened - but these same circumstances also make it damnably difficult to prove that it hasn't.

When we live ina world where the very survival of bands can depend on promotion and relegation decisions, is it too much to ask that we at least know the criteria the adjudicator(s) is/are looking for before the pre-results speeches?
A totally outstanding post, bravo Andy, lets hope more people voice opinions that will benefit bands and contests in the future.:clap:
I'm trying to get people to see the logic in this criteria based judging and if one reads ABBA's attempted demolition of these ideas on their website today (see The Chairman's Page) , there is still a long way to go !!
I never said my experiment at the Open was the definitive article, but to a man (and woman) the ABBA Executive Committee have passed 'judgement' on it, without any discussion on it with me whatsoever. Not a word . And they couldn't even wait for my own full report on the system. They clearly dont like debate and all those involved with this article are signing up to 'more of the same' that the band movement enjoys at the moment. What a pity.
 

brassneck

Active Member
It might be just me, but has anyone else detected a change in audience response to what constitutes a good performance? I feel that the powerhouse days of blowing is coming to an end with bands playing more with lyricism and better sound quality. Fairey is a good example of this change and starting to reap the rewards. Thing is, you rarely hear in public this being mirrored by adjudicators.
 

Thirteen Ball

Active Member
A totally outstanding post, bravo Andy, lets hope more people voice opinions that will benefit bands and contests in the future.:clap:
Crikey.... Thanks!

I'm trying to get people to see the logic in this criteria based judging and if one reads ABBA's attempted demolition of these ideas on their website today (see The Chairman's Page) , there is still a long way to go !!
I never said my experiment at the Open was the definitive article, but to a man (and woman) the ABBA Executive Committee have passed 'judgement' on it, without any discussion on it with me whatsoever. Not a word . And they couldn't even wait for my own full report on the system. They clearly dont like debate and all those involved with this article are signing up to 'more of the same' that the band movement enjoys at the moment. What a pity.
As my old grandad used to tell me, "If you always do what you've always done, you'll always get what you always got." I guess that sums up what I was trying to say in my previous post. That the pursuits which have endured the longest in the modern world have had to adapt to changing societied in order to do so.

Look at Rugby league's resurgance in recent years. From the preserve of beer-reeking social clubs, to a multi-million pound global sports venture, all through embracing new practices. Some of these practices have been pretty extreme (the change from a winter sport to a summer sport, for example) but were bold enough to pay off. Likewise the introduction of video technology has made their referees accountable, and provided a valuable resource to aid them with their decisions, and a salary cap has prevented the players from becoming too far removed from their supporting crowds - in stark contrast to the extremes of salary we've seen in football.

But here's the rub. The reason that Rugby managed to make these changes is because it had a governing body of forward-looking people who are willing to take tough decisions to ensure that their chosen pursuit survived. It had reached the point where their sport had to adapt or die, and in order to rekindle interest in it, a lot was risked. But at least they had the mechanism in place to make those choices. And we as a movement don't really appear to.

There's no unified approach in banding. So much happens because "that's the way it's always happened" that it's all but impossible to take a step back and ask if there's a better way we can do things, quite simply because there's no-one to ask! And this for me is the whole problem with the adjudication system. Who can we suggest things to. ABBA? They've already stated that they're happy with the status quo - and without their support, what can the rest of banding (not forgetting that non-adjudicators are hugely in the majority of people associated with bands) do about changing it? Quite simply, nothing. There's not even a mechanism where we can even consult our national network of bands, to find out if they really are happy with the current system!

In fact, this website and those like it are probably the closest to an official forum that banding at large has. Imperfect as it is, at least it gives us the ability to exchange opinions. But without a more official framework for consulting the movement as a whole on how we go forward into this young century in order to ensure there are still meaningful contests at the end of it, I don't see how banding at large can progress.
 

Red Elvis

Active Member
A totally outstanding post, bravo Andy, lets hope more people voice opinions that will benefit bands and contests in the future.:clap:
I'm trying to get people to see the logic in this criteria based judging and if one reads ABBA's attempted demolition of these ideas on their website today (see The Chairman's Page) , there is still a long way to go !!
I never said my experiment at the Open was the definitive article, but to a man (and woman) the ABBA Executive Committee have passed 'judgement' on it, without any discussion on it with me whatsoever. Not a word . And they couldn't even wait for my own full report on the system. They clearly dont like debate and all those involved with this article are signing up to 'more of the same' that the band movement enjoys at the moment. What a pity.
Just had a read of that statement on the Chairman's page on the ABBA site . Its an "interesting" read that does come across as seeming to pull up the drawbridge and retreating behind the castle walls.

The point is made in the document that ABBA need to be more vocal in supporting their adjudicators and I'm all for that but surely its got to be a two way street. If someone with the credentials and experience of Mr Mead can't get a hearing or debate on his suggestions then there's not much hope for the rest of us is there ? Guess this ties in with Andi's excellent point about the lack of a central body representing us to facilitate any engagement with.

That said , I do have a lot of time for Mr Lippeatt ( never met him or played with or under him , nor to my recollection been judged by him either ) if only because of what he's done with the Butlins contest. From what I've read / heard from him , he does have an eye on the future of the movement and as such the views he expresses on the ABBA site do strike me as somewhat defensive.

Also an interesting view taken on the policing or lack thereof on muting of parts - the view seeming to be that its not the adjudicator's role to judge on this and it's down to other competing bands to object !!

What I'd like to see ( and perhaps Butlins would be the right place to do it ) is some form of panel discussion / debate - Mr Mead and Mr Lippeatt together with a few other suitable personalities that could address some of these points in front of an audience. Questions could be submitted in writing beforehand ( to stop any debate descending into a free-for-all ) . Any takers for that ?
 

Owen S

Member
The only reason the results at the masters were so seemingly contradictory was that none of the adjudicators approached each performance from the same perspective. It's all very easy to say that because (if memory serves) one band were placed first, first and thirteenth by the three men in question, that the system "doesn't work" - but shouldn't that very fact be telling us something about the bewildering differences between the processes each adjudicator uses to assess performances? Surely it's not the separate boxes system that "doesn't work" - it's the adjudication system we already have.
Completely agree, Andi. What separating the adjudicators does is to show exactly how subjective the current adjudication system is. Contest adjudication can never be completely objective, since musicality is inherently subjective, and since each adjudicator ranks the importance of each one of the judging criteria (musical interpretation, tuning, splits, dynamics etc.) differently. However, bands can better prepare for that subjectivity if they know the criteria they are going to be judged by, whether that be a standardised adjudication scheme, or by the adjudicators providing some basic guidelines in advance. That means something like a month in advance, not at the draw!
 

Owen S

Member
I've just read through the ABBA comments, here: http://www.satiche.org.uk/abba/ahts.pdf

They're pretty interesting, and include some very good points. I find the comments about closed adjudication more persuasive than I thought I would, and David Read's piece is well written, though it does have a very conservative tone.

However, a few of the comments elsewhere simply don't logically follow, or don't understand the arguments. There are two examples that stand out:

Firstly, on policing "mute abuse", Stan Lippeat says:
To use a “fair play” adjudicator would also be difficult; nobody could keep their eyes on all players, listen intently to the performance and make a note of these misdemeanors. So therefore even if the adjudicators were in the open the problem would not be solved. Don’t forget they are also writing a full set of comprehensive remarks about the performance.
Except a "fair play adjudicator" would be only there to police mute abuse, and maybe part swapping, both things that only really happen in quiet and exposed passages, when there is little going on and plenty of time to watch. Can the use of a "fair play adjudicator" who does nothing but check this through a whole day's contest really be justified though? That's a different question.
With a “fair play” adjudicator it would be a case of “did he see it” and would it be “word against word”.
"The adjudicators word is final". I think most bands are comfortable with that, aren't we, even if we like to complain about it in the bar afterwards?


Secondly, on the experiment of fixed adjudication criteria, the question posed are bizarrely wide of the mark, particularly given that the article also makes several fair criticisms of the system:
Using Mr Mead’s results of this years British Open I ask these questions
1 when was the last time that the British Open Champions were marked as low as 191
2 when was the last time that the band finishing in last position were marked as low as 153
3 did Mr Mead and the 4barsrest team have Tredegar winning
4 did Mr Mead and the 4barsrest team write comprehensive remarks for all performances
5 did they use a score during their deliberations
6 did their final result compare favorably with the official (international) adjudication team
1&2: The marks a band receives are nowadays based on ABBA's suggested ranges of points, but these numbers are ultimately pulled out of thin air and mean nothing of themselves, something that is inherent in any judging system that is qualitative. Replace the current marking system with a new one, and 191 or 153 are entirely reasonable marks. The question is phrased very strangely too, in a way that implies doing something because "that's the way we've always done it". If you're introducing a new system, matching history isn't important in itself.
3&6: These questions smack of hubris, and simply assume that the actual adjudicators must be right. If question 3 is meant to question whether Stephen Mead's results unfairly favoured the big name bands, then it should have said that. As for question 6, how exactly are you supposed to "compare favourably" to someone else's subjective ranking, and according to whom?
4&5: These are reasonable questions, though for at least question 5, I suspect the answer is "yes".
 

MoominDave

Well-Known Member
Thanks for the link. Interesting reading! I too find the arguments in favour of closed adjudication more persuasive than I would have imagined.

Yes, the holding up of the fact that criteria-based marking gives a wider range of marks than people are used to is just
bizarre. If it's really that much of a big deal, simply scale the marks - say so that 200 stays 200, 190 becomes 195, 180 becomes 190, 160 becomes 180, 100 becomes 150, etc. Very easily done (it's simply mark/2+100), and completely removes this objection - which they seem to put forward as if they see it to be the clinching point.

Another quote struck me: "If it ain't broke, don't refix it!". I suspect many people would beg to differ on the degree of brokenness in the current system.
 
Last edited:

Thirteen Ball

Active Member
Another quote struck me: "If it ain't broke, don't refix it!". I suspect many people would beg to differ on the degree of brokenness in the current system.
The very fact we're having this debate suggests you're dead right, Dave.

All this talk of points in Abba's article seems to me to be a smokescreen - because the points themselves aren't linked to any criteria. If points were allocated for a performance and THEN placings decided, fair enough, but they aren't. What happens is that places are allocated, and the adjudicator(s) then allocate points to differentiate how closely the bands compared, in their opinion. We all know that if you win by four points you were a class apart, three points it was a walkover, two points it was impressive, and one point it was close, but that's only because that's what we're used to. The actual numbers of points allocated are immaterial, when the placing is what's used to decide them.

Who cares if the winners received 200 points, 30 points or 3000 points, the proposal was to change the criteria that the winners are decided by to more objective measures. And surely that is the crux of the matter - an issue which, dare I say it, ABBA have attempted at times to sidestep.

(Might it also be worth pointing out that bands don't even get points at Area Championships any more, so it's not like they're that important.)
 

Thirteen Ball

Active Member
PS - The article does put forward a number of reasons I'd not thought of for closed adjudication. Whilst I agree that mute abuse does need stopping, it's probably right that an adjudicator has a lot more to focus on than that. Plus there's no requirement for how a band should sit onstage, so it's very probable that even in the event of open adjudication, bands would probably just arrange their seating and stands so as to hide the muted players from the adjudicators.
 

MoominDave

Well-Known Member
It doesn't make any mention of the acoustic properties of transmitting a performance through the walls of a box, something I consider fairly key. I've heard some properly nasty over-ripe blowing given mindbogglingly good placings before; the most obvious explanation of that to me is that a material barrier takes the edge off the sound a bit.

It's a defence of closed adjudication, rather than a balanced consideration of closed vs open.
 

Aidan

Active Member
just read that abba pdf... it could have just as well been dated 1980 and read the same.
They really don't do themselves any favours at all, so much inwards/backwards/anywaybutforwards-looking. Even (with an exception or two) the very well selected series of commentors all of whom just so happen to agree with the chairs views.
I especially enjoyed the comments along the lines of "if it ain't broke don't fix it"
Aren't they the only ones who think it isn't broke? People come along with genuine ideas and aspirations and are immediately poo-poo'd by a crowd of fuddy-duddys banding together in self-defence of their impregnable holier than thou procrastinating system.
It's not a shock to see that the europeans are now looking to.... europeans for their inspiration is it. Don't let british banding go to the dogs along with the rest of the country.
 
Top