Judith Masson has just published her research on judicial feedback – it’s a very readable piece of research and has a lot of interesting nuggets within it.
Click to access FINAL_REPORT_pub.pdf
It looked at the sort of feedback that Judges get, or do not get, and the benefits of feedback and the drawbacks. I nearly headed this piece “The Loneliness of the Long-Distance Judger” as one of the real themes that kept coming across to me was how isolating a role it can be.
As one of the Judges points out, when he had been a barrister, there were really clear indicators as to the quality of the job he was doing – firstly, the solicitor would brief him again (or not), secondly the client would ask for him again (or not), thirdly, you’d tend to get some direct feedback in terms of comments or points from the solicitor (you did a really good job on this / the client felt you were a bit pushy / thanks for your really helpful note, or whatever) and finally you would get the outcome of the case. When the man became a Judge and felt a bit disconnected at knowing whether he was doing a good job or not, he spoke to his Designated Family Liaison Judge who told him “well, we’ve had no complaints, so you must be doing okay”
The types of feedback came into these categories – how is the Judge actually performing in the Court (in practice, this just comes from former colleagues at chambers over a drink saying “I hear that you’re interrupting quite a lot), are the decisions right (in practice just hearing about individual cases when they see the same lawyers and ask “How are things with little Anthony?” or seeing the case come back), overall statistical information (how often parents with heroin problems maintain abstinence, how long does it take a seven year old with sexual problems to be found an adoptive placement, placement breakdown rates etc – which are massively absent from the systems) and feedback from court users – the title of the piece is drawn from that section. If you assess the quality of a restaurant by what the people who ate the food thought of it, surely that must come into play when assessing the quality of anything else. If you don’t ask people who actually used the service, how can you assess what the service is like?
The actual feedback systems that are in place are these – Judges are made very well aware of how they are performing on PROCESS – how quickly they reach a decision, and they are made very well aware if they really mess up because of an APPEAL. But they don’t get provided with feedback about whether they get an individual decision right, or whether there is a trend of them getting a lot of stuff right or a lot of stuff wrong, or whether they are solid on say neglect, but a bit wobbly on physical harm or vice versa.
It’s a strange old system where the predominant focus is on getting it DONE, and telling everyone to get it done quickly, to get it done more quickly, to make the decision, but there’s no reflection process on whether the decisions made quickly were right or wrong.
The research is very interesting at just how split the samples of Judges were between those who really wanted to have feedback on their performance and felt that only by learning more about whether they were getting things right could they improve and those who were really resistant to it and felt that it had the potential to adversely affect outcomes in an individual case. Both of the groups make compelling arguments.
It also highlights that within the sample group, understanding about published research was very limited and that people’s perception of what research had to tell us about outcomes and likely outcomes was extremely negative (both in terms of what they felt they could learn from it and what they thought the research actually told them about how children do).
Judges received little or no feedback on individual stories, judicial performance or service compliance. What they received largely depended on happenstance and is more often positive than negative. These judges saw the current unsystematic approach as problematic,and most wanted to receive more feedback than they did currently. Most judges were uncertain how they could use feedback from their cases; some were anxious that it might have a negative effect on their decision-making. Views on the purpose and function of case feedback ranged from satisfying curiosity and their human needs to encouraging reflection and personal development, in effect increasing judicial wisdom.
Judges were generally unaware of the large body of aggregate data on the operation of the care system. They had only very limited knowledge of the care system generally and some held unrealistically negative views about the system.
It also identifies that whilst the HMCS systems gather a lot of data, they don’t necessarily gather the right sort.
The use of data for judicial and system feedback should not be limited to timeliness as it is at present but should cover substantive issues of decision-making, the orders made or refused etc. The data currently published by HMCTS indicates the numbers of orders of a articular type made each year. In terms of providing feedback on the operation of thesystem it is useless – it shows only throughput not the number and proportion of applications resulting in particular orders or other outcomes. It is not therefore possible to explore how this varies over time or between areas without the time-consuming process of collecting data manually.
The research uses a quotation from Thorpe LJ in the introduction
‘I often think that the judge receives far too little information as to developments after judgment both on a statistical plain and in relation to individual cases. Could we not gradually acquire experience and even wisdom derived from evidence as to how he solution for which we opted served the welfare of children.’ (Thorpe 1993, 326)
It feels quite amazing that twenty two years later, we still don’t have any systems to provide this. [I hesitate to suggest this, but I think the most accurate form of judicial feedback would be to put a bugging device in the nearest coffee shop to each Court and to listen in over the lunch adjournment. That is not a recommendation, by the way]
suesspiciousminds 
Pingback: You wouldn’t ask a restaurant critic to w...
Without being too partial – there is a huge disconnect and frustration at the mutual understanding between social work practice and the law. Social workers simply cannot understand the depth of ignorance of and disregard for research, statistics and outcomes displayed by the bench , Judges are frustrated by the lack of understanding of evidence and Human Rights issues by social workers
The focus seems to be on outcomes: was the decision the correct decision? With the assumption that it can only be a result test – if things turns out ok, limited it seems to whether or not the placement breaks down, then it was the correct decision. Surely it’s clear that this is a simplistic, binary approach?
What about whether or not witnesses and parties (services users) advocates perceive the tribunal to be knowledgeable, polite and fair? A party is in my experience much more likely to be able to accept and work with a decision if they believe that the court heard all parties politely and assessed them fairly and people’s views were respected. Thus itself for example could make the difference as to whether or not for a parent can come in time to accept placement and may then in turn affect whether or not post adoption contact takes place.
Tribunals appear sometimes to forget that the way they speak to an advocate, even the opposing party’s advocate, affects how the parties assess their fairness. They also forget that the social worker, children and parents have known each other a long time and will have to pick up the pieces and work with the fallout of the hearing for some time to come. The tribunal’s disposition has a great effect on this and paying compliments to one party and criticising another has implications of which they are sometimes only superficially aware.
Then there is the question of assessing whether or not a witness is honest and open. Too many lawyers, including judges, do not understand their own misconceptions about how they assess evidence, despite the great deal of psychological research on this. Even when this is known, it’s hard to catch ones own confirmation bias at work.
Finally, tribunals become battle weary and hardened and there is something to be said for sabbatical placements in a different region getting different, non-legal, experiences.
Yes, it seemed that the judicial sample was fairly divided on whether they wanted to know outcomes of cases or whether they only really wanted to know happy outcomes. But I’d agree that outcome feedback is just one part of the whole experience and that the other issues that you raise are possibly even more important.