RSS Feed

Tag Archives: expert assessment

who assesses the assessors?

Always nice to get a little Alan Moore / Juvenal nod into the title if you can.

The Family Justice Council report on the quality of expert psychologists used in care proceedings (as trailed on Channel 4 news) is up .

You can find it at http://www.uclan.ac.uk/news/files/FINALVERSIONFEB2012.pdf

They looked at 126 reports from 3 courts, and used four independent assessors to judge the quality of the reports, both against the guidance of the CPR and a piece of American caselaw (which I have to confess was unfamiliar with me until today) giving guidance on the construction of expert reports and their own views as to the quality of the report. They found, as you may have heard, that :-

 One fifth of instructed psychologists were not deemed qualified on the basis of their submitted Curriculum Vitae, even on the most basic of applied criteria.  Only around one tenth of instructed experts maintained clinical practice external to the provision of expert  witness  work.   Two thirds  of  the  reports reviewed were rated as “poor” or  “very poor”, with one third between good and excellent.

Without wishing to be unkind, my preliminary view is that they’d obviously got  a particularly strong batch. I have found most psychological reports to be a blend of regurgitation of information already found elsewhere, a statement of the bleeding obvious, recommendations plucked from thin air and if you’re particularly lucky a hefty dose of God Complex thrown into the mix.   [I would add, however, that if you get a really good psychological report, it sings, and makes the gulf in quality even more visible. I've got a few psychologists, who are always snowed under and have huge timescales, but always, without fail produce a report that adds something worthwhile to the process. Sadly, their numbers are dwarfed by the people who tell you very little, and take 160 pages to do it]

Here are some of the particular issues that the report considers have been problematic with psychological assessments : -

Research has identified a range of criticisms of psychological reports in general.  These  include occasions where:
Psychological evidence has been presented as scientific fact when in fact it is speculation and conjecture 

There has been an absence of psychological theory;


Evidence has been provided concerning concepts which are not accepted within the field and have not been demonstrated empirically.  At times this has had a negative
impact on the outcomes of proceedings (e.g. with one of the most heavily criticized concepts being that of „recovered memory‟)

There has been a failure to provide evidence which is outside the knowledge of the typical judge or juror  

Psychometric evidence has been submitted as scientific fact when it does not meet the criteria for this (e.g. Daubert criterion).  Rather the evidence  has represented
specialised knowledge at most, with some submitted psychometric evidence based on research and not clinical assessment tools

An over-use of psychometrics, not all of which are applicable to the case being assessed.  Over-use of jargon and speculation, with poor content and style and a
failure to include the data from where inferences are drawn 

The credibility of the source has not been included, with no attempt  made  to evaluate the reliability and validity of the methods used to collect data

Psychological risk assessments have focused on first and second generation approaches (e.g. unstructured clinical and actuarial) as opposed to the more reliable
and valid third generation approaches (structured clinical, with or without actuarial anchoring)

Allegations have been reported as facts

Emotive terms have been applied where these could prejudice a decision

They found that 29% of the reports provided insufficient facts and moved ahead to a conclusion. That 22% had significant missing data but still expressed a conclusion.

To illustrate examples concerning missing data, these are as follows:
- Reports on more than one child which failed to include the data on all children but still cited an opinion on all the children;
- Reports drawing conclusions which have not been mentioned in the report, as noted by one reviewer: “Indicates in conclusion that any individuals assessing this
client should be knowledgeable of Aspergers type characteristics and the impactof this on parenting.  This was never mentioned in the report, or assessed, and
appeared as the last sentence” [rater comment].
- Reports where opinions are presented where data was completely absent, i.e.  “Comments on self-esteem, emotional loneliness, perspective taking, sexual risk,
but include no data” [rater comment].
- Reports where the data is completely missed, “Does not include fact section  –goes straight to opinion” or “cites psychometrics but no scores” [rater comment].17
- Report citing opinion without conducting a formal assessment, “stated that client presented as being of average intelligence without deficits in comprehension or
expression, formal intelligence testing was not undertaken” [rater comment].  
Further examples were:  “he seemed, at times, to be quite a jumpy person with arousal levels higher than an average baseline.  No assessment completed of this”
and “did not assess for personality and yet draws opinion on it”.
- Refers to the opinion of another as their opinion, “Refers to someone else‟s report in response to an instructed question” [rater comment].

Ouch.

They then considered the conclusions against the main body of the report  (a particular bugbear of mine, since if you can’t tell why the conclusions have been reached, how is any professional supposed to explain to their respective client why the expert is with them or against them, and whether they should shift their own position?)
Specific background missing/unclear (1). 34.0 %
Limited opinion (2). 17.0 %
Opinion confused or not clearly explained (3) 17.0 %
No background, just opinion (4) 9.4 %
Some opinions, not linked to factors  (4) 9.4 %
Opinions not substantiated (6) 7.5 %
Questions not answered (7). 3.8 %
No opinion (8). 1.9 %

Okay, the “no opinion” at all has a pretty low score, but that probably still represents from that pool five families who waited for three or four months for a psychologist to help decisions about their future to be made and who got nothing more than an expensive Scooby Doo report  (shrug of shoulders, “I-dunno”)

They found that 60% of the reports had missed the requirements of the CPR for an expert report.

They give some examples of the expert straying into areas reserved for the Judge (I point this out, because in general I agree with the report, but I think the example given here is quite badly flawed and rather weakens some of the other criticisms  –  “I am of the view that these children have all suffered significant harm”   – the ultimate decision on that is of course for the Judge, but there are many, many times when an opinion from the expert as to that is helpful, and generally it is provided as an answer to one of the questions. That, I think highlights the difference between the reports commissioned under the CPR for civil matters and for children matters – the expert is there to help the Court with specialised expertise rather than as a ‘gun for hire’ as happens/happened in civil cases. )

But the report isn’t just a woe-is-me hatchet job, it does go on to make some recommendations. They are worth reading in full, but these are the ones that I considered to be very important

 That instruction of experts should be restricted to those currently engaged in practice which is not solely limited to the provision of court reports.  Only
approximately one tenth of the instructed experts were engaged in practice outside of court work.  This is not in keeping with the expectation of an “expert” as a
senior professional engaged in current practice, suggesting that courts are accessing those whose profession is now solely as an “expert witness”.   There
should be an expectation that  psychologists providing court reports should continue to hold contracts with relevant health, government or educational bodies
(e.g. NHS, Private Health, Prison Service, Local Authority etc) or demonstrate  continued practice within the areas that they are  assessing (e.g. treatment
provision).   This is a means of ensuring they remain up to date in their practice, are engaging in work  other  than assessment, and are receiving supervision for
their wider work as psychologists.  Connected to this, courts should be wary of experts claiming to complete excessive amounts of independent expert work.

 That the instruction is clearly for the expert to conduct all aspects of the work and not graduate psychologists or assistants.  Such individuals are not qualified with
the term „graduate psychologist‟ used to describe those who have completed approximately one third of the required training (e.g. an undergraduate degree in
psychology and nothing more).  There was evidence of their over-use by experts,who were relying on them in some instances to review collateral information and
interview clients.  Courts should only be paying for the expert witness to complete all aspects of the report


Care should be taken with the use of psychometrics and these should not unduly  influence final judgments.   The current research indicated a wide range of such
assessments being used and not all relevant or up to date.  If tests are utilised then experts should be providing  courts with sufficient information to allow them to
judge their quality.  Using the Daubert criteria as a reference for this would assist with the quality of this information (e.g. provision of error rates, evidence of the
theory or method the test was based on), and assist courts to judge how it should be admitted as evidence.

A need for psychologists to provide provisional opinion and alternative opinions.  

The data from which opinions are drawn needs to be clearly indicated to the court.

The use of tested and/or generally accepted psychological theory to support core findings.    Courts are paying for  psychological assessments and this should be
evidenced to distinguish the opinions from those provided by other disciplines

(Hallelujah to that last one.)

The report doesn’t really get into the other side of the coin, which is – are we asking psychologists routinely to assess parents when it is not the right sort of assessment? When I started, psychological assessments were confined to cases where there was some unusual feature or behaviour and the professionals simply couldn’t understand fully and called in a psychologist to advise on that aspect  (I would add that the professionals at that time would have generally been a social worker very skilled and experienced at assessing families rather than a ‘commissioner of assessments’ and an old-school guardian whose role was to dig into the LA work with the family and see if things ought to have been, or could have been, done differently).  Now, a psychological assessment is routinely considered in neglect cases, where common sense tells everyone concerned that the problems are either motivation, lack of comprehension of what is needed to run a family in a non-chaotic way, or exposure as a child to poor parenting and thus no internal models of how to parent.

We go to psychologists when a social work assessment is what is needed. It is one of my main bugbears with both the Family Justice Review and the LSC cost-caps, that the ISW reports which are independent, swift, cost-effective and actually genuinely informative are sneered at and undermined and costs slashed to the point of extinction, whereas the bloated and we see often of varied benefit escape that exercise.

Rant over !

Respect my prior authoriteh !

 

“I guess one person can make a difference… but most of the time, they probably shouldn’t”   – Marge Simpson

 

I would be very interested to know if this is a local problem, or more widespread, but I’ve had a spate over the last five months (getting steadily worse) of cases being delayed and my email being clogged full of problems about Prior Authority.  This tension seems to have arisen because the LSC appear to intepret a Court order that says “The costs of this expert be shared in equal one quarter shares between the Local Authority and the public funding certificates of the mother, father and Child” to actually mean “The costs be split one quarter to the LA, who have to pay up and shut up, whatever we feel like we want to pay, and the rest out of the solicitors profit costs – providing of course that we think the assessment should actually happen at all”  and “the report to be filed and served by 1st April 2012″  to mean “The expert report will be filed at some indeterminate time in the future, after we’ve processed prior authorities, granted one of them, rejected one of them, and refused one, then reconsidered on appeal”

 

 

If that’s sounding familiar, I have a suggested order, and a generic skeleton below, which I have been using in a concerted effort to educate the LSC that in Court proceedings, it is the Court who decide what reports take place, and who pays for them. Hint – the clue is in the wording of the initial order, and the omission of the words “Whatever we feel like we want to pay and the rest out of the solicitors profit costs”

 

Please let me know of problems or solutions in your area. It will all be helpful should the LSC decide to challenge the Court’s jurisdiction on costs.

 

Order :-

The Court orders that the costs of the assessment be met in equal one quarter shares between the Local Authority and the public funding certificates of the mother, father and Child/ren, it being a reasonable and proportionate disbursement for the purposes of public funding, and the Court having determined that the report is necessary for the resolution of the case.  In the event that the Legal Services Commission, who adminster the public funding certificates and payments made, seek to vary or set aside this order, such application should be made on notice to the parties, no later than                (2 weeks time).  If no such application has been made by that date, this order shall stand. The publicly funded parties shall serve both the sealed order, and a typed version of this order (to avoid delay in waiting for the sealed order) upon the branch of the LSC dealing with their certificate, forthwith.

 

Skeleton

Case No: 

IN THE                                  COURT

 

IN THE MATTER OF

 

AND IN THE MATTER OF THE CHILDREN ACT 1989

 

B E T W E E N:

Applicant

-and-

 

1st Respondent

-and-

 

 

2nd Respondent

-and-

 

 

(by his/her/their Guardian)

3rd Respondent

 

_____________________________

Skeleton argument

Prepared by the Local Authority

______________________________

 

 

Brief background

 

 

Proceedings in relation to                                            were commenced on                          .  [Information re dates of birth of the children, who the parents are, where the children are living and under what orders]

 

The concerns in the case relate to                                           as set out in the threshold document [page reference].

 

 

 

 

On [date] , the Court made the following direction relating to the instruction of an expert:-

 

 

 

 

Certain of the publicly funded parties made an application to the Legal Services Commission (hereafter LSC) for “Prior Authority”  – that is, agreement in advance of receipt of the invoice from the expert that the LSC would honour that payment.

 

Obtaining “Prior Authority” from the LSC is not a required element of the solicitors firms contract with the LSC, but many firms, locally and nationally, take the cautious and not unreasonable view that they would wish to ensure that the LSC will pay any costs incurred, as if they do not, the firm themselves are left paying any shortfall, thus taking a financial loss on dealing with the case.

 

The Local Authority would emphasise that they have sympathy and understanding for the solicitors firms involved, who have to operate in a financial climate where making up the shortfall between what an expert charges and what the LSC pays towards that expert fees can mean a Mr Micawber-esque outcome :- “Annual income twenty pounds, annual expenditure nineteen nineteen six, result happiness. Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.”

 

 

The “Prior Authority” mechanism, whereby the solicitors firms seek reassurance from the LSC that their allotted share of the expert fees will be recouped in full, in advance of the expert incurring any fees (by commencing the work which has been directed), is sadly not flexible, fluid or swift enough for such results to be known in good time for the expert to undertake the work and hit the deadlines imposed by the Court. In many cases, the process is taking a period of months, rather than weeks, leading to significant delays in the expert commencing the work, and hence the report being available when directed. This in turn, leads to delays in the Court being able to resolve decisions for children.

 

 

 

The Local Authority stance is that the Court have ordered, legitimately and lawfully, that an expert report be commissioned, and ordered, legitimately and lawfully that the costs of that report be apportioned in a certain way. If the LSC now resist that legitimate and lawful order, they should seek to apply to vary or discharge it.

 

It is suggested that to clarify this position in future, it should be made explicit on the face of the order that if the LSC seek to vary or discharge the order as to the apportionment of costs, they do so within 14 days of the order being made, and that the publicly funded parties shall file and serve the order (or a typed note thereof) upon the branch of the LSC dealing with their particular certificate.

 

This then avoids the need for any application for Prior Authority, as the Court will have ordered how the costs are to be paid, and the LSC will have their opportunity to challenge that within timescales which are more suitable for the child, and the administration of justice.

 

 

 

 

Notwithstanding the legitimate desire of the LSC to manage their budget and to drive down the costs of expert assessment, the Local Authority submit that where this causes delay for the child, the system has not worked properly.

 

 

 

The law

 

 

Section 38(6) of the Children Act 1989 gives the Court the power to order that assessments be conducted within care proceedings.

 

That this power extended to directing how the assessments were to be paid for derives from a number of authorities, notably

 

CALDERDALE METROPOLITAN BOROUGH COUNCIL V (1) S (2) LEGAL SERVICES COMMISSION (2004)

 

[2004] EWHC 2529 (Fam)

 

In which the High Court determined that the Court had jurisdiction to order that the costs of obtaining an assessment be divided in whatever way it saw fit, including making provision  (as in this case) that the Local Authority pay one quarter, and each of the three publicly funded parties pay their own one quarter share through their public funding certificate.

 

It will be noted that the LSC played an active role within that case.

 

The principles in Calderdale were revisited in

 

LAMBETH LBC v S (2005)

 

[2005] EWHC 776 (Fam)

Fam Div (Ryder J) 03/05/2005

 

Where the High Court determined that funding of section 38(6) assessments was not outside the remit of the LSC, and importantly that the Commissions own guidance on funding was not binding on the Court.

 

 

Some extracts from that judgment which are pertinent to the issue here (and given that it was made nearly seven years ago, prescient)  :-

 

Paragraph 43 : – “It is equally correct that the Community Legal Service Fund has fixed and limited resources but so do local authorities… the services they both provide are inextricably linked to the obligation on the Court to ensure within the Court’s process the exploration rather than the exclusion of expert assessment and opinion that might negate the State’s case for the permanent removal of a child from his parents

 

Paragraph 62 : -  “ There is already a healthy delegation of the Commission’s powers and duties to the parties legal advisors. That practice of delegation was very properly exercised on the facts of this case and as a matter of practice around the country great care is taken by publicly funded practitioners to abide by their duties. A paper review of a case by the Commission is in any event a poor substitute for the Court’s overall impression gained by its continuous case management”

 

Paragraph 63 “It is a matter for them (the LSC) to put in place guidance to deal with exceptional expense provided that any prior authority or notification systems do not cause delay”

 

 

 

 

 

 

 

 

 

The Court do have the power, under Rule 25.4 (4) of the Family Procedure Rules 2010 , set out below, to limit the amount of an experts fee and the expenses that may be recovered from any other party.  There is nothing within that power to circumscribe HOW the Court may limit the amount, and certainly nothing to indicate that they are bound by the LSC’s own internal policy or guidance.

 

Court’s power to restrict expert evidence

25.4.—(1) No party may call an expert or put in evidence an expert’s report without the court’s permission.

(2) When parties apply for permission they must identify—

(a) the field in which the expert evidence is required; and

(b) where practicable, the name of the proposed expert.

(3) If permission is granted it will be in relation only to the expert named or the field identified under paragraph(2).

(4) The court may limit the amount of a party’s expert’s fees and expenses that may be recovered from any other party

 

 

The Court must consider, in any application to vary or discharge the original order :-

 

Section 1 (1) of the Children Act 1989  “when a Court determines any question with respect to (a) the upbringing of the child; the child’s welfare shall be the court’s paramount consideration.

 

And section (1) (2) of the Children Act 1989 “in any proceedings in which any question with respect to the upbringing of a child arises, the court shall have regard to the general principle that any delay in determining the question is likely to prejudice the welfare of the child”

 

The paramount consideration is the child’s welfare, and that delay is likely to be prejudicial to that welfare; rather than the financial aspects (important as they legitimately are to both the LSC and the firms involved)

 

 

It is submitted as a result of all that has preceded,  that :-

 

(a)    the Court has power to direct that an assessment take place (pace s38(6) of the Children Act 1989)

(b)   the Court has power to direct that the costs of the assessment be apportioned in such way as they see fit, including directing that the parties public funding certificates bear all or some of the costs  (pace Calderdale)

(c)    The LSC own internal policy on funding, and the limits they will pay in relation to experts is not binding on the Court (pace Lambeth)

(d)   The Court does have the power to set a cost limitation when instructing an expert, and also when considering any application to vary the original order.  (pace rule 25.4 (4) of the Family Procedure Rules 2010)

(e)    If the consequences of setting a cost limit and varying the existing order, mean that a fresh assessment be commissioned, or significant delay incurred, the Court cannot make that variation without considering the provisions of section 1 (1) and section 1 (2) of the Children Act 1989

 

And that

 

(f)    the interests of the child would be better served by the report which is so close to completion being completed and filed and served, as originally intended, and for the existing order to remain in place, with no cost cap being added.

 

 

The Local Authority would accept that in some cases where the LSC actively seek to become involved and make representations, that the balance might well fall another way, and that the LSC’s perfectly legitimate motivation in controlling costs and curbing what had been excesses might justify the Court setting a cap pursuant to rule 25.4 (4) of the FPR.

 

In this case, however, it is not. Decisions here need to be made about this child/these children, and what the appropriate arrangements for his/her/their family life should be.

 

In general, the Local Authority would suggest that where Prior Authority is  refused, then there is a need for the case to be urgently restored for directions, to consider whether the original direction needs to be varied, and the impact on the timetable generally.  The Local Authority would remark that a great deal of their time is currently spent on wrangling with decisions in relation to Prior Authority and whether expert assessments which have been directed by the Court can take place, and many of these disputes have led to delay for the children concerned.

 

 

 

Follow

Get every new post delivered to your Inbox.

Join 2,038 other followers