(or “Raw Shark? Why would I be interested in Raw Shark?”)
A discussion about psychometric tests, and their clandestine nature.
I happened to stumble upon something which piqued my interest the other day.
Psychometric tests are standardised tests sometimes administered by psychologists and sometimes used within reports used for Court proceedings. The tests themselves are confidential, and I learned that the questions posed within those are not to be disclosed by anyone, including if asked for by a lawyer.
This, from the British Psychological Society guidance :-
Confidentiality and security of tests
Psychologists should be mindful at all times of the confidential nature of test materials. Many tests are invalidated by prior knowledge of the specific content of tests and their objectives. Psychologists who use tests are required to respect the confidentiality of test materials and to avoid release of test materials into the public domain (unless this is explicitly allowed in the nature of the test and by the test publisher). A court, though, can legally request test materials and such disclosure is allowed within the Data Protection Act. Psychologists, however, should take reasonable steps to prevent misuse of test data and materials by others. Misuse includes release of such data and materials to unqualified individuals (although see later with regard to legal release), which may result in harm to the client and/or release of data and materials without an adequate explanation with regard to how they are to be interpreted or used.
Legal scrutiny of tests
Some court proceedings are open to the general public and most are ultimately a matter of public record. In those cases where the practitioner has used standard test materials such as psychometric tests, he or she will need to be careful to ensure that all parties are aware of the possible dangers of discussing the content of standard test materials in open court. It may give rise to a leaking of confidential information and may put information into the public arena, which would damage the integrity of subsequent testing using those standard materials. Most courts, which are open to the public, will be sympathetic to a request that the details of such tests remain confidential or are restricted to a small group of participants in the case, allowing the practitioner to make reference to the tests in a general way, which will not affect their usefulness after the proceedings. Psychologists should not engage in detailed presentation and discussion of the content of test materials in open court.
This restriction may be less important in cases not routinely open to the public; nevertheless, it is still wise for the practitioner to guard the integrity of the materials in this way.
The practitioner should raise this matter with the solicitor who instructs him or her, or with the lead solicitor in the case of joint instructions to provide reports.
However, although these materials will be protected where appropriate by the court, practitioners can be expected to be examined, sometimes in detail, about the results of the tests and about all aspects of the report filed, including the interviews undertaken and any background to the conclusions reached. In answering questions of this nature it is likely that the court will require answers to be full and accurate and will be less concerned to guard what would otherwise be confidential material.
http://www.bps.org.uk/sites/default/files/documents/statement-_psychometric_evidence.pdf
I know that this is also how psychiatrists treat the Rorschach inkblot tests. You will have seen facsimiles of Rorschach tests in magazines or in films or TV, but you should never have seen an actual Rorschach inkblot unless you’re a psychiatrist or undertaking the test. The actual inkblot cards that they use are intended to be kept secret and can’t be reproduced.
(this is not a real one)
That makes sense, to be honest, because if you could look at the inkblot and see what various interpretations meant, you would avoid saying “that looks like a crab” if you knew in advance that crab = serial killer.
[Of course, the internet being the internet, you can see them with pretty minimal effort – the most obvious place you would go to find out about a subject has all of the card images and the broad interpretations. Don’t take a long time answering if the card has any red ink on it, is a top tip, if you find yourself in a Hollywood movie.]
And the same principle applies to psychometric testing. If you could say to your client in advance of the psychological session, “by the way, if they ask you which you would rather be, a crab or a clown, don’t say crab” it would invalidate the tests.
But here’s my interest. One of the psychometric tests that I see (and frankly, the only one that I’m interested in) deals with the validity of the other answers.
What they do, is amongst all of the other questions, is put some questions that anybody who was being honest would answer in a particular way.
For example (not a real one) : “Do you ever lose patience with other people sometimes?”
Now, of course, everyone in the real world says yes to that, of course we do. I lose patience at least nine times a day (more if I go shopping). So, that’s a control question to see if the person doing the test is being truthful, or trying to create a favourable impression. If the person says “no, I never lose patience” to that question, then they might well be trying to make the test come out in a favourable way, rather than answering all of the questions truthful. It’s something called ‘faking good’.
And it isn’t done on one question, but multiple ones, and they are probably not as transparent as that. (because that one alone, you might just be the Dalai Lama, although he doesn’t crop up that often in care proceedings)
Now, if you get a psychological report with a psychometric test which shows that the person being assessed was ‘faking good’ it undermines the rest of the test because the person is deliberately trying to appear better than they really are. And it is an alarm bell for not just the test, but makes you question what the assessment as a whole says, because if you know or believe that someone ‘faked good’ in the test, you start to disbelieve what they say in the narrative parts of the assessment, where they are having a conversation with the expert and giving answers. Were they ‘faking good’ in those bits too?
So, the consequence of your client having been identified as someone who was ‘faking good’ in a psychometric test is possibly serious. Even if the expert is careful not to say that the psychometric test can be relied on to show that the client is a liar, is that not still an impression that is left?
And when did the expert do the interviews? After the tests were known, and they thought there was a considerable chance that the client was ‘faking good’? did that colour the interview.
It does feel a little peculiar to me that if you wanted to challenge a psychologist on whether that ‘faking good’ indicator was robust, you wouldn’t be able to get under the bonnet at all, and see the questions or the answers.
For example, there is a debate about whether ‘faking good’ responses are a conscious or unconscious process, and therefore, seeing the specific questions and answers that led to the validity of the test being questioned might turn out to be very important. There might be something particular about that question for that client that meant that the answer was actually genuine, rather than an attempt at ‘impression management’
This is the final piece of guidance from the BPS, about what to do if the expert finds themselves in the position of a lawyer wanting to get under the bonnet, and not taking no for an answer
What action should be taken if a court or other legal entity requires the psychologist to release detailed test data?
Very rarely, a psychologist may be required by a court to disclose information regarding test materials or test data so that it comes into the public domain. If the psychologist believes that such disclosure may invalidate or damage the integrity of the test, then he or she should inform the court of the consequences of compliance. The psychologist should make the court and any relevant lawyer aware of the British Psychological Society’s policy concerning the security of test materials and the psychologist’s obligations under this and other ethical and professional codes, including the Society’s Code of Good Practice for Psychological Testing. In many cases, those concerned will be able to negotiate an accommodation which minimises the degree to which the psychologist’s professional standards are compromised by his/her overriding obligation to the court. An example of such an accommodation may be to allow a scoring sheet to be observed in court by the advocates but not for this to leave the court or for any copies to be made. Judges will generally be open to such compromises and will not seek to deliberately invalidate tests by allowing unguarded or full public disclosure.
When conflicts in reaching negotiated accommodation do arise, psychologists should identify the relevant issues, make known their commitment to relevant standards, and attempt to resolve them in a way that conforms both to professional practice and the law. In rare cases where, following such negotiations, a psychologist finds that the court’s demands have nevertheless compromised his/her position in relation to these issues, he or she should notify the Society’s Ethics Committee and the Steering Committee on Test Standards.
[I have to say that I have never actually seen a psychologist refuse to answer a question in order to preserve the sanctity of the tests, but that is what their professional body advises them to do. In fact, the only time I’ve ever seen anyone try to unpick the questions, I do recall one of the questions that was revealed (not a faking good one) being – well, in the light of this article, I can’t say, but save to say that there was intense debate outside Court amongst the advocates as to what the ‘right’ or ‘good’ answer to the question should have been. And not on partisan grounds, just on the basis that it felt a bit like a ‘and when did you stop beating your wife?’ question]
The first time I was exposed to the ‘and when did you stop beating your wife’ question was in a Bugs Bunny cartoon, and it made an impression on me as being simultaneously very clever and very unfair, so I guess I was always going to end up being a family lawyer.
suesspiciousminds 