What’s in a name (2): domains, scales, scores, factors & dimensions

The original commissioning specification for the CORE system required that the items in the measure covered domains of wellbeing (or “well being”, or “well-being”: there’s another naming issue!), problems/symptoms, functioning and risk.  The questions were supposed, where possible to include intrapsychic ones and interpersonal ones and functioning was to cover both more personal/intimate and more social functioning and risk should cover intrapunitive and extrapunitive risk, i.e. risk to self and risk to others.  We liked this framework and noted that the first three domains had some links to the phase model of change in therapy which suggests that well-being change comes first, then symptom/problem improvement, then functioning improvement (Howard, Lueger, Maling & Martinovich (1993) A phase model of psychotherapy outcome: causal mediation of change. J Consult Clin Psychol. 61(4):678-85).

We thought the commission specification was right that these were fairly conceptually distinct domains of experience that should be covered by a measure of change in therapy and that was supported by extensive surveys of therapists/practitioners, managers, commissioners (“purchasers” in the jargon of the time) and end users and lay people and we thought we should say which items we saw as belonging most strongly to which domains and offer the opportunity to study scores, and changes in scores, on each domain.  However, we never imagined that these would form clear “factors” or principal components in cross-sectional psychometric studies nor that the chronological relationships between them over time in cohorts or even within a single person in therapy would be neat.  If you feel lousy (low wellbeing) it’s likely that you will have or develop problems and even symptoms, and vice versa.  Similarly, struggling to function well either in personal interactions and/or at work or in caring duties will dent a sense of wellbeing and lead to problems: these simply aren’t independent factors or dimensions. 

With the advantage of hindsight it’s easy to see that we should have been clearer about that.   We tried to use the terms “domains” and “domain scores” in preference to “factors”, “dimensions”, “scales” but slipped from time to time.  We thought we were sufficiently explicit about our use of exploratory factor analysis being exactly that: exploratory, and mainly to check that there was a large main factor and a good collection of smaller factors.  We were unsurprised in our early work (Evans, Connell, Barkham, Margison, McGrath, Mellor-Clark & Audin (2002) Towards a standardised brief outcome measure: psychometric properties and utility of the CORE-OM. British Journal of Psychiatry, 180, 51–60) to find a structure that didn’t reflect the domains but which seemed to some extent to separate positively cued from negatively cued items and to separate the risk items from the other items.  We have never expected that this structure would replicate strongly in different cultures and samples and we only used confirmatory factor analysis to show just how poor the fit to a simple factor structure (Lyne, Barrett, Evans & Barkham (2006) Dimensions of variation on the CORE-OM. British Journal of Clinical Psychology, 45, 185–203).  That paper was intended to be a definitive statement about the expected psychometric structure, at least in British clinical samples.  Here’s the statement from the abstract:

The CORE-OM has a complex factor structure and may be best scored as 2 scales for risk and psychological distress. The distinct measurement of psychological problems and functioning is problematic, partly because many patients receiving out-patient psychological therapies and counselling services function relatively well in comparison with patients receiving general psychiatric services. In addition, a clear distinction between self-report scales for these variables is overshadowed by their common variance with a general factor for psychological distress.

And the end of the discussion:

These considerations with respect to the CORE-OM domains are of importance for future research and scale development, but the utility of CORE-OM has already been demonstrated as a widely used benchmarking measure and reliable indicator of change in psychotherapy research and practice. The scoring method that has proved most
useful in this regard is that in which all 28 non-risk items are scored as one scale and the
risk items as the other. This research confirms that the scale quality of CORE-OM when
scored in this way is satisfactory.

So some suggestions/pleas:

  1. by all means report change on specific domain scores if they are pertinent for the work that went on with the client/patient but don’t imply that the specific scores are well defined factor analytic scales;
  2. the risk and non-risk items are sufficiently distinct in cross-sectional psychometric studies that it may be wise to report the non-risk and risk scores as well as the total scores in almost any study;
  3. if you possibly can, talk about the scores from the CORE-OM and CORE-SF/A and SF/B as “domain scores” not “dimensions” or “factors”.

Putting a CORE measure into software

Since the first of January (2015) anyone can put any CORE measure into software under the terms of the Creative Commons Attribution-NoDerivatives 4.0 International licence.  Previously, we had restricted this permission just to people using the measure in software just for a research project and otherwise only to CORE Information Management Systems (CORE IMS) and people sublicensing through them.  See our joint statement for more information.

The Creative Commons licence puts two restrictions on anyone putting the measures in software: they must acknowledge the origins of the measure and they mustn’t change the measures.  This is in line with the situation for paper reproduction of the measures which has always been on these “copyleft” terms.  However, both the acknowledgement and the “no changes” were easy for reproduction on paper: just print out the PDF and the acknowledgement is there in the copyright statement at the foot of every page and any printing to any sensible printer from PDF guaranteed no meaningful changes.

The situation is not so clear when a measure is put into software and we’re gaining experience, and remembering some of the early learning we did with CIMS some years back.

Acknowledgement is fairly easy: if the line “Copyright to CST: http://www.coresystemtrust.org.uk/copyright.pdf” that is in the PDF versions is there, and if ideally the link is clickable and opens to that URL, then you have done the minimum we ask by way of acknowledgement.

No changes is  a bit more complicated as obviously the formatting is pretty well bound to change depending on the device on which the text appears.  Where the intention is for end users (patients, clients, service users, research or survey participants) to use the measure in the software the following must be true:

  1. The introduction (“IMPORTANT – PLEASE READ THIS FIRST This form has 34 statements about …” should be unchanged in content but the line “Please use a dark pen (not pencil) and tick clearly within the boxes.” can be replaced with whatever is sensible to tell the user how to fill in the items.
  2. The time frame (“Over the last week”) must come before the items and at the head of each block of items.  If the items are presented oen at a time, this must be there with each item.  We recommend if the items are presented sequentially that the system offer an option to see the introductory instructions (#1 above) with every page.
  3. The response anchors (“Not at all” to “Most or all the time”) must be unchanged and against each response option.
  4. Clearly the “Please turn over” instruction from the CORE-OM can be dropped but something similar should be used to try to ensure that people do page through all the items.
  5. The system does not have to calculate the overall score and present it to the end user.  If it does, it must do so correctly, see #6.
  6. Scoring is simple and the “clinical scoring” of 10x the the mean item score is used for all measures.  Perhaps counterintuitively the “well being” domain score is scored in the same direction as the other domains, i.e. higher scores indicate lower well being.  Prorating can be applied for any score provided that fewer than 10% of the items on the score have been omitted.  Clearly for scores with fewer than ten items this means that no prorating of missing items is supported.  It is the responsibility of whoever is putting the measure into software to ensure that the right items are mapped to the right scores, that “positively cued” items (e.g. “I have been happy with the things I have done” have their scores included correctly (i.e. scoring 4,3,2,1,0 rather than 0,1,2,3,4 for the “problem cued” items) and that scoring is correct.
  7. We are happy to review any implementation with you if we have time for this. Currently, if this is not particularly urgent, we do not charge for this but we may have to change that if the work load becomes significant.
  8. We could provide sample data to be entered for all the measures that could be used to check scoring and can help with that but that is a significant piece of work for which we would have to charge and sample data are under development currently.  The responsibilty will still remain with whoever programs the scoring.

This is all work in development and we will turn this into an FAQ as experience develops but experience so far has been that complying with the licence conditions and checking that is not onerous for the programmers/designers nor for us to check.

Do get in touch with us if you would like us to work with you on this or if you are confident you have done it fine and just want us to know you are making the measures available with computer support.

Best wishes,

Chris Evans (for CST)