Lara Schwicht with Dr. Jeffrey Lee

Background

This study focuses on assessment procedures used to evaluate the treatments offered to severely and persistently mentally ill patients (SPMI). By way of definition, SPMI individuals: (1) number between one and five million (NAMHC, 1995; Arns, & Linney, 1993), (2) use a significant percentage of mental health care inpatient and community services ($148 billion annually in the United States), and (3) are diagnosed with mental disorders such as “schizophrenia, schizo-affective disorder, bipolar disorder, major depression, autism, and obsessive compulsive disorder” (Carey, & Carey, 1999). The size of this patient population coupled with its financial impact on health care in North America underscore the importance of research efforts aimed at refining assessment attempts to identify successful and unsuccessful treatments.
SPMI populations are a very difficult patient population to assess with respect to treatment success. Self-report measures have distinct limitations with the SPMI population given the cognitive and reality impairment of patients. This has led to the use of rating scales completed by independent observers. The most widely used “other-rated” scale to assess SPMI outcomes is the Brief Psychiatric Rating Scale (BPRS). However, this instrument requires significant clinician resources to produce reliable data (Burlingame, et al., under review). In short, it takes 20 hours to train a mental health professional in this protocol and an additional ½ day every 3-month to avoid rater drift (Burlingame et al., 2002).

The resources to support this type of assessment protocol are rare in public mental health settings—the primary setting treating SPMI patients. However, the BYU-Utah State Hospital (USH) partnership has developed such an outcome assessment protocol. This assessment protocol has received national recognition as best practice (Payne, Earnshaw & Burlingame, 2003). The objectives of this outcome assessment protocol were recently published in a special issue focusing on “best practices” (Burlingame, et al., 2002). Moreover, the BYU-USH partnership was supported by a state grant (Burlingame & Wells, 2000) to collect and examine BPRS data using sensitivity to change analytics; item, subscale & total score item slopes. The results of this study suggest that 22 of 24 BPRS items are differentially sensitive to change. However, none of the items or subscales is differentially predictive of patient or diagnostic characteristics, e.g., chronicity of disorder (Burlingame, et al, under review). Given that the BPRS is sensitive to change across a spectrum of SPMI diagnoses, we have suggested that normative and change indices are the next logical step.

Despite its widespread use with the SPMI population, the BPRS literature has not yet provided clinicians with several straightforward psychometric indices. No attempt has been made to provide normative values for this public domain instrument. In the absence of these values, interpretation of scores becomes less meaningful to clinicians and administrators alike. In addition, while the sensitivity to change data are available (Burlingame, et al., in preparation b), no published study to date has proffered change indices; a recent yet invaluable addition to the outcome literature. The reliable change index (RCI) provides a single value indicating the amount of change needed on a scale to label patient change as greater than chance variability (cf. Burlingame, et al., 2001).

In addition, the literature is void of any empirically based community normal scores. According to the literature it is the assumption that community normals will score a “two” or less on any individual symptom (Ventura, et al., 1993). However, it is also true that within any given community there may be, clinically speaking, normals who score more than a two on some of the individual symptoms (e.g. anxiety, guilt). Therefore, collection of community normal data is important because we may uncover endorsement of minor psychopathology by normals. Uncovering this data would clarify distinctions between community normals and inpatient psychopathology. This assumption and lack of empirically supported community normal BPRS scores presents a lack of validity for the instrument; which, in completion of our project, will likely rectify inaccurate ratings of SMPI patients and strengthen its validity.

Methods

The community normals sample, consisting of 100 individuals, will be drawn from three sources: college students at Brigham Young University, Utah State Hospital staff, and community neighborhood volunteers. Fifty of the individuals will be BYU students, while 50 will consist of individuals from the other two populations. Each person will be asked to complete a BPRS interview lasting approximately thirty minutes. Exclusion criteria will be participation in treatment for any psychological disorder in the last six months, those under the age of eighteen, those who are pregnant and those who have been pregnant in the last six months, and those who are asked to participate but decline of their own free will.

The community population will consist of an equal sex ratio, unmatched for age and education. It is, however, requisite that all participants be over age eighteen. Anonymity will be maintained throughout the experiment by assigning numbers and no names to all BPRS data.

Two statistical means will be reported for a global BPRS score, each subscale comprising the measure, and every item on the BPRS.

Current status

Our project was focused on establishing a cut-score for inpatient and community normals. Due to the semi-structured interview nature of the BPRS 20 hours of training was necessary to calibrate the accuracy of our assessments with the “gold standard” developed by Dr. Ventura at UCLA. We accomplished this by sending one graduate student, in the Masters of Psychology program, and three undergraduates, majoring in Psychology, to USH’s quarterly BPRS training program. These students were familiarized with inpatient symptoms and trained on how to interact with SPMI patients. These students then recruited a convenient sample of 53 BYU undergraduate psychology students and 52 Utah State Hospital staff members as a representation of community normals to administer the BPRS to.

The project is currently in data analysis phase in which average total scores, subscale scores, and individual item ranges will be analyzed on both the inpatient and community normal assessments. These scores will then be compared using a T-test analysis. Upon completion of this project and the outpatient project cut-scores and an RCI will be established.

We have applied to the Society for Psychotherapy Research (SPR) conference that will be held in June of 2005 in Montreal to present this project in conjunction with a current continuation of this project (establishing similar indicies with the SMPI outpatient population). The SPR conference is a premier division of the American Psychological Association which is focused on psychotherapeutic treatment assessment and improvement.

1. Indicate where you presented or published your project. SPR 2005
2. About how many hours did you spend with your mentor over the course of the project? on average 2 hours per week
3. About how many hours did you spend working on your project outside of the time spent with your mentor? On average 4 hours per week
4. What was the best thing about your mentoring experience?
a. In addition to research methodology, the procedures of this project required skills in clinical assessment. During the course of working in this project I was required to assess and improve on my clinical and research skills. As a fledgling in research methodology and clinical assessments I required a lot of time from Dr. Burlingame and graduate students for support and training. I remember being quite anxious during my first BPRS assessment. Throughout the process though, I became accustomed to the techniques and processes of research and clinical assessment. The acquired skills have helped me learn more about how to write for scholarly journals and how to conduct clinical interviews.
This process gave me the background and experience that I needed to be prepared for graduate school in clinical psychology. Now in graduate school, I recognize the skills I gained through early experience with research and clinical work. I began my clinical psychology graduate program with, at least, a partial understanding of what is required in terms of professionalism, commitment, and skill acquisition.
5. How could your experience be improved?
a. If I were to change one thing about my experience I would have done it a year earlier. I think the growth I gain through the clinical experience specifically prepared me for an advanced program in clinical psychology.
6. Did any personal or spiritual development take place as a result of this mentoring project that likely would not have occurred without it?