Research Study Conducted By Pine Rest Clinicians Published In the Journal “Psychiatric Services”

June 25, 2016

“Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting: The VOCATIONS Trial,” was published online in “Psychiatric Services in Advance” in June and will be in the October print edition.

Three Pine Rest staff members were involved:  Scott Halstead, Ph.D.; Eric Achtyes, M.D., M.S., F.A.P.A.; and LeAnn Smart, B.A. Dr. Halstead was the principal investigator; Dr. Achtyes was co-investigator and lead author. Contributors included Tara Moore, M.A., M.P.H.; Ellen Frank, Ph.D.; David J. Kupfer, M.D. from the University of Pittsburgh; and Robert Gibbons, Ph.D. from the University of Chicago.

With expansion of Medicaid eligibility and passage of the Affordable Care Act, there is pressure on the mental health care system to efficiently and effectively provide mental health assessment and treatment for millions of additional people seeking care.

Computerized adaptive testing (CAT) provides an alternative to fixed-length assessments like the 9-item Patient Health Questionnaire (PHQ-9), and allows for rapid, computerized assessment of psychiatric symptoms in patients.

This study tested whether a computerized suite of tests, called the CAT-MH (mental health), could provide a rapidly administered, accurate assessment of depression diagnosis and symptom severity across a broad range of mood and anxiety symptoms in an adult, community outpatient psychiatric population.

The VOCATIONS trial (Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting) commenced as a cross-sectional validation study of the CAT-MH suite of tests and was conducted between April 2012 and March 2013 at the Pine Rest outpatient clinic located on the main campus in Cutlerville, Mich.

A total of 145 adults, including 19 with no history of a mental disorder (control group), were prospectively evaluated with CAT for depression, mania, and anxiety symptoms.

The original CAT studies developed a 1,008-item question bank consisting of 452 depression items, 467 anxiety items, and 89 mania items. The items were selected on the basis of a review of more than 100 existing depression and depression related scales.

“There are two major differences between current testing and computerized adaptive testing,” says Dr. Halstead. “Classical and the logic of item response theory (IRT) methods of measurement differ dramatically in the ways they are administered and scored.

“These measurements can be hampered by a ‘practice effect,’ which results from retaking the same measure repeatedly over time. Because CAT adapts to the current severity level of a patient, these practice effects are eliminated because the patient receives different items each time the test is administered. Second, for repeated assessments, traditional tests make no use of the information contained in the preceding test administrations. By contrast, the last CAT-based severity measure can be used to start the next CAT.”

Analysis of the research found that participants took, on average, 51.7 items and 9.4 minutes to complete the CAT-MH. Patients found the computerized adaptive tests acceptable to use, felt comfortable answering personal questions about themselves, answered them honestly, preferred computerized adaptive tests over pencil-and-paper tests, and felt the test accurately reflected their mood. There was some concern that older patients would not find the computerized test as easy to take. This was not found to be the case.

“As measurement-based care becomes the standard for assessment of illness severity and improvement with treatment, well-validated, affordable, and quick measures are needed to help busy clinicians treat patients rapidly and effectively,” says Dr. Achtyes.

A link to the complete article is here:


Related Announcements

Announcement Categories