Graduate Thesis Or Dissertation
 

Cross-Validation of Two Supervision Instruments with Counseling Trainees from CACREP-accredited Programs

Public Deposited

Downloadable Content

Download PDF
https://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/4j03d597v

Descriptions

Attribute NameValues
Creator
Abstract
  • Supervision is considered a pivotal professional intervention for counselors-in-training as they develop during graduate education and beyond. But, research on clinical supervision suffers from a lack of common instruments that can be utilized across disciplines, including counseling, and national boundaries. While a complex phenomenon to empirically address through research, supervision scholars have called for greater scrutiny in the types of instruments, measurement models, and research designs employed so that the field of supervision may be advanced (Goodyear et al., 2016). Additionally, how supervision-related phenomena are conceptualized, like supervision effectiveness and supervisor competence, within the research is foundational to the development of a robust research ecosystem. In order to contribute to the need for psychometrically robust supervision instruments that are grounded in robust theory, this dissertation project included two supervision measurement cross-validation studies on counselors-in-training in the United States. In these studies, I examined the psychometric properties of two different supervision instruments that share a similar theoretical conceptualization of supervision: (a) effective supervision (Study 1) and (b) supervisor competence (Study 2). These two measures were developed in non-U.S. clinical settings but whose psychometric properties and utility have yet to be verified for U.S.-based counseling practitioners. The overarching research question that both studies sought to address was: “Do existing supervision evaluation instruments maintain rigorous psychometric evidence for a sample of CITs from CACREP-accredited programs?” Conceptually, supervision effectiveness and supervisor competence were defined terms of the Proctor Model of Supervision. Each study drew from one sampling of 86 participants who were master’s-level counselors-in-training at CACREP-accredited programs from every region in the United States. The first study considered the psychometric properties of an instrument, the Manchester Clinical Supervision Scale (MCSS-26), that has been long-utilized to measure clinical supervision effectiveness outside of the U.S. In addition to the overarching research question identified above, Study 1 evaluated item-level performance and instrument-level internal consistency, concurrent validity, and social desirability threats to validity. The MCSS-26 was subjected to item-level analysis using a Generalized Partial Credit Model (GPCM) to explore the item difficulty, discrimination, and satisfaction to theoretical assumptions. Results of the study indicate acceptable instrument-level validity and reliability but poor item-level fitness for multiple items from the original 26-item instrument. Based on sample data, results suggest the revision of the MCSS-26 to a 9-item instrument that more appropriately fits within the item-response theoretical model of analysis for the study sample. Fitness indices for the revised scale suggest a better model fit compared to the fitness indices of the original instrument. Further revision, through continued research, is necessary in order to critically revise the MCSS-26 for use with a US-based counselor-in-training population. The second study examined an instrument that assesses supervisor competence, the Supervision Evaluation and Supervisor Competence (SE-SC) scale, from the supervisee’s perspective. Study 2, similar to Study 1, evaluated the item-level and instrument-level psychometrics of the subscales of the SE-SC. Item performance, internal consistency, concurrent validity, and social desirability threats to validity were all considered. The SE-SC was subjected to item-level analysis based on a GPCM that resulted in difficulty and discrimination parameters while also considering key theoretical assumptions of the model. Data from the current sample indicate acceptable instrument validity and reliability; however, item-level fitness to the model was poor, or “misfitting,” for a number of items. Results of Study 2 indicate the need for ongoing refinement of the SE-SC before use with a U.S.-based CIT population. Results further indicate that a 15-item revised SE-SC could be further developed with scrutiny. The revised scale possessed improved fitness indices compared to the original instrument, indicating a better fit to the GPCM. Supervision instruments that are relevant for U.S.-based CIT are sorely needed and considered critical to the development of the supervision scholarship in the years to come. As two supervision instruments that have been used to assess effectiveness and supervisor competence, the findings from both studies cast doubt on their utility for the population of U.S.-based CIT. Implications for Study 1 and Study 2 are presented with respect to instrument revision/development, counselor education and training, and the common measurement approach in supervision research. Additionally, findings from both studies suggest the urgency of constructing, refining, and developing psychometrically robust supervision instruments that can precisely assess supervision effectiveness and supervisor competence in future research. Overall, each study contributes to the supervision scholarship by casting doubt on two extant supervision instruments for use with a U.S.-based CIT population.
Contributor
License
Resource Type
Date Issued
Degree Level
Degree Name
Degree Field
Degree Grantor
Commencement Year
Advisor
Committee Member
Academic Affiliation
Rights Statement
Funding Statement (additional comments about funding)
  • This dissertation was supported, in part, by a $500 research grant from the Western Association for Counselor Education and Supervision (WACES).
Publisher
Peer Reviewed
Language

Relationships

Parents:

This work has no parents.

In Collection:

Items

Thumbnail Title Date Uploaded Visibility Actions
file details LitherlandGideonR2020.pdf 2020-08-05 Public Download