This document is written to include the original self-study (text in black), and for each standard the relevant CAEP offsite Formative Feedback Report (FFR) narrative (text in blue). COE's response to the FFR is incorporated throughout and at the end of each standard (text in red).
The following link takes the reader to the FFR comments along with the EPP's response.
Standard 3: Candidate Quality, Recruitment, and Selectivity
The provider demonstrates that the quality of candidates is a continuing and purposeful part of its responsibility from recruitment, at admission, through the progression of courses and clinical experiences, and to decisions that completers are prepared to teach effectively and are recommended for certification. The provider demonstrates that development of candidate quality is the goal of educator preparation in all phases of the program.
The two sources of evidence used to document candidate quality, recruitment and selectivity in address Standard 3 are the Transition Point data (3A.1) and the Developmental Assessment Matrix or DAM instrument and data (3B; 3B.1).
TRANSITION POINT DATA SUMMARY
In 2012, department chairs across the College worked collaboratively to establish commonalities in admissions standards for its initial and advanced programs. Requirements for initial programs include:
- GPA of 2.5 or better
- Professional goals statement
- Student interview with department
- Three letters of recommendation
- Background check
Graduate programs require:
- Baccalaureate degree from a regional accredited, four-year institution of higher education
- Minimum GPA of 2.75 for completed undergraduate work and 3.0 for graduate work
- Personal goal statement
- Professional interview with department faculty
- Three letters of recommendation
In 2012-13, the Assessment and Accreditation Committee (A&A) began to work in earnest to build a college-wide assessment plan that included a more robust data collection system. One subject that needed to be addressed was to identify signature assignments and key assessments for each program, ensuring alignment with standards and designated point of completion. A&A led the work which resulted in departments defining transition points for their programs. The committee created an Assessment System Transition Point plan that spans both initial and advanced programs. The five common transition points include:
- Transition Point 1: Admissions
- Transition Point 2: Completion of Core Courses and Entry into Field Experience
- Transition Point 3: Mid-point of Field Experience
- Transition Point 4: Completion of Field Experience
- Transition Point 5: Post Graduation Success
An introduction to the plan notes that "initial programs measure professional dispositions at four of the five transition points and advanced programs measure dispositions at a minimum of two points. Both initial and advanced programs review professional dispositions data each semester and combine this data with progress and performance date to make decisions about each candidate's continuation in the program."
The introduction of Cherwell, an electronic student management system explained in Standard 5, allows the College to report all phases related to the transition points. Information collected in this system interfaces with real-time data from the campus's enterprise data system (CUSIS) and is accessible 24/7 to COE faculty and staff.
As a result of Colorado's Senate Bill 191, the state will soon be submitting employment evaluation and student growth data that will inform the College on numerous aspects related to Transition Point 5: Post Graduation Success.
As noted above, departments use interviews as one of the requirements for admissions. Included in Standard 3 (AIMS) is a document that contains the interview forms for Principal Licensure, Special Education, the Teacher Education Licensure Program and Counseling. Interviews are scored, based on departmental criteria, and the scores become part of an electronic spreadsheet compiled by the Student Resource Office that also adds other admissions criteria (GPA, experience, goal statement, references, etc.) and a total score. The composite score is used in determining what or not a student is admitted. Also included in the EE are examples of the summary report from Transition 1 that indicates student admissions results (3A.1 Counseling and 3A.2 Teacher Education Licensure Program).
Departments decide on criteria for admissions based on several factors: state and national standards, specialized professional associations, the Graduate School, campus undergraduate requirements, and professional practices. Candidates generally have multiple interviewers, providing more than one perspective on the candidates' potential success.
The evidence collected indicate candidates meet qualitative-DAM, interviews, and letters of recommendation-and quantitative-GPA and GRE/MAT-metrics for entry and retention into the College's programs. Those candidates outside the established guidelines are provisionally admitted, identified for additional support, or exited from programs.
The Transition 1 summary reports provide departments with details about the number and quality of applications, admissions through the use of multiple measures. The summary also gives department chairs or directors an understanding of the percentages of students accepted, provisionally accepted or rejected. Trend data and admissions analysis helps departments refine their admissions process and provides a glimpse into what factors are indicators of predicting candidate success.
As a result of moving to Cherwell and CUSIS, faculty are able to easily access data for Transition Point 1: Admissions and Transition Point 2: Completion of Core Courses and Entry into Field Experience. The College's SRO and Office of Assessment and Accreditation are creating the systems to make the data from the remaining Transition Points (3-5) more easily accessible.
THE DEVELOPMENTAL ASSESSMENT MATRIX (DAM) & DISPOSITIONS
The DAM instrument (3B), was developed and implemented by faculty in the Department of Counseling and Human Services. All students in the DCHS are formally evaluated on each of the three program candidate learning outcomes at the completion of each semester on a standardized program rubric. The rubric (the Developmental Assessment Matrix), is informally referred to as 123s. During the first year of graduate study, faculty will discuss individual student performance every semester in each course and will assign scores in three specific areas: Counseling Knowledge, Skills and Self Awareness. Results of the Development Assessment are made available to students and all first year students receive their score and specific feedback at the end of the summer, fall and spring semesters (i.e. after the first three semesters). Students who are not meeting expectations in any the above discussed three areas may be asked to do one or more of the following: meet with their advisor, meet with the whole faculty, develop a remediation plan, seek individual counseling, postpone fieldwork, sit out for a semester or withdraw from graduate school. Faculty make every attempt to work with students in a proactive, preventive manner in hopes of helping them move successfully through their respective graduate program without interruption. Faculty are committed to providing support and specific mentoring to improve student achievement and assist students in developing the attitudes and behaviors that will lead to academic, personal and professional success. Counseling has been using the DAM for four years and the Department of Special Education is adapting DAM for its 2014-15 cohort use. Other departments are also considering its adoption.
Also included in the EE is an analysis of the data for the last four cohorts of Counseling students (3B.1). For each cohort, there's information on the number of students in the cohort that year, the number of students flagged for support or remediation, the areas needing attention, and the interventions taken. While DAM has not been externally validated, its quality and appropriateness of use were upheld in legal proceedings in 2012.
The data analysis for DAM indicates that every cohort has had at least some students who have raised concerns in at least one of the three categories (Personal Growth/Professionalism; Knowledge; Skills). For three of the four cohorts, Personal Growth/Professionalism was by far the category with the greatest number of concerns while for the '13-'14 cohort, no students raised concerns in that area. Concerns are taken seriously, as is shown by the number of students and types of interventions taken. There's a range of actions from spending more time with a faculty advisor to removal from program. The reason DAM is such a powerful and meaningful instrument for student performance and dispositions is that the faculty meet frequently to discuss student progress and any concerns. Based on the lessons learned from examining cohort data, the Department of Counseling and Human Services made applicant screening more intense by adding an on-campus group interview process.
STANDARD 3: SUMMARY
In order to address Candidate Quality and Selectivity in a more thoughtful, systematic way the College's department chairs collaborated on determining common admissions criteria for undergraduate and for graduate programs. Based on that work, the Assessment and Accreditation Committee created 5 common Transitions Points. Cherwell was designed to help track students at various phases of the student life cycle (inquiring, applying, matriculated, and program completer). One of the requirements of COE programs is that students complete at least one interview in order to be admitted. Tracking interview scores and other admission criteria is the responsibility of the departments with the assistance of the Student Resource Office. The College is systemizing a process for increasing the automation of data collection and decision-making. While in the early stages of development, the two summaries provided offer a preview of the direction for all programs. The Developmental Assessment Matrix also signals a shift in how the College thinks about dispositions and the intensity of follow-up needed to ensure candidates stay on track in meeting program expectations in an arena that is difficult to assess. Special Education is adapting DAM for fall, 2014 implementation. Even if the other departments don't adopt DAM, the instrument has certainly generated valuable discussion about how programs assess the traits they expect their candidates to exhibit.