This guide is designed to assist those who complete the annual UCCS assessment progress reports for their academic unit. If assistance is needed please contact the Director of Student Achievement Assessment or any SAAC member .
Assessment Can:
Help faculty/departments/programs determine goals and objectives/outcomes
- What demonstrates understanding?
- What demonstrates knowledge?
- What demonstrates mastery?
Assist in the development of assessment instruments that collect data useful for determining attainment of objectives
Evaluate progress
Provide diagnostic feedback
- How are students progressing in the program?
- What pedagogical approaches are most effective?
- What changes or modifications are needed to help students?
Motivate student learning
- What are the student’s needs?
- Where could students improve and the program improve in order to meet those needs?
- Are the instruments collecting useful data?
Help the institution remain accountable
- For students: How am I doing? How can I learn better?
- For faculty: What is working for the students? What can we do to help students more?
- Are we providing students with the educational experience we say we are?
Each major is required to have an assessment plan in place and report progress annually. Programs that have minors not affiliated with a degree granting program (stand alone) must also submit a progress report. If you need assistance in determining if a program is required to submit a progress report, please contact the chair of the SAAC Committee .
Goals or Mission Statements are a relatively broad statement of overall intended outcomes while Outcomes/Objectives divide these goals/mission statements into specific, measurable areas. Currently the goals/mission statement you will find in WEAVEonline are those of the you College. Modify them if you choose. However, SAAC is most interested in outcomes/ objectives for student learning for your program. Please list them in this section. The outcomes/objectives should be clearly written, specific, and measurable. Each program should have between three to five outcomes/objectives. Please note that Outcomes/Objectives for each program are listed annually in the Bulletin.
One way to write outcomes/objectives is to finish the sentence, "At the completion of the major students will be able to (insert specific ability which should be met at a specific point in time)." When writing objectives, vague words and phrases should be avoided (i.e. 'gain a good grasp of', 'are exposed to', 'acquire', 'have knowledge of', 'understand', 'be familiar with', etc.). Instead use action verbs that provide more description of how students will show that specific knowledge, skills, and abilities are obtained (i.e. illustrate, differentiate, analyze, criticize, translate, summarize, etc.). A helpful tool in finding appropriate words is the Blooms Polygon.
Examples of measurable objectives at UCCS include:
Full Description
Describe the instrument (survey, rubric, national exam, etc.) and explain who it is administered to, when it is used, and how it is utilized. Rubrics, survey items, forms, questions posed during exit interviews, etc. must be sent to SAAC for review. Please note also if the instrument is formative or summative and if it is direct or indirect.
Direct and Indirect Measures
At least one direct and one indirect instrument should be used. Direct measures are those that directly assess student learning (oral presentations, a selection of exam question answers, practicum rubric, etc.). Indirect measures are those that provide information from the student on what they have learned rather than demonstrating it (surveys, interviews, focus groups, etc.). Please see examples of appropriate direct and indirect measures for more information.
Formative and Summative Assessment
Formative and Summative refer to when during a student's course of study the assessment takes place. Formative assessment is focused on improvement (improve student learning, improve teaching) while summative assessment is focused on judgment. A formative assessment takes place in time to make changes, or as Weston, McAlpine, and Bordonaro, (1995) states, "The purpose of formative evaluation is to validate or ensure that the goals of the instruction are being achieved and to improve the instruction, if necessary, by means of identification and subsequent remediation of problematic aspects." Summative assessment on the other hand evaluates the learning process in its entirety. Did the students learn what you expected them to learn? Assessment plans should include both formative and summative forms of assessment.
"When the cook tastes the soup, that's formative; when the guests taste
the soup, that's summative." --Robert StakesExamples of assessment instrument descriptions at UCCS include:
Scholarly public presentation
Coached by instructor of capstone course in senior year, evaluated by group of alumni, other Art History students and faculty; rubric developed by Art History faculty.
American Chemical Society Exam
Normed national exams were administered to all core chemistry courses at the end of each semester.
Survey of clicker usage and technology use
Surveys were administered at the end of general chemistry and organic chemistry courses to allow students to assess how the technologies used in the courses impacted their perceived learning and satisfaction.
Thesis or project report
A student must either perform research and write a thesis or complete a project and write a project report. Both of these require a literature search and an analysis of the materials that are related to the thesis or project work. The thesis or report is generally submitted during the student's final semester. The student's advisor fills out a form indicating the student's performance on the report.
Exit statement
A one-page exit statement that (a) reflects upon and synthesizes their overall experience in the minor, specifically as it relates to their respective disciplinary fields; (b) assesses the overall effectiveness of the program.
Employment survey of graduates
Survey asks supervisors to comment on the student's preparation for a mathematically oriented career. The preparation includes specific knowledge, the ability to use that knowledge in a professional environment, and the ability to continue technical learning both in the classroom and by the independent study.
Internship self-evaluation tool
Self-evaluation completed by students at completion of senior internship.
UCCS baccalaureate alumni survey
Measure will include specific questions added to the UCCS Baccalaureate and Alumni surveys. Administered by the Institutional Research Office annually. Benchmark: Surveys should have a mean score of 3.5 or higher on a 6-point scale.
Rubrics
Rubrics provide a means of capturing quantitative data from otherwise qualitative student work. By applying a rubric to a presentation, papers, portfolio items, etc. scores can be tallied, compared, and used to pinpoint areas in need of improvement. For more information regarding rubrics visit the UCCS Assessment Rubrics web site; a document regarding rubrics and their creation; and the free customizable rubric generator Rubistar.
The Findings (summary of collected data) section is a place to synthesize the data that you have collected that show how students are meeting the learning objectives. Please note whether the target level of achievement was met, partially met or not met and if any further action is planned.
Examples of data summaries at UCCS include:
MSEE written report questionnaire |
MSEE written report questionnaire has 8 questions and was evaluated in a scale of 1 (needs improvement) to 6 (outstanding) by faculty members in the thesis committee. The average (N = 8) for all questions is 5.4 which show that the written report is above of excellent standard. For a question on "What is the contribution level to the engineering field", the average score was 5.6 and this reflects that our MSEE student theses were close to outstanding quality. A minimum average score of 4.8 was found on the theses that were written based on IEEE standards. |
Senior comprehensive exam |
SCE Results 03-04: In Fall 2003, 19 students took the exam, 3 received Honors, 2 Failed, and 14 Passed. The students who failed met with the chair and received copies of their exams with comments collated from the entire faculty; their teachers worked with students on idea development and students took Lab work on grammar and spelling. In Spring 2004, 18 students took the exam, 3 received Honors, 1 Failed, and 14 Passed. The two students who failed in October passed in April; the student who failed in April is a strong student who seemed overcome by nerves. |
Exit survey |
The exit survey predictably indicates that students are frustrated at the fact that EST cannot offer a greater variety of courses on a more regular basis. Students consistently feel as though the Program has accomplished the goals and objectives outlined in section 1. |
Exit exam |
During the 2003-2004academic year, 51 students took the GES Exit Exam. During the Fall, 2003 semester, 14 students took the Exit Exam. Mean score was 51.1 (73.0%), standard deviation = 6.8. During the Spring, 2004 semester, 37 students took the Exit Exam, with a mean score of 51.2 (73.1%), with a standard deviation of 7.1. Range was 37 to 65. Exams beginning with the Fall 2003 had a different suite of questions. A comparison between the old test and results from Fall 2003 to Fall 2004 exams will be analyzed in the Spring of 2005. Given the general nature of the exam it is believed that students are demonstrating a proficiency in the stated GES goals. |
Thesis evaluation form |
Between fall and spring semesters, 40 students completed the Senior Thesis and therefore were evaluated through use of these forms. While the department has not conducted an elaborate statistical analysis, it has noted that of its five objectives the one for which the greatest number of students received either 'Poor' or 'Fair' is 'Research Skills' (Objective 2); the number was only three, low, but departmental faculty consider it worth considering as a possible indication of a trouble spot. |
Graduate alumni survey
|
Greater than 85 % of the alumni surveyed reported the program met the expected outcomes related to the professional standards of their discipline. NP students requested more business content. |
Academic Profile exam
|
Beth El Students (sophomores) scored above the mean on the academic profile in critical thinking. (Mean summary score was 448 and the National mean for Comprehensive Colleges and Universities was 444.1. Benchmark Met. |
This section should list any actions/improvements planned or implemented to the program based on data obtained through the assessment process. Improvements fall into three broad categories: Curricular , Academic Processes , and Assessment Processes . Examples include:
| Assessment Based Improvements | |||||
| Curricular | Academic Processes | Assessment Processes | |||
| Course Content | Personnel | Objectives | |||
| Prerequisites | Advising | Measurement Instruments | |||
| Course Sequence | Information Sharing | Data Collection and Analysis | |||
| Program Requirements | Technology | Personnel Involvement | |||
| Adding/Deleting Courses | |||||
|
Upon an analysis of student work in an entry level course, the course has been redesigned and standardized across sections to ensure that students have the skills they need to take upper-division art history courses. |
|
Past Major Surveys have pointed to an important curricular gap in the area of Middle Eastern/North African History. This fall the department has requested a new hire in this area, a request that has been ranked first by both the LAS Budget and Planning Committee and the LAS Chairs and Directors Committee. |
|
A lack of quantitative data in the past led us to develop two rubrics for the Qualities of Effective Counselors Skills Rubric and the Personal Characteristics Rubric. |
| ` |
The data interpretation should answer the following questions:
Strengths/Progress
Areas in need of attention
Consider answering the questions for each learning objective in turn as illustrated in the first example.
Excerpts from data interpretations across UCCS programs:
Annual Reporting within WEAVEonline includes the following standard areas (in bold). SAAC requests that for those that have been struck through that the unit instead provide the typical responses for Faculty Involvement, Course Level Assessment, Roles & Communication, and External Constituents. Units may choose to respond to the bold/unstruck annual reporting areas.
Executive Summary
Contributions to the Institution------- COURSE-LEVEL ASSESSMENT:
Highlights
Teaching Activities------- FACULTY INVOLVEMENT
Research and Scholarly Activities
Public/Community Service------- ROLES & COMMUNICATION
International Activities------- EXTERNAL CONSTITUENTS
Challenges
Please respond to this essential assessment area in the annual reporting section of WEAVEonline that is titled Teaching Activities.
The assessment process cannot work without the input of faculty. It is important for the SAAC to see in this section that faculty participate in as many aspects of the process as possible. It is hoped that faculty meet on a regular basis to discuss outcomes, revise aspects of the assessment plan, receive and review assessment findings, and suggest and implement improvements and receive encouragement for a job well done.
Please respond to this advanced assessment area in the annual reporting section of WEAVEonline that is titled Contributions to the institution.
Course-level assessment provides the assurance that students are gaining elements (skill/knowledge) of the student learning objectives as they progress through their course of study. Consider the first step of this process to be the mapping of where specific objectives are met in each of the program courses. Use the Course-Level Mapping worksheet to assist in this process. After mapping, determine specific objectives to measure at various stages/specific courses throughout a student's course of study.
Please respond to this advanced assessment area in the annual reporting section of WEAVEonline that is titled Public/Community Service.
SAAC expects that the assessment process be a collaborative activity involving as many members of the program as possible. Questions such as who creates the annual report, who collects and maintains assessment data, how is information regarding the assessment process disseminated, how often does the department meet to discuss assessment, and how were faculty involved in creating potential improvements should be answered in this section.
Please respond to this advanced assessment area in the annual reporting section of WEAVEonline that is titled International Activities.
It is important for external constituents such as alumni, employers, practicum supervisors, advisory board members, community partners, potential employers, etc. to take part in the assessment process. Every effort possible should be made to include these constituents in some part of the assessment of either student learning (are students meeting standards expected the community, do alumni feel they learned what they needed for employment) and the curriculum (do external constituents, especially future employers, feel the program is providing the training, skills, knowledge that is required for employment; do alumni have suggestions as to what areas the curriculum is satisfactory or needs improvement).
Consider having external constituents attend and complete rubrics for class presentations or poster sessions, have them fill out evaluations of student projects, critique thesis and projects. Ask graduating seniors for feedback through focus groups, exit interviews or surveys. Utilize the UCCS IR office surveys or create a survey specific to your program. Feedback of these types provides unique information that can strengthens programs and assist in the attainment of learning objectives.
Last Updated :
July 10, 2007