A-Z/Search | Calendar | Directory | News | Help | UCCS Home

Assessment Progress Report Guide

This guide is designed to assist those who complete the annual UCCS assessment progress reports for their academic unit.  If assistance is needed please contact the Director of Student Achievement Assessment or any SAAC member .            

Why Assess

Assessment Can:

Help faculty/departments/programs determine goals and objectives/outcomes

• What demonstrates understanding?
• What demonstrates knowledge?
• What demonstrates mastery?

Assist in the development of assessment instruments that collect data useful for determining attainment of objectives

Evaluate progress

• How are students progressing in the program?
• What pedagogical approaches are most effective?
• What changes or modifications are needed to help students?

Provide diagnostic feedback

• What are the student’s needs?
• Where could students improve and the program improve in order to meet those needs?
• Are the instruments collecting useful data?

Motivate student learning

• For students: How am I doing? How can I learn better?
• For faculty: What is working for the students? What can we do to help students more?

Help the institution remain accountable

• Are we providing students with the educational experience we say we are?
  

-return to top-

Who Fills Out Assessment Progress Reports

Each major is required to have an assessment plan in place and report progress annually.  Programs that have minors not affiliated with a degree granting program (stand alone) must also submit a progress report.   If you need assistance in determining if a program is required to submit a progress report, please contact the chair of the SAAC Committee .  

-return to top-

ESSENTIAL ASSESSMENT

Student Learning Outcomes/Objectives

Goals or Mission Statements are a relatively broad statement of overall intended outcomes while Outcomes/Objectives divide these goals/mission statements into specific, measurable areas. Currently the goals/mission statement you will find in WEAVEonline are those of the you College. Modify them if you choose. However, SAAC is most interested in outcomes/ objectives for student learning for your program.  Please list them in this section.  The outcomes/objectives should be clearly written, specific, and measurable.  Each program should have between three to five outcomes/objectives. Please note that Outcomes/Objectives for each program are listed annually in the Bulletin.

One way to write outcomes/objectives is to finish the sentence, "At the completion of the major students will be able to (insert specific ability which should be met at a specific point in time)."  When writing objectives, vague words and phrases should be avoided (i.e. 'gain a good grasp of', 'are exposed to', 'acquire', 'have knowledge of', 'understand', 'be familiar with', etc.).  Instead use action verbs that provide more description of how students will show that specific knowledge, skills, and abilities are obtained (i.e. illustrate, differentiate, analyze, criticize, translate, summarize, etc.).  A helpful tool in finding appropriate words is the Blooms Polygon.    

Examples of measurable objectives at UCCS include: 

• Graduating organizational communication majors will demonstrate growth and competence in writing and analytical skills.  Specifically, students will demonstrate effective use of language, clarity of expression, critical thinking, and ability to apply theoretical knowledge to actual cases, research skills, and appropriate organizational structure.  
• Students should be able to synthesize personal reality with the larger ethnic minority groups realities through theoretical and practical knowledge gained in EST courses. 
• Use basic knowledge in science and mathematics as well as knowledge and tools in engineering disciplines to analyze and synthesize real-world engineering problems.
• Be able to do political analysis- this implies the ability to think critically about and find rigorously defensible answers to political questions as well as the ability to understand and use empirical and other analytical methods in the pursuit of answers to political questions.
• Demonstrate food preparation, presentation and menu planning skills.

-return to top-

Measures (Assessment Instruments)

In this section list the instruments that will be used to determine if students are reaching the listed objectives.  At least two assessment measures should be listed (grades and passing rates are not acceptable assessment instruments).  Please note that one instrument may measure more than one objective and each student learning objective must be measured.  To assist in this task an Aligning Objectives and Instruments  worksheet has been developed.  Select the outcome(s)/objective(s) that are measured by each instrument and provide a target performance level (benchmark) for the instrument.

Full Description

Describe the instrument (survey, rubric, national exam, etc.) and explain who it is administered to, when it is used, and how it is utilized. Rubrics, survey items, forms, questions posed during exit interviews, etc. must be sent to SAAC for review.    Please note also if the instrument is formative or summative and if it is direct or indirect.

Direct and Indirect Measures

At least one direct and one indirect instrument should be used.  Direct measures are those that directly assess student learning (oral presentations, a selection of exam question answers, practicum rubric, etc.).  Indirect measures are those that provide information from the student on what they have learned rather than demonstrating it (surveys, interviews, focus groups, etc.).  Please see examples of appropriate direct and indirect measures for more information. 

-return to top-

Formative and Summative Assessment

Formative and Summative refer to when during a student's course of study the assessment takes place.  Formative assessment is focused on improvement (improve student learning, improve teaching) while summative assessment is focused on judgment.  A formative assessment takes place in time to make changes, or as Weston, McAlpine, and Bordonaro, (1995) states, "The purpose of formative evaluation is to validate or ensure that the goals of the instruction are being achieved and to improve the instruction, if necessary, by means of identification and subsequent remediation of problematic aspects."  Summative assessment on the other hand evaluates the learning process in its entirety.  Did the students learn what you expected them to learn?  Assessment plans should include both formative and summative forms of assessment.

"When the cook tastes the soup, that's formative;  when the guests taste
the soup, that's summative."  --Robert Stakes 

Examples of assessment instrument descriptions at UCCS include:  

Scholarly public presentation	Coached by instructor of capstone course in senior year, evaluated by group of alumni, other Art History students and faculty; rubric developed by Art History faculty.
American Chemical Society Exam	Normed national exams were administered to all core chemistry courses at the end of each semester.
Survey of clicker usage and technology use	Surveys were administered at the end of general chemistry and organic chemistry courses to allow students to assess how the technologies used in the courses impacted their perceived learning and satisfaction.
Thesis or project report	A student must either perform research and write a thesis or complete a project and write a project report. Both of these require a literature search and an analysis of the materials that are related to the thesis or project work. The thesis or report is generally submitted during the student's final semester.  The student's advisor fills out a form indicating the student's performance on the report.
Exit statement	A one-page exit statement that (a) reflects upon and synthesizes their overall experience in the minor, specifically as it relates to their respective disciplinary fields; (b) assesses the overall effectiveness of the program.
Employment survey of graduates	Survey asks supervisors to comment on the student's preparation for a mathematically oriented career. The preparation includes specific knowledge, the ability to use that knowledge in a professional environment, and the ability to continue technical learning both in the classroom and by the independent study.
Internship self-evaluation tool	Self-evaluation completed by students at completion of senior internship.
UCCS baccalaureate alumni survey	Measure will include specific questions added to the UCCS Baccalaureate and Alumni surveys. Administered by the Institutional Research Office annually.  Benchmark: Surveys should have a mean score of 3.5 or higher on a 6-point scale.

-return to top-

Rubrics

Rubrics provide a means of capturing quantitative data from otherwise qualitative student work.  By applying a rubric to a presentation, papers, portfolio items, etc. scores can be tallied, compared, and used to pinpoint areas in need of improvement. For more information regarding rubrics visit the UCCS Assessment Rubrics web site; a document regarding rubrics and their creation; and the free customizable rubric generator Rubistar.

-return to top-

Findings (Summary of Collected Data)

The Findings (summary of collected data) section is a place to synthesize the data that you have collected that show how students are meeting the learning objectives.  Please note whether the target level of achievement was met, partially met or not met and if any further action is planned.

• Please report findings even if they are incomplete. 
• Include information regarding how many students were assessed by each assessment instrument. 
• Note how these findings compare with earlier results, other cohorts, etc. 
• If a standardized test was utilized also note how the data compares to other institutions, other existing cohorts, etc.
• If Institutional Research surveys, national exams, etc. are utilized, please limit the summaries to results to data that specifically relates to the learning objectives. 
• If other evidence is found which illustrates that students are meeting the learning objectives, consider adding the instrument from which this data was collected into the assessment plan. 
• Provide an explanation if data are missing.

Examples of data summaries at UCCS include:

MSEE written report questionnaire	MSEE written report questionnaire has 8 questions and was evaluated in a scale of 1 (needs improvement) to 6 (outstanding) by faculty members in the thesis committee.  The average (N = 8) for all questions is 5.4 which show that the written report is above of excellent standard.  For a question on "What is the contribution level to the engineering field", the average score was 5.6 and this reflects that our MSEE student theses were close to outstanding quality. A minimum average score of 4.8 was found on the theses that were written based on IEEE standards.
Senior comprehensive exam	SCE Results 03-04: In Fall 2003, 19 students took the exam, 3 received Honors, 2 Failed, and 14 Passed.  The students who failed met with the chair and received copies of their exams with comments collated from the entire faculty; their teachers worked with students on idea development and students took Lab work on grammar and spelling.   In Spring 2004, 18 students took the exam, 3 received Honors, 1 Failed, and 14 Passed.  The two students who failed in October passed in April; the student who failed in April is a strong student who seemed overcome by nerves.
Exit survey	The exit survey predictably indicates that students are frustrated at the fact that EST cannot offer a greater variety of courses on a more regular basis.  Students consistently feel as though the Program has accomplished the goals and objectives outlined in section 1.
Exit exam	During the 2003-2004academic year, 51 students took the GES Exit Exam. During the Fall, 2003 semester, 14 students took the Exit Exam.  Mean score was 51.1 (73.0%), standard deviation = 6.8. During the Spring, 2004 semester, 37 students took the Exit Exam, with a mean score of 51.2 (73.1%), with a standard deviation of 7.1.  Range was 37 to 65. Exams beginning with the Fall 2003 had a different suite of questions.  A comparison between the old test and results from Fall 2003 to Fall 2004 exams will be analyzed in the Spring of 2005.  Given the general nature of the exam it is believed that students are demonstrating a proficiency in the stated GES goals.
Thesis evaluation form	Between fall and spring semesters, 40 students completed the Senior Thesis and therefore were evaluated through use of these forms. While the department has not conducted an elaborate statistical analysis, it has noted that of its five objectives the one for which the greatest number of students received either 'Poor' or 'Fair' is 'Research Skills' (Objective 2); the number was only three, low, but departmental faculty consider it worth considering as a possible indication of a trouble spot.
Graduate alumni survey	Greater than 85 % of the alumni surveyed reported the program met the expected outcomes related to the professional standards of their discipline. NP students requested more business content.
Academic Profile exam	Beth El Students (sophomores) scored above the mean on the academic profile in critical thinking. (Mean summary score was 448 and the National mean for Comprehensive Colleges and Universities was 444.1.  Benchmark Met. -return to top-

Action Plan (Improvements)

This section should list any actions/improvements planned or implemented to the program based on data obtained through the assessment process.  Improvements fall into three broad categories: Curricular , Academic Processes , and Assessment Processes . Examples include:  

Assessment Based Improvements
Curricular		Academic Processes		Assessment Processes
	Course Content		Personnel		Objectives
	Prerequisites		Advising		Measurement Instruments
	Course Sequence		Information Sharing		Data Collection and Analysis
	Program Requirements		Technology		Personnel Involvement
	Adding/Deleting Courses

 

Curricular	Upon an analysis of student work in an entry level course, the course has been redesigned and standardized across sections to ensure that students have the skills they need to take upper-division art history courses.
Academic Processes	Past Major Surveys have pointed to an important curricular gap in the area of Middle Eastern/North African History.  This fall the department has requested a new hire in this area, a request that has been ranked first by both the LAS Budget and Planning Committee and the LAS Chairs and Directors Committee.
Assessment Processes	A lack of quantitative data in the past led us to develop two rubrics for the Qualities of Effective Counselors Skills Rubric and the Personal Characteristics Rubric.
`	-return to top-

Analysis (Data Interpretation)

The data interpretation should answer the following questions: 

Strengths/Progress

• Based on the data collected, are students meeting the learning objectives? 
• Are students surpassing your expectations in any areas? 

Areas in need of attention

• Are there areas of concern, areas that need improvement? 
• Are the instruments producing useful information? 
• Are any changes needed to the objectives, the instruments, or the methods utilized to assess the objectives?   

Consider answering the questions for each learning objective in turn as illustrated in the first example.   

Excerpts from data interpretations across UCCS programs:  

• For Goal 2: Overall our direct measures show that the student learning has not significantly improved or deteriorated in the past year. Our indirect measure of the MBA Survey does provide a couple of areas of concern particularly in the areas of management and interpersonal skills. We need to address this in greater detail in the upcoming year. Alternatively, one of our weakest areas in the past, team skills, showed a huge improvement, from 41% to 61%.  
• In terms of student survey data, we were pleased to see the high ratings for the courses overall. For example, more than 90% of the students rated their career preparedness experience as excellent or very good. Our goal is to examine the curriculum more closely this year, so that ratings for all courses and in all Program areas are rated as excellent or very good.   The narrative comments, too, help us see the strengths of the Program and provide us with useful suggestions for Program improvement. We will be discussing these points this fall as we begin to consider an ongoing assessment protocol for the Program.  
• Data generated by LTC usage and S-Cape proficiency exams for lower level courses indicate we are meeting student-learning objectives. In as far as language acquisition is concerned, upper level courses will be evaluated as per the pre-exit S-Cape exam, which we have just instituted and will have quantitative findings by next year. Of course, the objectives dealing with literary/cultural aspect of upper division courses cannot be measured in this way, hence we are developing the portfolio, and monitoring grades and student reaction to changes in the curriculum.

-return to top-

Annual Reporting

Annual Reporting within WEAVEonline includes the following standard areas (in bold). SAAC requests that for those that have been struck through that the unit instead provide the typical responses for Faculty Involvement, Course Level Assessment, Roles & Communication, and External Constituents. Units may choose to respond to the bold/unstruck annual reporting areas.

Executive Summary
Contributions to the Institution ------- COURSE-LEVEL ASSESSMENT:
Highlights
Teaching Activities ------- FACULTY INVOLVEMENT
Research and Scholarly Activities
Public/Community Service ------- ROLES & COMMUNICATION
International Activities ------- EXTERNAL CONSTITUENTS
Challenges

Faculty Involvement

Please respond to this essential assessment area in the annual reporting section of WEAVEonline that is titled Teaching Activities.

The assessment process cannot work without the input of faculty.  It is important for the SAAC to see in this section that faculty participate in as many aspects of the process as possible.  It is hoped that faculty meet on a regular basis to discuss outcomes, revise aspects of the assessment plan, receive and review assessment findings, and suggest and implement improvements and receive encouragement for a job well done. 

ADVANCED ASSESSMENT

Course-Level Assessment

Please respond to this advanced assessment area in the annual reporting section of WEAVEonline that is titled Contributions to the institution.

Course-level assessment provides the assurance that students are gaining elements (skill/knowledge) of the student learning objectives as they progress through their course of study.  Consider the first step of this process to be the mapping of where specific objectives are met in each of the program courses.  Use the Course-Level Mapping worksheet to assist in this process.  After mapping, determine specific objectives to measure at various stages/specific courses throughout a student's course of study.

-return to top-

Roles & Communication

Please respond to this advanced assessment area in the annual reporting section of WEAVEonline that is titled Public/Community Service.

SAAC expects that the assessment process be a collaborative activity involving as many members of the program as possible.  Questions such as who creates the annual report, who collects and maintains assessment data, how is information regarding the assessment process disseminated, how often does the department meet to discuss assessment, and how were faculty involved in creating potential improvements should be answered in this section.

-return to top-

External Constituents

Please respond to this advanced assessment area in the annual reporting section of WEAVEonline that is titled International Activities.

It is important for external constituents such as alumni, employers, practicum supervisors, advisory board members, community partners, potential employers, etc. to take part in the assessment process.  Every effort possible should be made to include these constituents in some part of the assessment of either student learning (are students meeting standards expected the community, do alumni feel they learned what they needed for employment) and the curriculum (do external constituents, especially future employers, feel the program is providing the training, skills, knowledge that is required for employment; do alumni have suggestions as to what areas the curriculum is satisfactory or needs improvement).   

Consider having external constituents attend and complete rubrics for class presentations or poster sessions, have them fill out evaluations of student projects, critique thesis and projects.  Ask graduating seniors for feedback through focus groups, exit interviews or surveys.  Utilize the UCCS IR office surveys or create a survey specific to your program.  Feedback of these types provides unique information that can strengthens programs and assist in the attainment of learning objectives. 

-return to top-

Last Updated : July 10, 2007  

Search UCCS: A/Z   Google UCCS   Staff/Fac   Students