Assessment Progress Report Guide

   

This guide is designed to assist those who complete the UCCS assessment progress reports for their academic unit.  If assistance is needed please contact the Student Learning Outcomes Coordinator or any SAAC member.            

Why Assess   

Assessment Can:

1). Help faculty/departments/programs determine goals and objectives/outcomes

• What knowledge is integral to a student graduating from our undergraduate/graduate program?
• What skills are integral to a student graduating from our undergraduate/graduate program?
• What does a student graduating with a degree from our program 'look like'?
• What skills and knowledge separate this student from a student in another degree program?

2). Assist in the development of assessment instruments that collect data useful for determining student achievement related to your student learning outcomes.

Identify appropriate instruments

• What demonstrates knowledge?
• What demonstrates understanding?
• What demonstrates skill?
• What demonstrates mastery?

Evaluate progress

• How are students progressing in the program?
• What pedagogical approaches are most effective?
• What changes or modifications are needed to help students?

Provide diagnostic feedback

• What are the student’s needs?
• Where could students improve and the program improve in order to meet those needs?
• Are the instruments collecting useful data?

Motivate student learning

• For students: How am I doing? How can I learn better?
• For faculty: What is working for the students? What can we do to help students more?

Help the institution remain accountable

• Are we providing students with the educational experience we say we are?

-return to top-

Who Fills Out Assessment Progress Reports   

Each major is required to have an assessment plan in place and report progress during their scheduled reporting cycle.  Programs that have minors not affiliated with a degree granting program (stand alone) must also submit a progress report.   If you need assistance in determining if a program is required to submit a progress report, please contact the chair of the SAAC Committee.

-return to top-

   ASSESSMENT ESSENTIALS

Student Learning Outcomes

Mission Statements and Goals are a relatively broad statement of overall intended teaching objectives while Outcomes divide these goals/mission statements into specific, measurable areas. Currently, completing the mission statement/goals sections of WEAVEonline is optional, although completing them may help focus your assessment project. Student Learning Outcomes must be completed.  Please list them in this section.  The outcomes should be clearly written, specific, and measurable.  Each program should have between three to five outcomes. Please note that student learning outcomes for each program are listed annually in the Bulletin.

One effective way to write outcomes is to finish this sentence; "At the completion of the major students will be able to (insert specific ability which should be met at a specific point in time)."  When writing objectives, vague words and phrases should be avoided (i.e. 'gain a good grasp of', 'are exposed to', 'acquire', 'have knowledge of', 'understand', 'be familiar with', etc.).  Instead use action verbs that provide more description of how students will show that specific knowledge, skills, and abilities are obtained (i.e. illustrate, differentiate, analyze, criticize, translate, summarize, etc.).  A helpful tool in finding appropriate words is the Blooms Polygon.    

Examples of measurable objectives at UCCS include: 

• Graduating organizational communication majors will demonstrate growth and competence in writing and analytical skills.  Specifically, students will demonstrate effective use of language, clarity of expression, critical thinking, and ability to apply theoretical knowledge to actual cases, research skills, and appropriate organizational structure.  
• Students should be able to synthesize personal reality with the larger ethnic minority groups realities through theoretical and practical knowledge gained in EST courses. 
• Use basic knowledge in science and mathematics as well as knowledge and tools in engineering disciplines to analyze and synthesize real-world engineering problems.
• Be able to do political analysis- this implies the ability to think critically about and find rigorously defensible answers to political questions as well as the ability to understand and use empirical and other analytical methods in the pursuit of answers to political questions.
• Demonstrate food preparation, presentation and menu planning skills.

-return to top-

Measures (Assessment Instruments)   

In this section list the instruments that will be used to determine if students are reaching the student learning outcomes.  At least two assessment measures should be listed (grades and passing rates are not acceptable assessment instruments).  Please note that one instrument may measure more than one objective and each student learning objective must be measured at least once.  To assist in this task an Aligning Objectives and Instruments  worksheet has been developed.  Select the outcomes that are measured by each instrument and provide a target performance level (benchmark) for the instrument.  You may find that determing your baseline and then setting a minimum acceptable student achievement level and an aspirational student achievement target to be useful.

Full Description

Describe the instrument (survey, rubric, national exam, etc.) and explain who it is administered to, when it is administered  (sampling plan and schedule) and how it is administered. Rubrics, survey items, forms, questions posed during exit interviews, etc. must be sent to SAAC for review.    Please note also if the instrument is formative or summative and if it is direct or indirect.

Direct and Indirect Measures   

At least one direct and one indirect instrument should be used.  Direct measures are those that directly assess student learning (oral presentations, a selection of exam question answers, practicum rubric, etc.).  Indirect measures are those that provide information from the student on what they have learned rather than demonstrating it (surveys, interviews, focus groups, etc.).  Please see examples of appropriate direct and indirect measures for more information. 

-return to top-

Formative and Summative Assessment   

Formative and Summative refer to when during a student's course of study the assessment takes place.  Formative assessment is focused on improvement (improve student learning, improve teaching) while summative assessment is focused on judgment.  A formative assessment takes place in time to make changes, or as Weston, McAlpine, and Bordonaro, (1995) states, "The purpose of formative evaluation is to validate or ensure that the goals of the instruction are being achieved and to improve the instruction, if necessary, by means of identification and subsequent remediation of problematic aspects."  Summative assessment on the other hand evaluates the learning process in its entirety.  Did the students learn what you expected them to learn?  Assessment plans should include both formative and summative forms of assessment.

"When the cook tastes the soup, that's formative;  when the guests taste
the soup, that's summative."  --Robert Stakes 

Examples of assessment instrument descriptions at UCCS include:  

Scholarly public presentation	Coached by instructor of capstone course in senior year, evaluated by group of alumni, other Art History students and faculty; rubric developed by Art History faculty.
American Chemical Society Exam	Normed national exams were administered to all core chemistry courses at the end of each semester.
Survey of clicker usage and technology use	Surveys were administered at the end of general chemistry and organic chemistry courses to allow students to assess how the technologies used in the courses impacted their perceived learning and satisfaction.
Thesis or project report	A student must either perform research and write a thesis or complete a project and write a project report. Both of these require a literature search and an analysis of the materials that are related to the thesis or project work. The thesis or report is generally submitted during the student's final semester.  The student's advisor fills out a form indicating the student's performance on the report.
Exit statement	A one-page exit statement that (a) reflects upon and synthesizes their overall experience in the minor, specifically as it relates to their respective disciplinary fields; (b) assesses the overall effectiveness of the program.
Employment survey of graduates	Survey asks supervisors to comment on the student's preparation for a mathematically oriented career. The preparation includes specific knowledge, the ability to use that knowledge in a professional environment, and the ability to continue technical learning both in the classroom and by the independent study.
Internship self-evaluation tool	Self-evaluation completed by students at completion of senior internship.
UCCS baccalaureate alumni survey	Measure will include specific questions added to the UCCS Baccalaureate and Alumni surveys. Administered by the Institutional Research Office annually.  Benchmark: Surveys should have a mean score of 3.5 or higher on a 6-point scale.

-return to top-

Rubrics   

Rubrics provide a means of capturing quantitative data from otherwise qualitative student work.  By applying a rubric to a presentation, papers, portfolio items, etc. scores can be tallied, compared, and used to pinpoint areas in need of improvement. For more information regarding rubrics visit the UCCS Assessment Rubrics web site; a document regarding rubrics and their creation; and the free customizable rubric generator Rubistar.

-return to top-

Findings (Summary of Collected Data)   

The Findings (summary of collected data) section is a place to present the data that you have collected that demonstrates how successfully students are meeting the learning outcomes.  Please note whether the minimally acceptable level of achievement was met, partially met or not met and if any further action is planned.  In WEAVE, the findings are entered in the same section as the measures.

• Please report findings even if they are incomplete.
  
• Provide an explanation if data are missing.
• Discuss when and how the data was collected, compiled, analyzed and discussed.
  
• Please limit data presentation and discussion to data that are specifically related to student achievement related to the student learning outcomes.
  
• Include information regarding how many students were assessed by each assessment instrument. 
• Discuss how these findings compare with earlier results, other cohorts, changes in program/curriculum, etc. 
• If a standardized test was utilized also note how the data compares to other institutions, other existing cohorts, etc.
• If Institutional Research surveys, national exams, etc. are utilized, please limit the presentation and summaries/discussion to results from data that specifically relates to the learning objectives.  
• If other evidence is found which illustrates that students are meeting the learning objectives, consider adding the instrument/source from which this data was collected into the assessment plan. 

  Examples of data summaries at UCCS include:

MSEE written report questionnaire	MSEE written report questionnaire has 8 questions and was evaluated in a scale of 1 (needs improvement) to 6 (outstanding) by faculty members in the thesis committee.  The average (N = 8) for all questions is 5.4 which show that the written report is above of excellent standard.  For a question on "What is the contribution level to the engineering field", the average score was 5.6 and this reflects that our MSEE student theses were close to outstanding quality. A minimum average score of 4.8 was found on the theses that were written based on IEEE standards.
Senior comprehensive exam	SCE Results 03-04: In Fall 2003, 19 students took the exam, 3 received Honors, 2 Failed, and 14 Passed.  The students who failed met with the chair and received copies of their exams with comments collated from the entire faculty; their teachers worked with students on idea development and students took Lab work on grammar and spelling.   In Spring 2004, 18 students took the exam, 3 received Honors, 1 Failed, and 14 Passed.  The two students who failed in October passed in April; the student who failed in April is a strong student who seemed overcome by nerves.
Exit survey	The exit survey predictably indicates that students are frustrated at the fact that EST cannot offer a greater variety of courses on a more regular basis.  Students consistently feel as though the Program has accomplished the goals and objectives outlined in section 1.
Exit exam	During the 2003-2004academic year, 51 students took the GES Exit Exam. During the Fall, 2003 semester, 14 students took the Exit Exam.  Mean score was 51.1 (73.0%), standard deviation = 6.8. During the Spring, 2004 semester, 37 students took the Exit Exam, with a mean score of 51.2 (73.1%), with a standard deviation of 7.1.  Range was 37 to 65. Exams beginning with the Fall 2003 had a different suite of questions.  A comparison between the old test and results from Fall 2003 to Fall 2004 exams will be analyzed in the Spring of 2005.  Given the general nature of the exam it is believed that students are demonstrating a proficiency in the stated GES goals.
Thesis evaluation form	Between fall and spring semesters, 40 students completed the Senior Thesis and therefore were evaluated through use of these forms. While the department has not conducted an elaborate statistical analysis, it has noted that of its five objectives the one for which the greatest number of students received either 'Poor' or 'Fair' is 'Research Skills' (Objective 2); the number was only three, low, but departmental faculty consider it worth considering as a possible indication of a trouble spot.
Graduate alumni survey	Greater than 85 % of the alumni surveyed reported the program met the expected outcomes related to the professional standards of their discipline. NP students requested more business content.
Academic Profile exam	Beth El Students (sophomores) scored above the mean on the academic profile in critical thinking. (Mean summary score was 448 and the National mean for Comprehensive Colleges and Universities was 444.1.  Benchmark Met. -return to top-

Action Plan (Improvements)   

This section should list any actions/improvements planned or implemented to the program based on data obtained through the assessment process.  Improvements fall into three broad categories: Curricular, Academic Processes, and Assessment Processes . Examples include:  

Assessment Based Improvements
Curricular		Academic Processes		Assessment Processes
	Course Content		Personnel		Objectives
	Prerequisites		Advising		Measurement Instruments
	Course Sequence		Information Sharing		Data Collection and Analysis
	Program Requirements		Technology		Personnel Involvement
	Adding/Deleting Courses

 

Curricular	Upon an analysis of student work in an entry level course, the course has been redesigned and standardized across sections to ensure that students have the skills they need to take upper-division art history courses.
Academic Processes	Past Major Surveys have pointed to an important curricular gap in the area of Middle Eastern/North African History.  This fall the department has requested a new hire in this area, a request that has been ranked first by both the LAS Budget and Planning Committee and the LAS Chairs and Directors Committee.
Assessment Processes	A lack of quantitative data in the past led us to develop two rubrics for the Qualities of Effective Counselors Skills Rubric and the Personal Characteristics Rubric.
	-return to top-

Annual Reporting within WEAVEonline includes the following standard areas in the Achievement Summary/Analysis section

Data Summary  

The data analysis, presentation and discussion should answer the following questions: 

Strengths/Progress

• Based on the data collected, are students meeting at least the minimum acceptable achievement targets for the student learning outcomes?
  
• Are students surpassing your expectations in any areas? 

Areas in need of attention

• Are there areas of concern, areas that need improvement; e.g. areas that students consistently do not meet the minimum acceptable achievement target? 
• Are the instruments producing useful information? 
• Are any changes needed to the objectives, the instruments, or the methods utilized to assess the objectives?   

Consider answering the questions for each learning objective in turn as illustrated in the following examples.   

Excerpts from data interpretations across UCCS programs:  

• For Goal 2: Overall our direct measures show that student learning has not significantly improved or deteriorated in the past year. Our indirect measure of the MBA Survey does provide a couple of areas of concern particularly in the areas of management and interpersonal skills. We need to address this in greater detail in the upcoming year. Alternatively, one of our weakest areas in the past, team skills, showed a huge improvement, from 41% to 61%.  
• In terms of student survey data, we were pleased to see the high ratings for the courses overall. For example, more than 90% of the students rated their career preparedness experience as excellent or very good. Our goal is to examine the curriculum more closely this year, so that ratings for all courses and in all Program areas are rated as excellent or very good.   The narrative comments, too, help us see the strengths of the Program and provide us with useful suggestions for Program improvement. We will be discussing these points this fall as we begin to consider an ongoing assessment protocol for the Program.  
• Data generated by LTC usage and S-Cape proficiency exams for lower level courses indicate we are meeting student-learning objectives. In as far as language acquisition is concerned, upper level courses will be evaluated as per the pre-exit S-Cape exam, which we have just instituted and will have quantitative findings by next year. Of course, the objectives dealing with literary/cultural aspect of upper division courses cannot be measured in this way, hence we are developing the portfolio, and monitoring grades and student reaction to changes in the curriculum.

-return to top-

Faculty Involvement

The assessment process cannot work without the input of faculty.  It is important that faculty participate in as many aspects of the process as possible.  It is hoped that faculty meet on a regular basis to discuss outcomes, revise aspects of the assessment plan, receive and review assessment findings, and suggest and implement improvements and receive encouragement for a job well done.

Roles & Communication   

The assessment process is most productive and valuable to the program if it is a collaborative activity involving as many tenure/tenure track faculty members of the program as possible.  Questions such as who creates the annual report, who collects and maintains assessment data, how is information regarding the assessment process disseminated, how often does the department meet to discuss assessment, and how were faculty involved in developing and finalizing potential improvements should be addressed in this section.

-return to top-

Annual Reporting   

ADVANCED ASSESSMENT

Course-Level Assessment   

Course-level assessment provides the assurance that students are gaining elements (skill/knowledge) of the student learning objectives as they progress through their course of study.  Consider the first step of this process to be the mapping of where specific objectives are met in each of the program courses.  Use the Course-Level Mapping worksheet to assist in this process.  After mapping, determine specific objectives to measure at various stages/specific courses throughout a student's course of study.

-return to top-

External Constituents   

It is important and useful for external constituents such as alumni, employers, practicum supervisors, advisory board members, community partners, potential employers, etc. to take part in the assessment process.  Every effort possible should be made to include these constituents in some part of the assessment of either student learning (are students meeting standards expected the community, do alumni feel they learned what they needed for employment) and the curriculum (do external constituents, especially future employers, feel the program is providing the training, skills, knowledge that is required for employment; do alumni have suggestions as to what areas the curriculum is satisfactory or needs improvement).   

Consider having external constituents attend and complete rubrics for class presentations or poster sessions, have them fill out evaluations of student projects, critique thesis and projects.  Ask graduating seniors for feedback through focus groups, exit interviews or surveys.  Utilize the UCCS IR office surveys or create a survey specific to your program and student learning outcomes (the Assessment Office can help with the development of assessment surveys).  Feedback of these types provides unique information that can strengthens programs and assist in the attainment of learning objectives. 

-return to top-

Last Updated:  August 4, 2010