Mar. 16, 2004, Assessment Committee Report
The Assessment Committee met six times throughout the academic year and sponsored a writing assessment workshop in January to train new faculty volunteers on how to use the KCC Writing Assessment Rubric. The Assessment Committee website continued to grow with the addition of new web links related to assessment and KCC data. The website contains minutes of Assessment Committee meetings, results of student surveys, a listing of committee members, and links to web resources on assessment. The URL is: http://kauai.hawaii.edu/moxie/admin/acommittee/index.shtml
Professional Development - Three faculty members (Rick Randolph, Shelley Konishi, Molly Summers) representing the Language, Arts & Humanities Division and the Business Education Division attended the AAHE (American Association for Higher Education) Assessment Conference in Seattle, Washington from June 21-24, 2003.
Spring Semester Writing Workshop
Monica Stitt-Bergh, of the UH Manoa Writing Center gave our faculty a second writing assessment workshop in January 2003 (the first was in January 2002). Eleven faculty members were trained in the scoring rubric, which brings the total number of current faculty trained to 20, plus one administrator. [Unfortunately, 3 of the faculty members who were trained are no longer working at KCC.] The breakdown of faculty by division, who have been trained in holistic writing scoring techniques :
Business Education (BUS)
Language, Arts, and Humanities (LAH)
Science and Math (SAM)
Health and Early Childhood Education (HED)
Mary Jo Sweeney
Trades & Technology
On May 15, 2003, the Assessment Committee and volunteers scored 37 A.A., A.S., and A.A.S. degree graduating-student papers that were submitted to instructors in a variety of courses from the Liberal Arts, Business, Health, and Trade-Tech Divisions. Scorers read the assignment instructions given by the instructor, then read the student paper and scored it using a writing rubric adopted by the committee with a scale ranging from 4 (highest evaluation) to zero (lowest) on a number of writing components. Two scorers read each paper and the scores were added together to create a composite score.
Out of a possible composite score of 8, the average scores of the papers by degree:
A.A.S. - 5.09 (2.54 on a 4.0 scale) - 12 papers
A.S. - 5.83 (2.91 on a 4.0 scale) - 12 papers
A.A. - 5.61 (2.80 on a 4.0 scale) - 13 papers
The inter-rater agreement (the extent of a match between the raters in determining the score for each paper) demonstrated a rather low degree of consistency in the scores assigned by the individual raters and should be addressed in the next training session:
13 samples were rated the same (35% of the total sample)
17 samples were rated 1 point apart (46%)
7 samples were rated 2 points apart (19%)
Spring Semester 2003 Student Writing Assessment Spring 2003 volunteers:
Comments from the faculty volunteers about the writing samples and suggestions for campus wide initiatives to emphasize written communication skills:
1. The college should use the same writing rubric for assessing student writing across all divisions and should stress the use of communication skills across the curriculum. The rubric should be sent to all instructors.
2. The college should require writing intensive courses for certain programs (Liberal Arts).
3. At Convocation there should be workshops for faculty members to break into smaller groups to learn how to "sharpen" their writing assignments. Everyone could bring in one assignment to work on. The English department could also offer workshops on designing writing assignments.
4. The writing samples were difficult to compare because the writing assignments were so different, although it was noted that there was some improvement in the instructors' directions for the assignment since last year's writing assessment. For writing assessment purposes the directions/prompts for the assignment need to be very clear and more uniform across the sample. There needs to be common elements in all assignments. There should be separate scoring sessions for the degrees (A.A., A.S., A.A.S.) because the writing levels are so different.
The 2002-2003 Assessment Committee's recommendations for future goals and activities include:
· At least two members from each division should be trained in holistic scoring each year until all faculty members have been trained.
· The Committee should continue to investigate the possibility of obtaining student-writing samples at the point of entry to the college in order to effectively measure progress in writing.
· The Committee should develop a new mission statement, establish clearly defined goals and develop a timeline for assessment activities.
· The College should explore the possibility of creating an "Assessment Day" as an annual event, which would involve faculty, staff and students, and which would focus on student learning outcomes.