FAQs for General Education Assessment
Sampling Techniques - How does a course sample?
Each of the following sampling techniques may be appropriate for assessment of general education courses. A department should indicate how they are sampling in the course renewal documents.
Simple random sampling – randomly select a certain number of students or artifacts to review. For example, the History Department might use a list of all students enrolled in History 1101 (all sections) for a particular semester and use Excel, SPSS, or another method to randomly select 10 students to submit an artifact to review.
Systemic sampling – select the nth (5th) or grouping starting with nth (5th - 10th, etc.) from class rosters or from an organized list of artifacts. For example, faculty in First Year Seminar might review assignments from every student who is 7th on the (alphabetical) roster for each section.
Cluster sampling – randomly select clusters/groups (sections for general education courses) and evaluate the assignments of all students in those sections. For example, First Year Writing courses might randomly select two sections from all the sections of First Year Writing. Faculty members would evaluate writing samples for every student in those two sections.
Sample size – What is an adequate sample size?
A good general rule for an adequate sample size for assessment of general education courses is 10 students or 10% of the students, whichever is greater. This would mean the sample size should be 10 for courses with 100 or less students enrolled and 10% of all enrolled students for courses greater than 100 students. There are good reasons to make adjustment to this general rule. For example, a department may sample less than the 10 or 10% rule if the artifacts being assessed are particularly long (an honors thesis for example or an entire portfolio of work). On the other hand, the department may be able to sample more than the 10 or 10% rule if they have a large number of faculty evaluating artifacts. Obviously, if there are less than 10 students enrolled in a course, it would be impossible to sample from the population unless the department chooses to examine artifacts across multiple semesters.
What is the process of assessment?
Assessment for a general education course starts with some basic questions:
1) What General Education goal is related to this course?
2) What do faculty want students to take away from this course related to that goal? (learning outcome)
3) What evidence do we need to gather to know that the course was successful? (measure)
4) How will faculty define success? (numeric criterion)
5) What have faculty learned about teaching and learning in this course? (result)
6) If faculty are not satisfied with the results, what can they do about it? What improvements can be made to the course? (action plan)
What is an artifact?
In assessment, an artifact is a student product (assignment) used as evidence of student learning.
- Student papers
- Oral reports/presentations/demonstrations
- Lab reports
- Web board/electronic discussion comments
- Pre- and post-tests
- Evaluation rubrics
- Reflections on what you have learned
What types of assignments are assessable?
There are many options for an assessable assignment. The most important part in determining if an assignment is assessable for the purposes of general education is if the assignment allows students to demonstrate that they have met one of the goals of general education. Assessable assignment types include papers, writing reflections, posters, performances, lab reports, oral presentations, videos, PowerPoint or other electronic presentations, etc.
Another consideration is whether a reviewer other than the course instructor could rate the assignment to determine if it adequately represents learning related to a General Education goal. Even though a video, collage, or other visual presentation might be the chosen assignment to showcase students’ learning in a course, often a written reflection is needed for an outside reviewer to be able to apply a rubric to the assignment. In addition, a reviewer would benefit from seeing the instructions students were given for the assignment.
How do you determine if an assignment is assessable?
Is there an explicit connection between the assignment and a student learning outcome related to one of the goals of General Education?
Can students adequately demonstrate their achievement of a particular learning outcome related to a General Education goal with this assignment?
Check out the DQP Assignment Library for examples of assessable assignments: http://www.assignmentlibrary.org/search
What is a rubric?
A rubric is a scoring tool developed to help evaluate qualitative data or assignments. A rubric consists of a specific set of criteria to be rated and a rating scale, along with what is needed to achieve each rating for each criterion. Ratings might range from 1 to 4, unacceptable to excellent, or undeveloped to exemplary.
For assessment of General Education, we recommend departments consider using the VALUE (Valid Assessment of Learning in Undergraduate Education) rubrics from the Association of American Colleges & Universities (AAC&U). The rubrics were developed by teams of faculty experts from across the country and have been vetted considerably. The rubrics can be downloaded here: http://www.aacu.org/value-rubrics
Of course, faculty may choose to make modifications to one of the VALUE rubrics to fit their needs or may decide to create their own rubric. For help creating a rubric from scratch, go here: http://rubistar.4teachers.org/index.php
How might a department organize for assessment of general education courses?
If there is not an already existing curriculum or general education committee within the department that can take on the task of organizing for assessment of general education courses, establish a small committee to make recommendations about assessment methods, vet rubrics, communicate with faculty across sections about how to submit artifacts, review artifacts, and discuss action plans as needed. While one faculty member may coordinate these efforts, it is important that the responsibility of assessment does not fall on a single person. One of the most important part of the assessment process is the discussions assessment should generate among faculty about how to improve teaching and learning.
How should a department collect artifacts?
There are a variety of ways that a department might choose to collect artifacts for general education assessment. One option is that a faculty member may be designated as the person to collect artifacts from all course instructors for that course. Instructors may give hard copies of those artifacts, e-mail copies of the artifacts, or place all of the artifact into an assigned location such as a shared drive or AsULearn site.
Another option for collecting artifacts is an electronic portfolio, or a digital collection of a student’s work. Appalachian’s electronic portfolio, APortfolio, provides not only a place for students to collect and reflect on their work but it also provides a place where faculty can upload rubrics and rate students’ work. The course instructor can rate assignments for a course within APortfolio, but a team of reviewers could also be given access to rate students’ work. For more information about APortfolio, visit: http://aportfolio.appstate.edu/
Once artifacts have been collected, then what?
After artifacts have been collected, faculty members begin the work of applying a rubric to the assignments and rating them. Especially if this is the first time a rubric is being used, we recommend that the faculty members sit in a room together as they conduct their ratings. Each faculty member reviews the same artifact and rates it. Then, the faculty members compare their ratings to see how they match up. If the ratings are different, the faculty members should discuss why they each rated the assignment in the way they did. The purpose of this dialogue is to test the validity and reliability of the rubric. Faculty may decide that the rubric may need to be altered to clarify the criteria for each rating. In addition, faculty members may need to come to an agreement about what constitutes a score of a 1, 2, 3, and 4. In addition, faculty may discuss why they are unable to rate certain assignments and make future recommendations for what constitutes an assessable assignment.
What is an action plan?
When assessment results for the course do not meet the predetermined standard (numeric criterion) set by faculty in the department, then an action plan is needed. An action plan is a department’s response to student performance not being up to expectations. What changes will be made to the course or curriculum to try to improve student learning for this particular outcome? Taking appropriate action when results are not at the level that a department would like is the most important part of assessment.
What does it mean to “close the loop”?
Closing the loop in assessment happens when a program makes changes in response to a criterion that has not been met and then measures the outcome again to see if the changes had the intended impact.
How do we report on assessment?
Although we are not using Xitracs as a repository for course-level assessment, we recommend that departments use the same reporting format that is used for program-level assessment.
- General Education Goal
1.1 Learning Outcome
1.1.1 Assessment Method
18.104.22.168 Numeric Criterion
1.1.2 Action Plan
These course-level assessment reports should be collected by departments annually. When a course is up for renewal, the department should add these assessment reports to the course renewal form.