The following statement, revised from an earlier version, was prepared by the committee on the teaching of philosophy (Randall Curren, chair) and approved by the board of officers at its 2008 meeting. Revised version published in Proceedings and Addresses of the American Philosophical Association, Vol. 82, No. 5, pp. 80-89. Original version published in Proceedings and Addresses of the American Philosophical Association, Vol. 69, No. 2, pp. 94-95.

The purpose of this statement is threefold: to clarify the concept of Outcomes Assessment, to explain and illustrate how it is used, and to address concerns regarding its implementation.

Outcomes Assessment

The concept of Outcomes Assessment (OA) reflects an increasingly widespread desire to evaluate educational programs on the basis of clear and objective criteria. Its chief intent is to develop instruments that can measure the correspondence between the claims institutions make for their programs and what they actually achieve. It has come to be applied internationally at all levels of teaching and learning, from primary and secondary through higher education, and to entire institutions, degree programs and curricula, as well as to individual courses. OA typically focuses upon three factors: the student learning outcomes of a given course (SLOs), program (PSLOs), and institution (ISLOs); the means by which these outcomes are pursued; the degree to which these outcomes have been achieved by those who complete the program.

While assessment is not new, what is new is that assessment is now associated with accountability. In pursuit of accountability, accrediting organizations across the country have required that colleges and universities create assessment plans for their academic programs. Since accreditation is required both for the provision of federally guaranteed student loans and to ensure course transferability, administrators have directed their institutions to develop comprehensive plans for assessing student learning in ways that go beyond assigning grades for performance in courses. Other factors that have contributed to the pressures for accountability are loss of confidence in conventional grading systems due to grade inflation, doubts about the effectiveness of K-12 public education, and demands by state governments, businesses, and professional sectors that graduates exhibit greater readiness for the world of work. Institutions are expected to produce assessment results that reflect students' mastery of both disciplinary content and skills. Thus, departments may be asked to demonstrate "objectively" the differences their degree programs make to the development of students' abilities through their work in the discipline. In turn, instructors may be asked to formulate specific outcomes for each of their courses (and possibly programs), and to develop instruments that measure the degree to which students attain those outcomes.

It should be noted that there seems to be some concern on the part of college administrators to emphasize that the sole purpose of using and assessing SLOs is to improve student attainment of desired competencies; they are not to be used as a way of evaluating instructors and programs. Here's a statement of assessment philosophy from a leading community college:

No individual faculty, staff member, instructional department or support program has the sole responsibility for ensuring that students will acquire one or more of the college's ISLOs. Student attainment of the ISLOs should result from the collective learning experiences they engage in during their time at the college. Therefore, the assessment of ISLOs will not be used to evaluate any individual faculty or staff member or to measure the performance of any one instructional program or support department. The use and assessment of student performance in attaining the ISLOs is to provide the college community with information needed to increase the percentage of students that acquire the core learning competencies.

On the history of the outcomes assessment movement in the United States, see Barbara Wright's article "More Art than Science: The Postsecondary Assessment Movement Today."

The following two links comprise an explanation and defense of outcomes assessment:

• "Assessment From the Ground Up" (Donna Engelmann, Inside Higher Ed, 14 August 2007)
• "Encouraging Assessment From the Ground Up" (Donna Engelmann, Inside Higher Ed, 28 September 2007)

OA in Practice

To some extent, philosophy courses and programs, as traditionally conceived and practiced, have defined student learning outcomes and means of assessing these outcomes. These typically include specific skills that students and majors in philosophy should acquire and refine, such as competence in applying the critical thinking standards of clarity, accuracy, relevance, depth, breadth, and coherence. Most courses and programs also identify specific knowledge that students and majors should acquire, such as knowledge of the history of various philosophical debates, the main trends, traditions, concepts, terminology, etc. Syllabi and course outlines typically contain descriptions of what students are expected to learn in a course, the kinds of skills and competencies they are expected to demonstrate, and the nature of the instruments that will be used in assessing their demonstration of knowledge, skill, and competence. Outcomes normally are measured in philosophy courses by examinations and other written assignments that test whether students have mastered the objectives appropriate for the course and its level. Departments frequently establish curricula based on a consensus of their instructors regarding minimal standards of competence appropriate to given degree programs; and comprehensive examinations, written and oral, are sometimes used to decide whether students have attained that level of competence.

Currently, however, most philosophy courses and programs do not address or formulate student learning outcomes in ways that satisfy all of the expectations typical of the OA movement. Consider what are perhaps the main three expectations of OA: First, for each course, be it philosophy or physics, there are several outcomes that a student of the course should have demonstrably mastered (within a range from "inadequate" to "excellent") by the end of the course. Second, the student learning outcomes for each kind of course in a particular department should be identical irrespective of who teaches the course. Accordingly, a student learning outcome should be articulated in such a way that the outcome can be achieved in a variety of ways, allowing for academic freedom, choice of text, and assessment methods. For example, it is unreasonable to have as a course student learning outcome (SLO) for Introduction to Ethics, "distinguish between Pufenforf's and Kant's views of moral duty," as this would require all Introduction to Ethics courses to focus on this distinction. A more appropriate outcome would be: "identify the distinguishing features of the moral theories studied in the course." Third, it should be possible to link or "map" the student learning outcomes to program and institutional student learning outcomes. For example, the SLO "identify the distinguishing features of the moral theories studied in the course" may be mapped onto the PSLO "demonstrate knowledge of the main concepts and theories of ethics." In turn, this PSLO may be mapped onto the ISLO "define the issues, problems, or questions." This is meant to ensure that each course itself furthers the aims of the program and the institution of which it is a part.

The following is an actual course description that is tailored to meet the expectations of OA:

INTRODUCTION TO ETHICS

In this course we will read classical and contemporary writings on such matters as good and evil, relativism, happiness, virtue, egoism, moral education, abortion, and social policy. We will seek to answer, using critical reasoning, a series of questions about these issues as raised by the course readings. In addition, we will engage each other in sustained discussion of these issues. Listed below are the outcomes a successful student will attain by the end of this course in ethics:

1. Identify and define key philosophical terms studied in the course.
2. Distinguish among the moral theories studied in the course.
3. Apply moral theories to specific moral issues.
4. Identify major points and arguments of an essay in moral philosophy.
5. Critically analyze and evaluate moral arguments.

Notice how each of the outcomes specified here takes an active verb: identify, define, distinguish, apply, analyze, and evaluate. Student learning outcomes refer to what students can do; they are achieved and measurable competencies, in contrast to the objectives of the instructor, program, or institution. What the instructor, program, or institution intends to achieve can be unrealistic or simply too narrow, and students can fail to achieve them. Also, it is important to note the number of articulated SLOs. Though it is perhaps tempting to add more SLOs to this list, four to six SLOs for each course is quite common and frankly much more realistic. For instructors are asked not only to assess to what extent these outcomes have been met by each of their students, but also they are often asked to "map" their SLOs to program student learning outcomes (PSLOs) and institutional student learning outcomes (ISLOs). Each SLO should map onto at least one PSLO and at least one ISLO. So, for most instructors it is usually too unwieldy to implement more than six SLOs.

A good example of PSLOs can be found in the outcomes adopted by a Canadian program in Ethics. http://www.edu.gov.on.ca/eng/document/curricul/philoso/philoso.html.

Learning is a complex process, and philosophical learning is no exception. There are cognitive, affective, and social dimensions, for learning involves not only knowledge and understanding, but also values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom.

COGNITIVE (Knowledge and understanding)

1. Demonstrate knowledge of the views of some historically important moral philosophers (e.g., Plato, Aristotle, Kant, Nietzsche, Singer).
2. Demonstrate knowledge of the main concepts and theories of ethics (e.g., egoism, altruism, rights, duties, utilitarianism, Kantianism, virtue ethics).
3. Apply these concepts and theories to case studies and contemporary moral issues.
4. Articulate an understanding of connections between reason and feeling and between cultural and intellectual traditions.
5. Express conclusions with awareness of the degree to which these conclusions are supported by evidence.

AFFECTIVE (Skills)

6. Demonstrate imaginative, creative, and reflective abilities by articulating philosophical insights.
7. Present effectively in writing an extended argument on a topic of ethical importance.
8. Articulate counter-arguments to one's own position.
9. Ask questions to clarify problems further.

SOCIAL (Values)

10. Demonstrate openness and intellectual humility by approaching situations involving a conflict of views in a spirit of inquiry.
11. Identify and reflect on values through analysis of case studies in such areas as justice, abortion, and the impact of humans on the environment.
12. Reflect on one's intellectual and intuitive responses to issues concerning ethical values.
13. Demonstrate increasing awareness of the complexity of issues and of the necessity of examining issues from many different perspectives.

Such a philosophy program description in the OA mold will go on to specify what assessment techniques will be used to measure whether students have in fact matured sufficiently along these cognitive, affective, and social dimensions. These might include a comprehensive written and/or oral exam, a senior thesis or project, internship, narrative evaluations from faculty, and self-assessment. This might also include completion of a specific course or series of courses within the department's offerings.

(More information on OA techniques and expectations can be found from the Association of American Colleges and Universities.)

It is no surprise that the above examples of SLOs and PSLOs were taken from different institutions, for the given SLOs are not clearly or meaningfully connected to the PLSOs that address learning along the social dimension. Even so, not only can an instructor add an SLO that more clearly addresses competencies along the social dimension, but also there may be other courses or opportunities (e.g., internships, senior projects) in her philosophy program that better address such competencies. There is no expectation that each course within a given program address all the PSLOs.

Naturally, institutional student learning outcomes tend to be even more numerous. Differences in program outcomes across disciplines must be accounted for. And again, there is no expectation that each program within a given institution address all the ISLOs. The following example of ISLOs is from a community college in California:

CRITICAL THINKING, PROBLEM SOLVING, CREATIVE THINKING

1. Define the issues, problems, or questions.
2. Seek, collect, and analyze data and relevant information including alternative approaches.
3. Differentiate among facts, opinions, and biases.
4. Synthesize and generate solutions, and identify possible outcomes.
5. Use evidence and reasoning to support conclusions.

COMMUNICATION

6. Comprehend and interpret text.
7. Create documents that communicate thoughts and information appropriate to the given context, purpose, and audience employing the conventions of standard English.
8. Organize ideas and communicate orally in a way appropriate to audience, context, and purpose.
9. Receive, attend to, interpret, and respond appropriately to verbal and/or nonverbal communication.
10. Recognize and interpret images, graphic displays, and other forms of observable communication.

QUANTITATIVE ANALYSIS AND SCIENTIFIC REASONING

11. Apply quantitative skills to the interpretation of data.
12. Use graphs, symbols, and mathematical relationships to describe situations.
13. Apply mathematical concepts to solve problems.
14. Explain/articulate the scientific method to test theories, explanations, and hypotheses.

SOCIAL, CULTURAL, ENVIRONMENTAL, AND AESTHETIC PERSPECTIVES

15. Describe how the interactions among social, economic, political, cultural, environmental, and historic events affect the individual, society, and the environment.
16. Explain how culture influences different beliefs, practices, and peoples.
17. Recognize fine, literary, and performing arts as essential to the human experience.
18. Identify the social and ethical responsibilities of the individual in society.

INFORMATION, TECHNOLOGY, AND MEDIA LITERACY

19. Select and evaluate the accuracy, credibility, and relevance of information sources.
20. Use technology effectively to organize, manage, integrate, create, and communicate information, and ideas.
21. Evaluate critically how media is used to communicate information through visual messages.
22. Identify the legal, ethical, social, and economic rights and responsibilities associates with the use of media.

PERSONAL, ACADEMIC, AND CAREER DEVELOPMENT

23. Develop, implement, and evaluate progress toward achieving personal, academic, and career goals.
24. Demonstrate personal responsibility for choices, actions, and consequences, including but not limited to, attending classes, being punctual, and meeting deadlines.
25. Demonstrate the ability to work effectively in a group setting.
26. Demonstrate the ability to identify and use appropriate resources.

How should one "map" the five SLOs of the above Introduction to Ethics course with the above 25 ISLOs? The following guideline is typical: map a course SLO to one or more ISLO competencies only if instructors are directly measuring those competencies in this course and the connection between course SLO and ISLO competency is both clear and meaningful. Following this guideline yields this mapping:

If one then mapped the previously listed 13 PSLOs of the Ethics program with these ISLOs, the results would be something like the following:

Concerns regarding OA

1. OA threatens to be an exercise in measuring what is easy, rather than a process of improving what philosophy instructors (and presumably even students) really care about. If philosophy courses and programs do satisfy the enormous pressure from various sources to find objective measures of learning outcomes, then there is a real danger that OA imperatives will create pressures to tailor the teaching of philosophy to things that admit of "before and after" measurement, to its serious detriment. So, for example, students who take philosophy courses dealing with different ways of thinking about such problems as the idea and existence of God, the relation of our minds to our bodies, the nature of truth, the conditions and limits of human knowledge, or the status of moral principles and concepts, should be more sophisticated in their thinking about these issues after taking the courses than they were at the outset. This should be discernible in both discussion and written work. It is only on the most superficial level of treatment of any such topics, however, that one can find specific matters admitting of before-and-after measurement (e.g., being able to identify, define and distinguish different arguments for the existence of God, conceptions of truth, types of knowledge, or different moral theories, or knowing who said what about them in the history of philosophy). And to make instruction in such matters the focus of philosophical education (in order to yield dramatic before-and- after results) would be to reduce it to a caricature of the development of any real sophistication in students with respect to these issues. The basic aim of education in philosophy is not and should not be primarily to impart information. Rather it is to help students learn to understand various kinds of deeply difficult intellectual problems, to interpret texts that address these problems, to analyze and criticize the arguments found in them, and to express themselves in ways that clarify and carry forward reflection upon them. The worry is that these kinds of abilities are not amenable (though others might be) to patterns of outcomes measurement typical of OA. It is not to be expected that student progress in philosophy can either be specified to a degree beyond what is already possible by means of an essay examination or a term paper, or given a purely quantitative expression. It is essential that those values inherent in and specific to the process of teaching and learning in philosophy not be lost. In short, the adoption of OA in philosophy might seem to undermine, rather than improve, the quality of instruction.
   
   Yet many, if not most, of those who use OA do not find this to be the case. First, the extent to which an outcome in philosophy is easy to measure seems to have little to do with the degree to which it is worthy of measurement. Learning outcomes in traditional symbolic logic courses are often in the "easy to measure" category but are certainly worth caring about. Second, there may not be such a large gap between the easy to measure and the difficult (some would say impossible) to measure outcomes. The above sample lists of SLOs and PSLOs can be used to illustrate this. At first glance, most would undoubtedly consider SLO 2 and PSLO 2 (both of which refer to the demonstration of knowledge of the main concepts and theories of ethics) to be amenable to "easy," before-and-after measurement, since it is content and technique specific. Two straightforward exams, a beginning of term exam and an end of term exam would seem to suffice. Contrast this learning outcome with PSLO 12 ("reflect on one's intellectual and intuitive responses to issues concerning ethical values"). Surely most would initially register this as difficult (perhaps impossible) to measure in a way helpful to OA. But this is to exaggerate the differences between outcomes 2 and 12. For in either case a rote answer might be given; there are certainly instructors who discuss in class "different intellectual and intuitive responses concerning ethical values." The fact that a student holds an idea that others have held before her surely cannot be used against her when grading. In this sense, outcome 12 is similar to outcome 2, for how can the instructor tell in grading an exam or essay whether or not a student truly understands "the main concepts and theories of ethics" or is just recapitulating them on paper? This, however, suggests a solution that is often considered integral to the proper use of OA. Careful practitioners of OA use the student's self-assessment along with evidence from her performance in essays and exams to measure such things as attitudinal changes, for instance, a commitment to using philosophical methods and ethical concepts in resolving issues of personal and professional importance to the student. Such an approach greatly increases the chances of measuring accurately both outcomes 2 and 12. It also serves to avoid easily quantifiable measures of assessment that do not adequately reflect the complexity of student learning.
   
   None of this is to say that all outcomes are or should be measured in similar ways. It is not obvious, for instance, that self-assessment is needed in logic. In fact, different outcomes may require different kinds of instruments of measurement. But this is surely not equivalent to saying that some outcomes are easy to measure and others practically impossible.
   
   
2. At the time of this writing, there does not seem to be any rigorous research comparing different kinds of instruments for observing and measuring learning outcomes peculiar to philosophy. Controlled studies, where, for example, the same philosophy instructor articulates outcomes and regularly performs assessments in one of her ethics classes and not in another ethics class, do not seem to exist. However, it appears that there is much anecdotal evidence that outcomes like the ones expressed above can be achieved and demonstrated in a wide variety of learning activities. Learning activities range from written exams administered throughout the term, class discussions and quizzes, questions solicited by the instructor, group work on pre- selected or limited topics, essay assignments graded with a departmental rubric, and student self-assessment. That being said, the APA sees a need for further empirical research into the usefulness of different kinds of assessment instruments for measuring the outcomes of concern to philosophy courses and programs.

Recommendations

Note the above use of the phrase, "careful practitioners of OA." Certainly OA can be used in a careless and damaging fashion, for instance, where only one kind of measurement is used or where the outcomes are entirely along the cognitive dimension while ignoring the affective and social dimensions. OA must not be treated as an end in itself, but rather as one (albeit important) means for educational improvement. Educational values should guide not only what instructors choose to assess but also how they do so, and those values can be made clear to students through the methods of OA. Assessment should be an ongoing process and not episodic, especially for majors continuing beyond the term. In the spirit of continual development, student progress toward the intended outcomes should be monitored. And, importantly, the assessment process itself should be regularly assessed. OA must also take into account the peculiarities of each discipline to which it is applied. The APA calls upon administrators to recognize that philosophy is fundamentally a matter of the cultivation and employment of analytic, interpretive, normative and critical abilities. Learning outcomes and assessment methods must be devised accordingly. It is recommended that special consideration should be given to the means of assessment already in place at an institution.

The APA recognizes the interest of public agencies in establishing ways of assessing the success of colleges and universities in carrying out their educational missions, and accomplishing their objectives. It seems possible to create assessment instruments for both students and programs that satisfy administrators yet at the same time avoid easy measures that do not sufficiently mirror the complexity and special nature of student learning in philosophy. The concept of Outcomes Assessment may be of some help in achieving these ends, but it must be applied carefully.

Links

• Clark College's Outcomes Assessment Handbook
• Chico State's Critical Thinking Assessment Project (not philosophy-specific)
• Gustavus Adolphus College's assessment plan
• University of Michigan - Flint's after-graduation assessment
• Indiana University South Bend's philosophy assessment plan and annual report.
• Oklahoma State's document on philosophy outcomes assessment
• Report on assessment methods from the Spring 1998 issue of the APA Newsletter on the Teaching of Philosophy (APA Newsletters, Volume 97, Number 2)