Learning outcomes are one part of an educational strategy to promote greater accountability, equity, access and quality, four themes which predominated in discussions that led to the recommendations of the Vision 2000 Report (Ontario Council of Regents, 1990). Learning outcomes describe the academic standards against which learners and college programs will be assessed and evaluated and academic credit will be granted. Several projects currently being undertaken by the College Standards and Accreditation Council (CSAC), including generic skills, general education, vocational training requirements and program review leading to accreditation of programs rely on learning outcomes as the basis for defining program standards. These outcomes-based projects are part of a timely and valid college reform initiative whose ultimate success depends on an accurate interpretation of terminology, understanding of and agreement on the aims of reform and clear explanation of the philosophical origins of the principles behind the reform strategies.
The popular wisdom that “everyone resists change” is not always a fair explanation of the failure of reform initiatives. In this period of economic restructuring and heightened emphasis on the role of education and training to Canada's future wellbeing, there is considerable evidence in the colleges of openness to change and a climate for innovation. This article attempts to demonstrate that learning outcomes represent a strategic instrument of reform in Ontario. It will also clarify aspects of learning outcomes that should be understood and agreed upon in order to support meaningful, effective and constructive college reform. Some definition of college-level learning that addresses the training demands of the information age will also be proposed.
We are told that successful college graduates of the future, in addition to having superior vocational and generic skills, will have to be effective lifelong learners and adaptable employees who are able to update their skills and change jobs or careers several times in order to keep up with unremitting change. Global competitiveness requires involved, aware, conscientious citizens who recognize that their society's prosperity depends on individual initiative, innovativeness and creativity and ability to collaborate. As reform instruments, learning outcomes reflect a growing recognition that a global society is more competitive and, in order to prosper, has higher performance expectations of workers and citizens.
Learning outcomes are statements of standards which describe the expected role performances that learners must demonstrate before they graduate from a program. These statements are intended to serve as exit criteria for evaluating learners' achievement of significant, meaningful, verifiable and transferable role performances related to vocational and personal life roles. As exit standards, learning outcomes therefore describe the culminating results or “products” of learning. The “products” of learning refer to internalized, significant applications of knowledge and abilities, called role performances, that reflect understanding, integration and synthesis of the various aspects of the learning to be achieved. They do not represent the sequenced or levelled stepping stones, sometimes called benchmarks, which comprise the inherent stages in the development of a role performance.
College level learning outcomes describe broadly-based vocational and generic abilities that may realistically be attained by learners by the end of a college program.
College level learning refers to internalized integration of practical know-how with the ability to explain why certain practical applications are appropriate. It also implies conceptual understanding of factors and issues related to practical applications that promote investigation, problem solving and innovation.
Course learning requirements which lead gradually toward the achievement of program learning outcomes are benchmarks or stages in the learning process, not learning outcomes. During these early stages in the development of outcomes-based approaches, some faculty are using terms such as course learning requirements or even course learning outcomes to describe the achievement of a knowledge base, as in, “demonstrates knowledge of the stages of physical development from birth to age six”, or, a discrete skill, as in “plans play and learning activities that promote physical development”. Course learning outcomes or requirements are being used, in some cases, to define what the curriculum content will be and what discrete skills should be mastered. Eventually, course learning outcomes or requirements should be used to define only the essential, significant, transferable performances to be achieved by all learners before they receive credit for a course, module or unit of learning.
When they are philosophically congruent with the aims of educational reform, learning outcomes represent culminating, significant, transferable learning that occurs at the end of an extended period of learning. They are usually achieved after many types of learning experiences which facilitate the integration of knowledge, skills, concepts, and enduring habits of mind, sometimes referred to as “dispositions”. These higher level role performances are not necessarily dependent on one specific knowledge base nor on the mastery of an entire set of particular, discrete skills. They may be acquired through exposure to several different content bases, by practising a variety of skills in various contexts and through many types of teaching and learning methods. Therefore, learning outcomes should not dictate specific curriculum, particularly course content, teaching methods and assessment practices.
Learning outcomes are the cornerstones of outcomes-based education, a term used to describe curriculum models that support learner progress toward and eventual achievement of exit standards. In Ontario, program exit standards are being established Provincially, over time, by each program's internal and external stakeholders. Statements of program learning outcomes, which will be an important part of the program standards documents, have to reflect the vocational and generic abilities that enable graduates to perform vocational roles which are currently viable and anticipate vocational, economic and social change.
A paper entitled, Guidelines to the Development of Standards of Achievement through Learning Outcomes (CSAC, 1994) defined learning outcomes this way:
“Learning outcomes represent culminating demonstrations of learning and achievement. They are not simply a listing of discrete skills, nor broad statements of knowledge and comprehension. They describe performances that demonstrate that significant learning has been verified and achieved by graduates of the program....Readily amenable to system-wide application and review, with an increased emphasis on learner achievement of clearly defined performances, learning outcomes provide support for both of the fundamental goals of reform, namely, accountability and accessibility.”
Educational philosophy that puts the needs and interests of learners and the achievement of predetermined standards at the centre of the educational process provides an important theoretical framework for the establishment of learning outcomes and the implementation of outcomes-based education and training. The development of human potential as a means to the achievement of economic and social goals is an important cornerstone of this constructivist philosophy which borrows from cognitive psychology and the information-processing sciences. At the heart of this philosophical school is the understanding that through education learners “equip” themselves with transferable abilities which enable them to function effectively as workers and citizens in a changing world.
Learning theories derived from this philosophical position recommend learner-centred teaching which accounts for a variety of learning styles and individualized timeframes for learning. They also promote a range of delivery services that cater to various learning and lifestyle needs. Practice and eventual achievement of broadly-based role performances usually demands hands-on, active, experiential practice and learning experiences that address real-life performance expectations in authentic contexts. Standards-referenced assessment and evaluation practices provide for learners to be assessed against learning outcomes that describe expectations of performance in vocational- and life-roles.
The constructivist philosophical position and outcomes-based education contrast with the industrial age approaches of this century which have been influenced by the behavioral sciences. These traditional approaches have emphasized the cultural transmission of prescribed content and content-based evaluation using classroom tests and examinations. These standardized training approaches which were used to train workers for specific jobs and stable vocations met a need during the industrial period when approximately two-thirds of the workers in our society were needed to perform routine production jobs. The assumption was that most routine production jobs could be dissected into predictable, discrete and quantifiable tasks. Many of these jobs demanded a limited range of skills in highly-supervised, hierarchically-organized workplaces that did not change much from decade to decade.
To meet the specific demands of the industrial economy, colleges made large investments in training models based on detailed lists of behaviours related to specific jobs and vocations. Terms such as DACUM, terminal performance objectives and competency-based education described these objectives-driven approaches. These training models, when effectively implemented, served learners and colleges during years of economic prosperity, in a highly industrialized economy where jobs were plentiful and stable. Industrial training models were also relevant to a time when human resource development (employee retraining) for expanded or different roles in organizations that were restructuring was not a major concern for employers.
Traditional, content-centred, task-based training approaches have limitations in the information age which demands large numbers of knowledge workers who are able to function effectively in technologically-based, new economy workplaces. Nor do traditional approaches respond to times which require adaptability to change, teamwork, and a broader range of transferable vocational and generic skills, including the ability to think critically and be technologically literate, in order to fulfill several work roles in an organization and create new types of employment opportunities.
Another potential threat to college educational reform may be the tendency to link “learning outcomes” with total quality improvement (TQI) or total quality management (TQM) strategies. The risk here is that the importance of process, e.g. alternative forms of delivery such as technologically-mediated instruction and larger class sizes facilitated by “remote” teaching as a way of reducing costs and improving organizational cost effectiveness, may take precedence over the achievement of standards and the protection of quality in teaching and learning. Some efforts at college reform which focus on “teaching to large classes”, computer-based delivery services, and independent studies allow training institutions to serve larger numbers of learners through alternative delivery modes, sometimes at lower cost. While these are laudable reform efforts in themselves, to be educationally viable they have to maintain the rigour and quality of learning intended by the current reform initiatives by ensuring that graduates meet the program exit standards.
TQI projects which borrow “learning outcomes” to disguise management goals designed to improve the “bottom line” for organizations misrepresent and undermine the true purpose of learning outcomes as instruments of educational reform. Learning outcomes should not be used as an excuse to force widespread process-oriented changes upon a college system without proper regard for their role as exit standards, accountability mechanisms and proof of achievement. Any attempts to use “learning outcomes” as code words for quality improvement in a streamlined, profit-centred organization distract educational reform efforts from their central purposes which are to improve quality, access, equity and accountability. Reform efforts which make heavy demands on front-line workers (i.e. professors) to produce better results but withhold the necessary resources for improved performance should not be linked with program standards and outcomes-based approaches. Professional development for teachers, manageable learner-teacher ratios and provision of environments and resources which support effective learning and evaluation practices are essential to the success of reform.
The changes in teaching methods, management techniques, organizational structures, curriculum models, and evaluation practices needed to support an outcomes-based system will take time, dedication and human resource development in all sectors of college organizations. These changes, however, do not call for the adoption of just one curriculum model or one particular management theory for the whole college system. Colleges will likely continue to do as they have done successfully in the past, that is, develop models, strategies, techniques and instruments which work best within their own organizational and geographical environments. A wide range of curriculum and management models can peacefully co-exist with the implementation of Provincial exit standards for college programs. What counts for the future is that all college programs be able to demonstrate that all of their graduates have achieved all program exit standards expressed as learning outcomes.
Learning outcomes as the standards of performance for programs and learners have become the pivotal concept to be fully understood and integrated into all college learning activities. How they are stated, the essential role performances they represent, the levels of performance they describe, how they are to be used as standards for verifying the performance of learners and the integrity of programs, and what guidance they provide for making curriculum decisions, have become the key questions to address.
Learning outcomes should be used as instruments that add value to the already well-regarded training roles of Ontario colleges. They should explain more clearly than before what it is that colleges do and how they contribute to education and training in Ontario. Learning outcomes should make explicit for learners, teachers, employers and the general public the role performance expectations of college graduates in a more sophisticated, technologically-based workforce, in specific vocations and in everyday life. If learning outcomes succeed in describing clearly the ends to which the teaching and learning process is expected to lead, they will also help to define, support, guide and enhance curriculum design, including teaching methods and evaluation practices.
Given the different philosophical orientations and training goals of constructivist, outcomes-based models versus content-centred, skills- or competency-based approaches, attempts to use the term learning outcomes interchangeably with “competencies”, “skills” or behaviours are counterproductive. Grouping “competencies” which describe discrete, observable, measurable behaviours into broader “competency areas” which are deemed to be the same as “learning outcomes” alters the definitions of all three terms. Educational reform efforts are thereby reduced to a confusing change in the definition of terms without the prerequisite philosophical understanding and congruent changes in teaching and learning methods that are essential to successful, relevant, new educational practice. When terminology and definitions change but values and educational practices do not, all efforts at meaningful reform fail.
Changing times require substantial shifts in curriculum design such as movement away from norm-referenced to standards-referenced evaluation models and from content-centred direct instruction in classrooms to greater emphasis on activities-based, experiential learning in authentic contexts. New teaching strategies, alternative delivery services and improved assessment and evaluation practices, all of which recognize the obligation of colleges to serve many more learners in a variety of new ways have to be developed. A pedagogical plan is essential in order to break down broadly-based role performances expressed as program learning outcomes into bite-sized chunks. These incremental steps in the acquisition of learning outcomes should then be attached to progressively more complex learning activities that lead to the eventual achievement of each program learning outcome.
Ultimately, learning outcomes should serve as reliable indicators and tangible proof of the high quality of education and training services provided by the community colleges to the citizens of Ontario. They should therefore be used to ensure that quality is maintained in the delivery of education and training services which promote access, equity and accountability. Access is facilitated by providing for the public clear descriptions of program learning outcomes which may encourage people who have learned through experience and/or in other learning situations to apply for credit through prior learning assessment (PLA). This assessment mechanism differentiates the products of learning from the processes of learning and focuses on the products. The processes of learning refer to specific courses, course content, teaching methods and program assessment approaches. The products of learning are internalized, significant, thoughtful applications of integrated knowledge, abilities and dispositions, as expressed by learning outcomes, and demonstrated in authentic contexts. During the phase-in period, PLA credit is being granted for documented learning that is relevant to course learning requirements. Eventually, academic credit should be granted when the assessment process has been satisfied that the PLA candidate has achieved through alternative means the exit standard described by a program learning outcome. PLA credits are intended to promote access by reducing the amount of time a learner needs to spend acquiring a credential and encouraging experienced learners to return to colleges for upgrading, retraining and continuous learning.
Learning outcomes should support equity by providing assessment and evaluation mechanisms and processes which verify that all graduates have met all the program learning outcomes (exit standards). This approach focuses on individuals and the learning results they achieve relative to program standards, rather than on grades relevant to group performance with respect to the mastery of content.
The major shift implied by learning outcomes and outcomes-based education is the altered role of content and discrete skills in the demonstration of college-level learning. Learning content and demonstrating discrete skills will always be a significant part of any vocationally-relevant training experience. Content is the foundation of the learning process inasmuch as most significant, transferable learning builds on a recognized knowledge base which is then applied. Discrete skills are elementary stepping stones to be mastered which build progressively toward more complex role performances. This shift means, therefore, that ranking learners' ability to memorize a body of content for a test or to execute discrete tasks or skills in a circumscribed context is just one step toward the verification of a reliably demonstrated role performance.
Assessments of individual progress toward the achievement of program learning outcomes are ongoing and focus on whether or not a learner has achieved each level or benchmark. Final evaluation determines if a learner has achieved the program learning outcome. Traditional grading systems, on the other hand, denote a level of achievement from A to F relative to the performance of a group or class which means that learners graduate from programs with divergent levels of ability and performance and sometimes do not meet basic vocational requirements.
Assessment recognizes that learning is a developmental process and that learners need to be coached, supported and guided in their learning to help them capitalize on their mistakes and build on their successes at each stage in the learning endeavour. Assessment therefore relies on constructive criticism and careful documentation of progress. Benchmarks or levels may be attached to specific learning activities which lead progressively toward more and more complex role performances. Assessment reports document on a regular basis what the learner has to do in order to successfully complete each level or benchmark until the program is satisfied that the learner has met each program learning outcome and is therefore eligible to graduate. The meaningful integration and use of knowledge and know-how through problem solving, investigation, analysis and decision making, along with the development of appropriate values and dispositions, are paramount.
Learning outcomes are important criteria for program review and are, therefore, key instruments of accountability for colleges. The main goal of the proposed CSAC program review initiative is that programs will demonstrate, using some common instruments and consistent procedures, that their learners have achieved the Provincially-determined program standards before they are permitted to graduate. Faculty will be accountable for curriculum and evaluation approaches that facilitate progress in learning, verify that progressive benchmarks have been attained, and eventually that the program learning outcomes have been achieved. Benchmarks or levels of learning should be attached to learning activities, sequenced in order of complexity, during which the individual progress of learners may be assessed. Consolidation or integration activities may be attached to the final evaluation of a learner's achievement of a program learning outcome. Programs will likely be called upon to verify that the consolidation or integration activities represent the level of performance described in the program learning outcome statement and that the evaluation instruments and methods they are using are valid and reliable. Verification of learner achievement of learning outcomes depends on the reliable demonstration of a role performance not just once but in several more or less authentic contexts, in real-life or simulated work- or life-role endeavours. These consolidation activities may include broadly-based longer-term projects, complex problem solving, and work, field placement or internship role responsibilities at the level of a new employee.
The information age requires that educational institutions actively involve learners in “the meaningful use of knowledge through participation in long-term, self-directed, cognitively complex tasks”. (Marzano and Arredondo, 1987) The aims of reform in college education and training have much in common with reform movements currently in progress at all educational levels in Ontario. From elementary school through to post-secondary education, the new quest is to produce learners, workers and citizens who possess the enduring habits of mind that are needed for lifelong learning, involved citizenship, productive, continuous employment and successful living.
At the heart of the reform initiatives is the focus on fostering “intelligence” in our people, not the kind manifested by standardized test scores, but, in the sense of facilitating the development of effective thinking, awareness, astute perceptions, creativity and flexibility in individuals who are capable of solving new problems. Program learning outcomes should, therefore, describe broad abilities that represent intelligent human functioning at work and in everyday life. It is in this context that learning outcomes are a strategic instrument of reform.
College Standards and Accreditation Council. (1994). Guidelines to the Development of Standards of Achievement through Learning Outcomes. Toronto: Council of Regents.
Marzano, R.J. & Arredondo, D.E. (1987). Tactics for Thinking. Alexandria, VA: Association for Supervision and Curriculum Development.
Dale Shipley is an educational consultant with the Planning Department, Algonquin College in Nepean, Ontario. Her main responsibilities include program review and generic skills implementation. She is also a member of the CSAC Generic Skills team.