High Impact Continuing Education and Training -- A Design for Long-Term Results
High Impact Continuing
Education and Training:
A Design for Long-Term
Results
Diana K. Kelly, Ph.D.
Presentation
at NCCET,
DESCRIPTION: Is your program a short-term smorgasbord or a program
designed for long-term impact? In this
interactive session you will learn how to apply a program design method for
long-term results. First you’ll develop Learning Outcomes, then you’ll use
those outcomes to design a program for high impact step by step -- the
curriculum, teaching & learning methods, and assessment methods. The design
used in this workshop has been proven effective in work-related continuing
education programs internationally. Handouts and materials will be provided so
you can continue to develop your programs back at your college.
Introduction
Is your Continuing Education or Training program a smorgasbord of activities and programs, workshops and events – similar in scope and content year after year? Or is it a strategic program designed for long-term impact?
To make a long-term
impact it is important to have an excellent program design centered around the
learning outcomes desired for the participants.
The curriculum, teaching and learning methods, and the methods of
assessing learning are designed to ensure that the learning outcomes actually
happen. And finally, it’s important to
evaluate the program to see if it is achieving the intended long-term impact.
Designing Courses with Learning Outcomes
The most important first step in curriculum design is to consider carefully what the end result should be. What should participants know and be able to do upon completion of the course or training session? These are our desired “learning outcomes” which are at the heart of our program design for long-term impact. We decide on the desired learning outcomes before even considering the other elements of the program: curriculum content, teaching & learning strategies, and assessment/evaluation methods.
The Constructive
Alignment approach was developed by Professor John Biggs in
Learning Outcomes are developed through information gathered in consultation with employers and professionals to determine what a participant needs to know and be able to do in order to be successful in the job or task. In addition, perhaps there are “SCANS” competencies (transferable skills) that are important as learning outcomes: team work, problem-solving, professionalism, interpersonal skills, customer service skills, skills in learning how to learn, writing skills, numeracy skills, general computer skills, etc.
It is important to define Learning Outcomes
clearly, using the appropriate level from Bloom’s Taxonomy. Clear definitions are essential for both
learners and trainers. They are
particularly important when more than one trainer is teaching the course or
subject area (for consistency) and when one course is the pre-requisite for
another.
Curriculum Design: The Content
When designing
the curriculum for a course, it is important to address these three questions:
1.
What are the essential things a
person must know in order to achieve the desired learning outcome?
2.
Is the content realistic in the
timeframe available?
3. Does everything need to be “covered” during class sessions?
The content is normally determined in partnership with the business or a professional organization which has good knowledge of the things that a person working in this field needs to know on the job. The methods for determining the content include DACUM and other job analysis methods for determining the content of a training program.
Some principles
of curriculum design are important to consider.
First, one-shot workshops are not effective for long-term results which
transfer to application on the job. Second,
programs offered over time, perhaps in small regular chunks, have longer
lasting results. Third, a work-based
cohort group reinforces learning transfer to the workplace. And, most importantly, long-term impact is
the result of changing individuals from within.
Individuals will only be changed if they are in a long-term program
which includes the opportunity for reflection and application (Schon, 1987).
Teaching and Learning Methods
Which teaching and learning methods will achieve the desired learning outcome and result in a longer-lasting impact on learners? When learners are actively involved in constructing their learning, the learning is more likely to “stick” – particularly when the new learning is related to prior relevant experience. Learning methods which are related to work activities, in which learners have the chance to simulate or practice on the job, are most likely to have a longer lasting impact. The opportunity for immediate application in a real-life situation provides learners with the greatest retention of learning.
Attention span studies indicate that at the beginning of a class session there are approximately three to five minutes of “settling down.” There is usually a lapse of attention about ten to 18 minutes later. The attention span falls to three to four minutes at the end of a 50 – 75 minute lecture. On average, adults have an attention span of 15 to 20 minutes, and after that the mind starts to wander.
To increase attention and involvement, and ultimately learning, it is important to use a varied approach to class activities. Planning carefully when to insert an activity into a class session is important, and changing activities about every 15 to 20 minutes maximizes attention and involvement in learning. Work-related activities increase the interest and relevance of the training for learners. Involvement in learning increases when learners are asked to apply what they are learning.
Considering the learning outcomes, it is important to determine the most appropriate modes of learning to achieve the desired learning outcomes.
· Accreditation of Prior Experiential Learning – to help learners at any stage to determine what they may already know about a subject, and to either award credit or to waive the course based on their evidence. Evidence is normally in portfolio form and students need assistance to learn how to develop this.
· Work-Based Learning – aimed at those who are currently working in the field that they are studying so their classroom-based learning is directly related to their job; OR aimed at those on work placement as a way to use what they have learned in the classroom. In both cases work and learning are integrated.
· Online Learning – to support and enhance classroom learning by putting course materials online, providing additional links and resources for students, and providing opportunities for online discussion groups – particularly when students do not meet regularly if they are on a work placement.
·
Classroom-Based Learning – what happens in the classroom (see next section).
Plan to Use a Variety of Active Teaching Strategies
Recognizing that learners come with a variety of levels of educational background and a variety of learning styles, it is important to select a variety of learner-centered teaching strategies to foster student engagement in the learning process. But these teaching strategies should also be chosen because they are most appropriate to result in the desired learning outcomes.
·
Interactive lectures –
providing opportunities for students to reflect or discuss important points.
·
Active discussions in small
groups and in large group
·
Audio-visual illustrations of
processes and concepts
·
Demonstrations by teacher and
students
·
Team and individual projects
·
Hands-on laboratory work
·
Problem-based learning
· Formative assessments
Plan to Use a Variety of Out-of-Class Learning Activities
Learners should have a good idea of how much time they will be expected to spend outside of class sessions in work related to the course. The involvement of learners outside of the class will contribute significantly to their development as lifelong learners who are self-directed in their learning. Learners’ out-of-class activities could include the following, as appropriate:
·
Course-related reading
·
Preparation for exams
·
Individual assignments designed
to help students to master certain content areas
·
Online discussions and
activities to introduce new concepts or reinforce learning from the class
sessions.
·
Individual or Group projects or
presentations
·
Research designed to encourage
deeper learning of a particular aspect of a subject
· Visits/interviews of professionals in the field.
Assessing Learning
How will you know if the activity or class session has achieved the desired learning outcomes? When and how do you want to find out if learning has taken place? When assessing learning it is important to use both formative while the learning is in process, and summative assessments at the conclusion of the learning.
Formative assessments of learning are usually informal and may be built into the learning activities to check on learning. Are they “getting it?” If not, it’s useful to know what learners are finding confusing so that you can do something about it. “Classroom Assessment Techniques” are an excellent way to find out what students are learning while learning is in process (Angelo & Cross, 1993).
Summative assessments are the traditional end-of-course projects, exams, presentations or other papers, which are used after the training has ended. These are supposed to measure the extent to which the learning outcome has been achieved, but often they are so poorly designed that they only measure students’ ability to take the test or write a paper.
Instead, it makes more sense to carefully design the assessment activity so that it truly “fits” the desired learning outcome. Criteria to evaluate the summative assessment should be specific and measurable, based on the desired learning outcome. Ideally, scoring rubrics should be used for a consistent measurement of learning achievement.
Unless required by an accrediting body or by university regulations, it may not be necessary to include the traditional written exams. Here are some other methods of assessment that may be considered:
·
Individual or group projects
·
Individual or group
presentations
·
Individual or group
problem-solving
·
Problem-based learning
·
Inquiry-based learning
·
Work-based learning
·
Learning contracts
·
Performance of skills
·
Clinicals (in medical fields)
·
Short quizzes in class
· Participation in group work in class sessions
Evaluating Continuing Education & Training Programs for Long-Term Impact
In determining whether or not our programs are having long-term impact it is important to consider the evaluation levels outlined in the ASTD publication Measuring the Impact of Training (Wade, 1998):
- Level 1: Training Program Response
- Level 2: On the Job Action
- Level 3: Business-focused Results
- Level 4: Organizational Impact
- Level 5: Return on Investment (ROI) or Cost/Benefit
Too many programs only measure Level 1 to determine if training programs were “successful.” If everybody was “happy” with the training session, it’s considered to be successful. But to measure long-term impact, programs must measure levels 2, 3 and 4. And to analyze the costs and benefits over the long-term, level 5 must also be measured.
Level 2 evaluation would address the following questions: Have those who participated in a program actually changed their behavior on the job? What have they done differently, and how have they done it? Have the changes in practice been incorporated over several years, or were they attempted in a short period of time and then abandoned? Have participants continued their own self-directed learning and development long after the end of the program?
Level 3 evaluation may be used to find out the extent to which the training program had a positive impact on the intended business goals. For instance, if a business had specific concerns or problems that the training was supposed to resolve, has the training program made any difference? In some cases it may take several years to see a positive impact..
Level 4 evaluation is all about the big picture. Has the training made a difference over the entire organization? In some cases, this is not actually a goal of the training if it is designed to meet a very specific need. However, often specific needs are linked with larger organizational development strategies, so these should be measured over time.
Level 5 is one of the most difficult to measure because it will require some baseline data that shows the costs and benefits “before” (with the current training program) and “after” (with a different type of training program).
Finally, the
results of all of the long-term evaluation will help to identify new training
and organizational development needs which may be used in planning future
programs.
Conclusions and Recommendations
The following steps are recommended to implement this approach:
1) Provide training to those who design courses to ensure that the learning outcomes are the focus of the design and delivery of the training curriculum. Use Biggs’ “Constructive Alignment” approach by starting with the desired learning outcomes at the appropriate level, then develop the content, teaching & learning methods, and assessment methods.
In designing
curriculum that will support and enhance student learning it is essential to
have consistency between
- the learning objectives for the
module
- the learning methods that will best
achieve the learning objectives
- the assessment methods that will best measure students’ achievement of the learning objectives.
2) Build
in an evaluation process to ensure that the desired learning outcomes are being
attained, and that they are actually making a difference on the job and in the
business.
References
Angelo, T. and
Cross, K. (1993). Classroom Assessment Techniques:
A Handbook for College Faculty,
Second Edition.
Biggs, J.
(2002) Aligning the curriculum to
promote good learning. In Constructive Alignment in
Action: Imaginative Curriculum Symposium. Learning & Teaching Support Network
(LTSN) Generic Centre (
Kelly, D. K.
(1993).
Classroom Research and
Interactive Learning: Assessing the
Impact on Adult Learners and Faculty. Unpublished
doctoral dissertation, The Claremont Graduate School.
Schon, D.
(1987). Educating the Reflective Practitioner.
Wade, P.
(1998) Measuring the Impact of Trainin.
About the Author
Over the past
several years, Diana has successfully used these methods of “Constructive
Alignment” in teaching curriculum design for work-related learning in
Continuing Education and Training, Professional Development, Apprenticeship,
and College-level vocational programs.
Diana
Kelly has over 25 years of experience in higher education, including 19 years
in the California Community Colleges -- first as a full-time faculty member in
Communications, then as a faculty developer, and finally as Associate Dean of
Continuing Education. More recently Diana
has spent six years in leadership roles in two well-respected universities in
Since 1989 Diana
has provided many faculty development seminars and workshops in the
Diana Kelly earned her doctorate and master’s degree in Higher Education
at the
Comments
Post a Comment