0
Table of Contents
Analysis
Design
Development
Implementation
Evaluation
00
Bean on Working with Stakeholders
In her book, The Accidental Instructional Designer, Cammy Bean gives great advice for working with stakeholders and subject matter experts (SMEs). First, she advises that instructional designers approach the relationship with stakeholders as consultants. She asks, "What do good consultants do?" The most important point, she states, is that a consultant is "someone who adds value and doesn't just take orders. [He/She] pushes back when necessary, recommends alternate solutions, and sometimes delivers answers that people may not want to hear."
In addition, consultants:
"Analyze the problem and recommend solutions. Consultants don't just say, 'Absolutely, let me turn that deck into e-learning material.' They dive into the situation, they ask hard questions, and they get to the root of the problem. Sometimes the answer may not be training."
"Educate and provide an alternate viewpoint. Consultants have expertise and can push back. If the stakeholders or SMEs on your project have a bad idea, you can politely tell them why it won't work. It helps if you have research and data to back you up."
"Stay current with industry trends and issues. This means two things: know what's going on in the technology-based learning world and know what matters to your company."
"Understand business goals and objectives of [their] organization. If [a solution] supports the company's bottom line and ultimately affects what matters to people like your CEO, then you'll have done your job well."
"Hone [their] communication skills. You need to be a good communicator in written and verbal communication, which would require practice. To convince [stakeholders and SMEs] to accept your ideas you may need to argue with [them]. It's not about force-feeding [them] your ideas, but about educating [them] and helping [them] see learning in a different light."
Now, what is the process of working with stakeholders and SMEs?
First, schedule a time to sit down and plan things out. At the sit down, you'll need to find out what really matters to them during your meeting. Are there certain parameters that matter? Is the deadline most important? Or is quality more important? Budget? Ask, "What does success look like?" Document everything. Explain to them your instructional design process. Lay out a timeline with deadlines and a description of all key tasks. Be clear with what you expect from the SME. One very important thing is to visualize for your stakeholders and SMEs what your plans are. You can show examples of similar products, both in their prototype and final forms.
When sharing your instructional design philosophy, here are some handy tips. First, remind the SME that they are an expert in their subject. Because of this, they have a difficult time seeing things from a novice's perspective and try to shove a lifetime of information down a novice's throat. Try to get them to focus on the most pertinent information. Second, try to draw out real life stories from the SME that can enrich the learning experience. Next, see if you can come to an agreement on a way to chunk the content into 10 minute segments. If a SME passes you a slide deck, ask them to limit each slide to one idea. Too much content can be overwhelming.
What questions to ask to get strong performance objectives?
"What do you want people to be able to do?"
"What are the three takeaways that you want someone to get out of this program?"
"What mistakes do people most commonly make?"
"Where do they get things wrong?"
"Can you tell me a story about this content? How did you get into this area of expertise? What mistakes have you made along the way?"
"Where should people go for more help and information?"
References:
Bean, C. (2023). The Accidental Instructional Designer, 2nd edition: Learning Design for the Digital Age (2nd ed.). Association for Talent Development.
*All of the above information was either quoted directly from the book or paraphrased.
Gilbert's Behavior Engineering Model
Gilbert's Behavior Engineering Model identifies six causes of human competence or incompetence, the latter of which can be caused by poor management of any of the six elements. Three of the elements are related to environmental supports (data, instruments, incentives) while the other three are related to a person’s repertory of behavior (knowledge, capacity, motives). Identifying which of these elements are lacking and making an effort to improve them can increase human performance within an organization.
1. Data: An analysis should be made of whether or not the underperforming personnel has the necessary information to know they are underperforming. They should understand the performance gaps and expectations.
2. Instruments: Do the employees have the necessary tools needed to perform their tasks well? This could include technology, job aids, or access to a forum to discuss strategies with fellow employees.
3. Incentives: Do the employees have a reason to perform to worthy standards? Are they compensated at a rate equal to what is expected of them? Do they feel the work is meaningful?
4. Knowledge: Do the employees have the necessary knowledge? Do they need to be provided additional information through job aids and/or training?
5. Capacity: Is performing the necessary tasks something the employees are capable of doing? Do they have the necessary background and skills?
6. Motives: Do the employees feel properly motivated? Do they have intrinsic motivations that can be targeted and exploited in order to get them to improve their performance?
Changing any of the six elements can have a systemic impact on any of the other elements through the “diffusion of effect.” This means that changing the circumstances of one of the elements may impact other elements. For example,
Promising a proper level of compensation (incentives) may reduce equity tension in workers and help them feel appreciated and motivated (motives), which in turn may encourage them to pay more attention to the information required for the work (data) and become self-directed to teach themselves to be more competent performers (knowledge) (Chyung, 2005, p. 25).
Reference:
Chyung, S.Y. (2005). Human performance technology from taylor's scientific management to gilbert's behavior engineering model. Performance Improvement, 44: 23-28.
Greene's KSA Filter & PRIMED
Rance Greene's book Instructional Story Design offers two tools to use during stakeholder meetings. Used in combination, these tools will help you find the correct approach to fixing business issues, whether it be by using training or simply changing environmental controls. The PRIMED tool is useful for crafting stories that can help change attitudes and behaviors.
First, the KSA Filter can be used to help us decide if we should use training solutions or not. If the issues within the company are a result of lack of knowledge or skills and/or poor attitudes, training would be a beneficial solution. However, if the issues are a result of work environment or processes (for example, employees are overloaded in one particular area and so cannot apply adequate attention to another area), then other solutions should be explored.
Next, use PRIMED to find out what is causing the issues and craft appropriate stories to fix them. It is a mnemonic that presents the questions that should be asked to stakeholders when they approach you to find solutions for their business needs.
Personal Opinions: Not only will asking stakeholders their personal opinion help to build strong relationships, but will also get to the root of the problem. Ask questions such as: "What do you feel the root problem is? What would you like to see change as a result of training?"
Real Stories: "Can you share a real story that illustrates the problem? Can you provide a case study?" Look out for behaviors or systems that can be changed. Also, the stories you hear might be useful for stories in learning experiences. Find the characters and conflict and craft stories accordingly.
Initial Indicators: "What brought this problem to your attention? Describe how you became aware that training was need." You can glean desired business outcomes from such questions. What numbers would they like to increase or decrease? Are there scores that need to be raised? Behaviors that need to be changed? If someone else brought the issue to the attention of the stakeholder, find those people and interview them for additional information.
Metrics: "Can I take a look at the survey results? What are the reports indicating in regard to this problem?" Data can shine a light on the issues.
Examples: "Can you provide an example of the kinds of issues you've noticed in regard to this problem?" Follow up this question by asking for real stories and metrics.
Distractions: "Is there anything distracting employees from performing their jobs well? If training was not available, what would you change?" These types of questions can reveal if training is not in fact the best solution.
Reference:
Greene, R. (2020). Instructional Story Design: Develop Stories That Train. Association for Talent Development.
*All information above paraphrased from this book. The quoted questions are direct quotes from Mr. Greene's work.
Harless' Front-End Analysis
Mager and Pipe's Performance Analysis
McClusky's Theory of Margin
Howard McClusky's Theory of Margin, also known as the power-load-margin (PLM), is a formula and conceptual model that "addresses motivation as a measure of how many resources (power) the learner has to offset the demands (load) that potentially diminish motivation for learning" (Merriam & Bierema, 2013, p. 153). McClusky defined his theory as "Margin is a function of the relationship of load to power. By load we mean the self and social demands [e.g. family commitments, occupational responsibilities, or goals] required by a person to maintain a minimal level of autonomy. By power we mean the resources, i.e., abilities, possessions, position, allies, etc., which a person can command in coping with load" (McClusky, as cited by Merriam & Bierema, 1970). The idea is that "that adults need enough margin to handle life’s load of challenges, changes, and crises. Low margin might indicate that the adult is under undue stress or illness. Excess margin might indicate a life that has too little load where the adult is not fulfilling her potential" (Stevenson, as cited by Merriam & Bierema, 1982). "Margin is essential to the mental hygiene of the adult . . . A margin allows a person to invest in life expansion projects and experiences including learning experiences"(Main, as cited by Merriam & Bierema, 1979)
The theory is important for instructional designers because "instructors can unknowingly create surplus 'load' for learners by assuming a traditional, authoritarian stance and not respecting learners’ opinions or experience. Other instructor behaviors that contribute to learner load might be disorganization, distracting mannerisms, inappropriate assignments, or unclear evaluation guidelines" (Hiemstra & Sisco, as cited by Merriam & Bierema, 1990).
Reference:
Merriam, S. B., & Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Rummler & Brache's 9 Boxes Model
Wile's Human Performance Model
Bloom's Taxonomy
Instructional Scaffolding
Gagne's Nine Events of Instruction
Robert Gagne formulated a systematic nine step sequential process that utilizes all the possible events of mental processing that take place during instruction. It builds on his idea of “conditions of learning,” which include both “internal conditions [which] deal with what the learner knows prior to the instruction [and] external conditions [which] deal with the stimuli that are presented to the learner, e.g. instructions provided by the teacher” (Khadjooi et al., 2011). Gagne emphasized the need to specify the learning outcomes needed to be achieved and categorized possible outcomes into five types: verbal information, intellectual skills, cognitive strategies, attitudes, and motor skills (Khadjooi et al., 2011). Once that is clear, the process can be used to create a learning experience in the following order:
1. Gaining attention: Possible strategies include “an abrupt stimulus change, such as gesturing or speaking loudly,” “starting the lesson with a thought-provoking question or interesting fact,” or “providing an interesting visual or sound stimulus” (Khadjooi et al., 2011).
2. Informing the learner of the objective: The learners should know what the exact purpose of the learning experience is and what they should be able to do or know upon completion.
3. Stimulating recall of prerequisite learning: Learners should be asked to recall and/or discuss information that they already know that is connected to the lesson material.
4. Presenting the stimulus material: The content is presented in an interesting and engaging manner.
5. Providing learning guidance: Additional material is provided to help them gain a better understanding of the material. This can include case studies, demonstrations, graphical representations, etc.
6. Eliciting the performance: The learner puts their new knowledge to use and practices their new skill or behavior.
7. Providing feedback: The instructor evaluates the learners’ performance and provides suggestions for improvement.
8. Assessing the performance: After the learners process the feedback, they further demonstrate what they have learned and an assessment is made.
9. Enhancing retention and transfer: Additional opportunities for practice and spaced review is provided to increase transfer of knowledge.
Reference:
Khadjooi, Rostami, & Ishaq. (2011). How to use Gagne’s model of instructional design in teaching psychomotor skills. Gastroenterology and Hepatology From Bed to Bench, 4(3), 116–119.
Gardner's Multiple Intelligences
Howard Gardner listed eight types of intelligence that people may possess. Some people might be strong in one area while weak in another. It is very helpful for understanding that learning environments should be diversified in order to play on the strengths of everyone. A great application of the theory is giving learners different options for how to complete assignments and demonstrate knowledge. Here are Gardner's learning objectives and some of my own ideas for how lessons can be tailored for people strong in particular areas.
Linguistic: This group is made up of individuals strong in reading and writing abilities. To accommodate them we could offer the option that they write a report about the information they have learned.
Logical-Mathematical: Learners will this type of intelligence could benefit from presenting the learned information in graphs, or by adapting it to a science experiment or game.
Musical: Learners of high musical intelligence could write a song about what they have learned.
Spatial: Learners of high spatial intelligence could present the information visually through art, graphs, models, etc.
Bodily-Kinesthetic: Whenever possible, learners should be able to try to physically apply the information. Hands on experience is best for people of this type of intelligence.
Interpersonal: People of high interpersonal intelligence could create learning experiences themselves to teach others, or engage in dialogues with others to teach the information. Group work amongst multiple learners of interpersonal intelligence would be beneficial for them.
Intrapersonal: Someone with high intrapersonal intelligence can reflect on what they’ve learned and how it applies to their past experiences and how it has helped them evolve their thinking and knowledge.
Naturalistic: These folks excel in careers related with nature, such as biologists, farmers, and animal trainers. Giving them activities that allow them work within and utilize the natural world would be an ideal learning experience for them.
References:
Merriam, S. B., & Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Keller's Motivational Learning Model
The MVP model uses the ARCS-V design process to stimulate motivation during learning experiences. Tactics should be utilized from each of the five categories of ARCS-V, which are as follows:
Attention: Tactics should be employed that help capture learners' attention. A variety of media can be displayed such as graphics, animations, and videos. Variability is imperative because the repetition of even very interesting media will eventually lead to the learner losing interesting. In addition, storytelling techniques can be utilized such as introducing interesting characters and conflicts. Giving the learners problems to solve is also a great approach.
Relevance: The instructional aims should be congruent with the goals of the learners. Try to present material that is compatible with their learning styles and environment and connect to their experiences. Intrinsic and extrinsic motivations should be understood and exploited. Allowing intrinsically motivated learners to engage in freely chosen actions amplifies their feeling of self-determination, which leads to sustained goal oriented behavior.
Confidence: Confidence should be built by setting expectations. Provide clear objectives and reasonable possible achievements. In addition, students should perceive that their achievements are a result of their own knowledge, skills, and effort as opposed to luck, help, or lack of difficulty.
Satisfaction: Students should walk away from learning experiences with positive feelings. They should feel they have been treated fairly. They should feel their success has been recognized. They should receive tangible rewards. Finally, they should feel that the amount of work required of the learning experience was appropriate for the benefits received.
Volition: When goals are perceived to have high value, learners demonstrate a higher level of persistence. They will be more likely to overcome obstacles, setbacks, and distractions to do what needs to be done. These folks will employ self-regulatory tactics to achieve goals.
References:
Keller, J. M. (2017). The MVP Model: Overview and Application. New Directions for Teaching and Learning, 2017(152), 13–26.
Knowles's Assumptions of Andragogy
Andragogy refers to the theoretical framework for approaching adult learning. Malcom Knowles developed six assumptions pertaining to how adults learn.
1. Adults have the self-concept that they are self-directed rather than dependent. Learning environments need to give learners a sense of mutual respect and trust and they should project an atmosphere of collaboration. This helps learners feel like they are contributing to the construction of learning experiences which satisfies their need to be self-directed.
2. Adults have a wealth of experience that can be drawn from to aid learning. Learning facilitators should try to start with the experiences of their learners when planning instructional material. They can draw on events in their lives to craft stories and make things relevant to the learners. A downside to experience should be mentioned:
Adults can become dogmatic and closed-minded about learning something new because their prior knowledge and experience has worked for them in the past and they see no need to learn something else. Or, a traumatic life experience might function as a barrier to learning (Merriam et al., as cited in Merriam & Bierema, 2013).
In these types of cases, facilitators should craft experiences with empathy.
3. Adults are more willing to learn when the learning fulfills the needs of their social role. Where a learner is currently at in life, such as their social role in society or their age, is a way to craft "teachable moments," i.e., moments in one's life when it becomes imperative to learn something and so the learner has a "readiness to learn." For example, someone about to have children has a readiness to learn how to change diapers. Other things that might create a readiness to learn would be a shift in the demands of the market or the sudden occurrence of natural disasters.
4. Adults are more problem-centered than subject-centered when it comes to learning. Their readiness to learn increases when there is an immediate need to do so and they need to solve a problem that has just arisen in their lives. When learners engage in this kind of immediate application, the learning sticks more so than learning done by reading in books about things they may not face until years later.
5. Adults as a whole are more driven by intrinsic motivation than extrinsic. Internal motivating factors such as self-esteem, personal satisfaction, self-fulfillment, and improved equality of life are more powerful than money for many adults.
6. Adults need to know the reasons for learning something. And they need to know it beforehand! In some cases, there will be required training that doesn't have immediate and obvious benefit for the learner. In this case, a facilitator needs to make an intellectual argument to show the benefits which may not be realized until far down the line and won't be obviously apparent.
References:
Finlay, J. (2010, May 17). Andragogy (Adult Learning) [Video]. Youtube. https://www.youtube.com/watch?v=vLoPiHUZbEw
Merriam, S. B., & Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Mager's Performance Objectives
Robert F. Mager developed a method for writing instructional objectives. He called them "performance objectives" and differentiated performance from behavior by highlighting that the former is what people are able to achieve while the latter is simply what they do.
There are three necessary elements of a performance objectives:
Performance: what the learner is expected to be able to do or produce
Conditions: the conditions under which the performance is expected to occur
Criteria: the level of competence that should be met or exceeded (Mager, 1997, p. 51)
For example:
Given one hour, learners will be able to close down the store registers and deposit the money in the safe without errors.
The noted performances should be observable actions (be able to classify the species) and not covert actions (distinguish the species). The conditions should describe what learners may or may not have when needing to perform and any other factors that may affect their performance. Sometimes you don't need to include any conditions. The criteria should be a realistic goal that the learner is able to achieve and it should describe exactly what is needed to be considered acceptable performance.
References:
Mager, R. (1997). Preparing instructional objectives: A critical tool in the development of effective instruction (3rd. ed.). Atlanta, GA: CEP Press
Rigby & Ryan's Self-Determination Theory
C. Scott Rigby and Richard M. Ryan build a theory that helps organizations harness employee motivation to their benefit. They focus on the psychological needs of employees and how supporting them leads to high quality motivation.
First, they lay out of a spectrum of motivational quality, which "predicts important outcomes from employee engagement, to its byproducts of enhanced wellness, performance, and organizational citizenship" (Rigby & Ryan, 2018, p. 136). Their continuum of motivational quality ranges from amotivation (low quality) to intrinsic (high quality).
Amotivation: The employee finds little to no satisfaction with their work and/or they feel they are not effective.
External pressure: Both positive and negative external pressure (rewards and punishment, respectively) can lead to good short term results. However, when folks feel externally pressured, they often perform more poorly by taking ineffective shortcuts.
Internal pressure: This is the pressure people put on themselves in order to receive credit or project a certain appearance in order to gain approval.
Personal value: The employee understands the importance and value of the work and it feels valuable to them as well.
Intrinsic: The employee enjoys doing the activity for its own sake.
The consequences of low motivational quality:
lower productivity
lower creativity
less learning
less satisfaction with compensation
decreased commitment to values and policies
less loyalty and trust
The benefits of higher motivational quality:
stronger performance
more innovation
deeper learning
greater job and compensation satisfaction
greater commitment to values and policies
greater loyalty and trust
If organizations can support the following psychological needs in employees, they can increase higher quality motivation.
Autonomy: Workers feel more motivated when they feel a sense of choice within their work. If they can feel a sense of volition and ownership over the work they do, they feel less alienated and burned out and enjoy doing their work.
Relatedness: Workers want to feel like they are part of a team and that their contributions are valued and respected. When workers feel isolated and/or irrelevant, they feel like they don't belong or matter to an organization.
Competence: An environment should be built that develops competency in workers. Resources and training should be available so that expertise and skills can always be developed. Workers not only thrive when feeling effective and successful, but also when there are always new challenges and responsibilities to take up so one always feels a sense of growth.
References:
Rigby, C. S., & Ryan, R. M. (2018). Self-determination theory in human resource development: New directions and practical considerations. Advances in Developing Human Resources, 20(2).
4 Golden Rules of UI Design
6 Essential Graphic Design Principles to Guide Your eLearning Course Design
9 Key Tips for Effective User Experience Design for Online Learning
e-Learning Tools According to Knowledge Types
Yonnie Chyung points out three types of knowledge and gives examples of how to utilize e-learning tools to make effective e-learning objects for each knowledge type. Declarative knowledge (knowing what): This is the knowledge of concepts and facts. Robert Gagne's Nine Events of Instructions can be used to plan a lesson that teaches declarative knowledge. For the creation of an e-learning object, software should be used that allows the developer to "present visual representations to illustrate abstract concepts and facts, such as diagrams, images, animation, and video clips." (Chyung, 2007, p. 5). Procedural knowledge (knowing how): This is the knowledge of procedure and processes. Both Gagne's Nine Events of Instruction and the Show-Tell-Do-Check method can be applied to create e-learning for procedural knowledge (the latter method is especially useful for teaching technical procedures). Video editing and screen capturing software will come in handy. Situated knowledge (knowing when and why): This is the knowledge of principles. Simulations that utilize role-playing and problem solving and provide constructive feedback are best for teaching situated knowledge. Use e-Learning software that allows you to create branching scenarios. Reference: Chyung, Y. (2007). Learning object-based e-learning: Content design, methods, and tools. Learning Solutions E-Magazine.
C.R.A.P. Visual Design Principles
Mayer's 12 Principles of Multimedia Learning
Bandura's Social Learning Theory
Cognitive Apprenticeship
Situated cognition, also referred to as "contextual learning," is learning that takes place in context or through an experience (Merriam & Bierema, 2013, p. 118). While situated cognitive often refers to on-the-job training, roleplaying and simulation games can be authentic and effective substitutes.
Cognitive apprenticeship falls under the concept of situated cognition. It is process where a master carries out the task with a learner at his side.
As a quality assurance team lead and trainer in an Alaskan salmon cannery, it was my job to carry out cognitive apprenticeships for several new employees who joined the quality assurance team. I will detail the position I had the most experience training: Dock Q.A. The Dock Q.A. was the first line of defense, as they inspected the incoming tenders (boats that deliver the salmon) and the dock before intaking the fish. They also had to set up boot, hand, and fish sanitizer dips and inspect the incoming fish and document their findings. The difference between this position and the other quality assurance positions is that the dock intakes fish when the tide is in, which at the beginning of the season is generally between midnight and 3am. I had to remain with the trainee for at least one whole night shift (often it was 1.5 night shifts) to provide cognitive apprenticeship in real time. Pre-training them and throwing them straight into it on their own wouldn't be prudent as it would be with some of the other positions because I would be asleep while they are working, leaving them without help.
I carried out the cognitive apprenticeship like so:
Modeling: The trainee followed me around as I completed the required tasks.
Coaching: Then I let the trainee take over while I supervised, providing tips, feedback and guidance along the way.
Scaffolding: I supported them as long as necessary until they started to feel competent and confident. I gradually withdrew my support as they mastered certain tasks.
Articulation: Through all steps of the process, I quizzed them on the process and tried to get them to think deeply about why they were doing what they were doing. At the end of the first night I had them explain to me the process in detail.
Reflection: I asked them how comfortable they felt they were to do the job the following night without me by their side. Of the three people I trained at this position, only one felt comfortable. With the other two, I continued to coach them an additional half-night.
Exploration: Through discussion with me and their fellow quality assurance techs, they learned about the subsequent quality assurance processes in the plant that ensure safety and integrity of the product. They then had a big picture view of what our role was and how the plant operated.
References:
Exter, M., & Ashby, I. (2019). Using Cognitive Apprenticeship to Enculturate New Students into a Qualitative Research. The Qualitative Report, 24, 873-886.
Merriam, S. B., and Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Community of Practice
Click here to go to my blog post on the topic.
Four Stages of Competence
Kolb's Experiential Learning Cycle
David A. Kolb wrote that "learning is the process whereby knowledge is created through the transformation of experience" (Kolb, as cited by Merriam & Bierema, 2013). He then detailed a four stage learning process and four abilities that are necessary for learning to be effective.
1. Concrete experience: With concrete experience abilities (CE), learners embrace new experiences without bias.
2. Reflective observation: With reflective observation abilities (RO), learners reflect on their experiences from many different perspectives.
3. Abstract conceptualization: With abstract conceptualization abilities (AC), learners understand their experiences conceptually and create logical theories based on them.
4. Active experimentation: With active experimentation abilities (AE), learners use their theories to think critically to make sound decisions and solve problems.
Reference:
Merriam, S. B., and Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Mezirow's Transformative Learning
Jack Mezirow defined transformative learning as learning that
involves an enhanced level of awareness of the context of one’s beliefs and feelings, a critique of their assumptions and particularly premises, an assessment of alternative perspectives, a decision to negate an old perspective in favor of a new one or to make a synthesis of old and new, an ability to take action based upon the new perspective, and a desire to fit the new perspective into the broader context of one’s life (Mezirow, as cited by Merriam & Bierema, 2013).
Mezirow laid out a 10 step process of transformative learning that begins with a phenomenon he called a disorienting dilemma (1). A disorienting dilemma is an event that causes a crisis in one's life, such as losing something or someone we value or love or something very traumatic. It can be a single event or a culmination of a series of experiences that ultimately trigger the transformative learning process, which proceeds in the following steps:
2. Self-examination
3. Self-assessing previously held assumptions
4. Recognition that one's discontent and the process of transformation are shared
5. Exploration of options for new roles, relationships, and actions
6. Planning a course of action
7. Acquiring knowledge and skills for implementing one's plans
8. Trying out new roles
9. Building competence and self-confidence in new roles and relationships
10. Reintegrating the new perspective into one's life
Here is my own example of how we might apply transformative learning in the workplace. First, watch this video.
First, the VR situation puts the learners in the traumatic situation of having a gun pointed in their face in addition to receiving aggressive and life threatening demands. This represents the disorienting dilemma that makes the learner recognize that they have a need to learn something new, which in this case is how to deal with armed robbery. Next comes self-examination. The learner will examine how they reacted to the disorienting dilemma and take note of their strengths and weaknesses during the situation. They will come to know if they will experience anxiety in such a situation and if it is paralyzing, or if they are able to remain calm and cogent. Then they will "critically examine the assumptions [they] had been living with prior to the disorienting dilemma" (Merriam & Bierema, 2013, p. 85). In this case, they will examine whether they had previously thought themselves capable of handling such an event with ease, or perhaps they thought themselves too weak to handle it but are proven wrong. In addition, they might notice they had certain assumptions about what a potential thief might looks like, how they might talk, and how they would carry out an armed robbery. The learner might realize that the situation went very differently than what they had expected. Next, they realize their lack of experience and skill in handling such an event is shared with the other employees and each must also go through a process of transformation. Individually or with fellow employees, the learners explore new ideas for how to deal with armed robbery. How can they survive it? How can they deescalate? How can they contribute to the arrest of the perpetrator? Then they will create a plan of action to achieve acquiring the skills needed to do these things. They can then jump back into the VR training and try out their new role as a person equipped with the knowledge and skills to competently handle the dilemma. The more they practice and ruminate on the choices they make, the more they will build competency and self-confidence. Finally, they will adopt a new perspective of themselves as a strong and capable human being capable of handling life threatening situations.
Reference:
Merriam, S. B., and Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Reflective Practice
Reflective practice refers to the act of reflecting on experience. There are two types of reflection that are important to the concept. The first is reflection-on-action. It refers to when we reflect on our experiences after the fact. After an experience, we analyze what happened, how we handled it, and how we could have done better. Some effective ways to utilize reflection-on-action in learning experiences might be to have learners write about their experiences. They can rate their own ability and reflect on their mistakes and make a game plan for the future when faced with similar challenges. The other type of reflection is reflection-in-action. It refers to the act of reflection while in the middle of an experience. Those who are adept at reflection-in-action are those we think of as "quick thinkers" who "think on their feet" (Merriam & Bierema, 2013, p. 116). These people analyze the effectiveness of their actions in progress and adjust accordingly.
Reference:
Merriam, S. B., and Bierema, L. L. (2013). Adult learning: linking theory and practice. John Wiley & Sons.
Chyung's 10-Step Evaluation Procedure
Dr. Yonnie Chyung, of Boise State University fame, created a 10-step procedure for conducting an evaluation. Dr. Chyung draws from Michael Scriven's Key Evaluation Checklist, E. Jane Davidson's book Evaluation Methodology Basics, Robert Brinkerhoff's book The Success Case Method, and Michael Quinn Patton's book Utilization-Focused Evaluation as the basis for her procedure. These are my notes drawn from Dr. Chyung's book, 10-Step Evaluation for Training and Performance Improvement.
Her model consists of three phases.
Identification Phase
Step 1: Identity the program to be evaluated
During this step, the (potential) evaluator(s) should meet with a client to learn what needs to be evaluated.
Use the 5W1H method (Who, When, Where, What, Why, and How) to learn more about the evaluand
Trace the history of the program and identify the causal factors associated with the performance gaps the program was meant to address
Make use of Gilbert's Behavior Engineering Model
Step 2: Identify the stakeholders of the program and their needs
Upstream stakeholders/impactees, downstream direct impactees (direct consumers), and downstream indirect impactees (indirect consumers)
Step 3: Identify the purpose of the evaluation based on how the findings will be used
Decide if the evaluation will be formative or summative
Formative: Conducted to find areas for improvement
Summative: Conducted to identify the overall effectiveness of the program and if it met its goals
Decide if the evaluation will be goal-based or goal-free
Goal-based: Evaluating the quality of a program against the program's goals
Goal-free: Evaluating the quality of the program against the true needs of current stakeholders (which may not be reflected in the original goals)
Identify the evaluation's merit and worth
Merit: Internal quality, context-independent
"Great design!"
"This is not user friendly."
Worth: External value, context dependent
"Our users need this."
"We don't need this and it costs too much."
Use a system-focused evaluation instead of an intervention-focused evaluation
System-focused: addresses how well the organization implemented the intervention
Helps evaluators learn about the programs hidden needs, external processes, and unintended outcomes
Intervention-focused evaluation: addresses how well the intervention worked
These three steps may run concurrently. During all three stages, the evaluator(s) should be do a feasibility and risk assessment. This should investigate the:
Maturity of the evaluand (how long as it been implemented)
Scopre of the evaluation
Support for the evaluation
Ethical concerns for the evaluation
Resources for the evaluation
They should consider the degree of potential damage that each risk factor may produce and the likelihood of such damage occuring.
By the end of this phase, the evaluator(s) should submit a statement of work (SOW) to the stakeholders (if the evaluators see need for an evaluation). If accepted by the stakeholders, planning of the evaluation should begin.
The purpose statement should include:
Type and focus of the evaluation
Intended users and their intended uses
Organization and stakeholder needs
Planning Phase
Step 4: Develop or review a program logic model
The evaluators should involve the client and other stakeholders.
Step 5: Determine dimensions and importance of weighting
The dimensions should be aligned with stakeholder needs and their intentions with the evaluation's findings.
Step 6: Determine data collection methods
Direct measures and multiple data sets should be used.
By the end of this phase, the evaluator(s) should submit an evaluation proposal to the stakeholders. Based on what the client wants, the evaluator(s) can begin implementing the evaluation or forgo or delay it.
Implementation Phase
Step 7: Develop data collection instruments
Obtain approval from stakeholders.
Step 8: Collect Data
Adhere to ethical and professional standards. Maintain confidentiality of all participants.
Step 9: Analyze data with rubrics
Step 10: Draw conclusons
Organize and present findings based on stakeholder needs and intentions.
By the end of this phase, the evaluator(s) should submit a final report to the stakeholders and assist in their use of the findings.
Reference:
Chyung, S. Y. (2018). 10-Step evaluation for training and performance improvement. SAGE Publications, Incorporated.
Evaluating Scenario-Based e-Learning
According to Clark and Myer, here are five typical categories we build questions from for scenario-based e-learning evaluations.
Motivation: Are learners satisfied and engaged in the training? Do they happily finish it to completion? This can be measured with surveys, completion rates, enrollment rates, and ratings.
Learning Effectiveness: Were the learning objectives achieved? Tests can be used to measure factual and conceptual knowledge, strategic or principle knowledge, near or far transfer procedural skills, and near or far transfer open ended skills.
Learning Efficiency: How long did it take learners to complete the training? How long did it take learners to become competent? Measure the time acquired to complete the training and to acquire expertise.
Transfer to the Workplace: How well are new knowledge and skills applied to the workplace? Are the scenario knowledge and skills adapted to different contexts? These can be measured by job performance metrics, supervisor ratings, and tests that measure the use of skills.
Return on Investment: What is the ratio between performance gains and costs to produce and take training? Performance gains in workplace translated to monetary values should be divided by costs to produce and implement the training. Multiple by 100.
Reference:
Clark, Ruth C., and Richard E. Mayer. (2012). Scenario-based e-learning: Evidence-based guidelines for online workforce learning. Center for Creative Leadership. ProQuest Ebook Central, https://ebookcentral.proquest.com/lib/boisestate/detail.action?docID=1097790.
Kirkpatrick's Four Levels of Evaluation
Raymond Katzell laid out the following four purposes for evaluating the effectiveness of training.
To determine how the trainees feel about the program.
To determine how much the trainees learn in the form of increased knowledge and understanding.
To measure the changes in the on-the-job behavior of the trainees.
To determine the effects of these behavioral changes on objective criteria such as production, turnover, absenteeism, and waste (Thalheimer, 2018).
Donald Kirkpatrick expounded on this by creating his four step process:
1. Reaction: Surveys should be distributed to both participants in the learning experience and stakeholders to find out how well they liked the learning experience and if they felt it carried out its objective as intended. Typical issues to be measured include how well the learning experience:
Clarified what results were to be achieved
Was delivered appropriately and effectively by the instructor
Made effective use of time
Made effective use of audiovisual aids
Achieved the desired results (Rothwell et al., 2018, p. 125)
2. Learning: This level deals with evaluating how well the participants have learned the material. Tests can be used to measure results.
3. Behavior: This level wants to go beyond whether or not the participants learned the material and wants to know if they have actually applied it on the job. Evaluations need to take place several weeks or even months after training to see if transfer of training has occurred and the desired workplace changes have actually taken hold. Human performance improvement practitioners can do a structured behavioral observation where they “visit the work setting and watch what happens as individuals work with the area targeted for a performance improvement intervention” (Rothwell et al., 2018, p. 136).
4. Results: This evaluates how well the learning intervention affected business goals. Did profits increase? Did turnover rates decrease? Did the company waste less resources?
--
One note: from some perspectives the order of use should be reversed. According to Wendy Kirkpatrick, it is best to start from the “results” level so that the human performance practitioners can identity what business goals need to be met. From there, they should consider what people within the organization need to do to help reach those goals. Then, consider what they need to learn. Finally, after a learning intervention has taken place, its time to take toll of how the participants and stakeholders felt about the intervention (Sage Media, 2018).
These tools help demonstrate to stakeholders that the time and resources used for a learning intervention are well spent. It justifies the existence of a human performance improvement department.
References:
Rothwell, W. J., Hohne, C. K., & King, S. B. (2018). Human performance improvement: building practitioner performance (3rd ed.). Routledge.
Sage Media. (2018, September 5). Designing training with the end in mind - sage advice: episode 37 [Video]. YouTube. https://www.youtube.com/watch?v=udA7TGrqRGc&t=50s&ab_channel=SageMedia
Thalheimer, W. (2018, February 1). Donald Kirkpatrick was NOT the Originator of the Four-Level Model of Learning Evaluation. Work-Learning Research. https://www.worklearning.com/2018/01/30/donald-kirkpatrick-was-not-the-originator-of-the-four-level-model-of-learning-evaluation/