eLearning Object Review #4: Reverse Engineering eLearning Design Documents

Car Sale Dialogue

I found this eLearning object by searching for “branching scenario e-learning example” on Google. I clicked through a couple of dozen examples before I found one that was a true branching workplace scenario that I wanted to work with. This example was included in an article that also mentioned a few other eLearning objects I was familiar with and have previously analyzed.

Course topic

This eLearning object is a branching simulation that allows salespeople to practice their technique and communications skills in a low-risk environment before transferring their skills to real customers. This simulates a real-world conversation to challenge the learner.

Problem or opportunity

The learner is presented with a potential customer at their auto dealership. The client is looking to buy a new car and require the salesperson’s expertise and rapport to guide them during their decision.

Expected learning or performance outcome

The learner is expected to engage with a client in a productive and efficient way that supports the client’s needs and the dealership’s goal of selling a new car.

Relevant characteristics of the target learner audience 

The target learner is a salesperson with at least some experience in the field. They can be of any age, from any cultural background, and with any education level. It is assumed that learners know how to use and can access a computer. This learning is asynchronous, so can be completed according to the learner’s schedule, but can potentially take close to 20 minutes to complete.

Environmental analysis

Learners are potentially limited by their working schedules and their geographical location. Asynchronous eLearning allows them to access the content from any location at any time. eLearning is also preferred while learners practice skills that could be costly in the working environment. In this case, learners can practice selling cars during a simulation instead of to actual customers that they could lose business.

Required materials

This course requires learner access to an internet-enabled computer or mobile device. Because of the excessive text and potential length of the module, mobile devices are not recommended.

Design specifications

Testing/Certification

While there is not an explicit testing strategy, learners are required to master techniques or communication and sales that they encounter during the course in order to reach the conclusion and desired results. There is also a gauge of the client’s satisfaction that learners can use to navigate responses and keep the customer happy.

Knowledge and/or skill type

This module elicits decisions made, cues used, and monitoring. Some choices the learner encounters describe the rationale behind the response (decisions made), all responses show the customer’s body language during the conversation (cues used), and the duration of the module shows the customer’s satisfaction with the interaction (monitoring) (Clark, 2013, p. 165).

Learning domains 

This module practices the interpersonal learning domain, with the end-task deliverable being to “communicated effectively to achieve operational goals,” (p. 37). The learner will engage the client, respond to client needs, and select optimal solution.

Assessment strategy

This module tests for far transfer open tasks (p. 128). These tasks have no single correct answer and will always be different in the field than in practice. Therefore, the module has many opportunities to take detours and go on tangents, while looping back to the optimal path. It also includes many opportunities to scare the client away and lose the sale, much like in real life.

Trigger event

The scenario kicks off with the salesman recognizing a new client in the showroom. who is looking to purchase a new car. Immediately the learner is asked how to interact with the client.

Guidance techniques

This module has very few guidance techniques. Luckily the interface is limited and the responses are closed, as users are directed to click on the best response. Learners are practicing their communications skills, so there is nothing in the environment nor any computer software they need to practice interacting with.

Advisor type

The module gives constant feedback and suggestions to the learner as they make their way through the scenario. The feedback lets learners know what a more optimal choice would have been, or lets them know when they have selected the best answer and are on the right track. There is no final feedback given because there is only one correct outcome. I do think the module missed an opportunity to give the learner an overview of the learner’s choices compared to the most optimal choices.

Branching scenario path 

I used LucidChart to map out the flowchart of the module. Admittedly, I did not expect how long this module was going to take to map out. The orange shapes indicate the learner’s options and the white shapes indicate the client’s responses. Feedback is put on the directional arrows.

View fullsize

E-Learning structure/content outline

This eLearning includes one unit.

Training design matrix

• Scope statement: <Enter a brief statement concerning the scope of the lesson and delete this text.>

• Terminal Learning Objective (TLO): Learners will sell a car to an interested client.

• Enabling Learning Objectives (ELO):

  • Learners will display mastery of the product and competitor products’ information.

  • Learners will be polite and maintain professionalism while interacting with clients.

• Lesson Topics:

  • Instructional Strategy: Learners will receive feedback specific to their choices during the module. There are no outside resources or coaching sessions given to learners prior to using the module. The module could offer feedback in terms of how an expert would approach the conversation to guide the learners.

  • Assessment  Strategy: Successful learner performance is demonstrated by their ability to navigate to one correct outcome during the scenario.

Estimated contact time

• Introduction/Summary/Objective: .5 min per slide

• Content Presentation: .5 min per slide

• Knowledge Check (KC) / Practical Exercises (PE): 0 min

• Satisfaction survey: 0 min

Estimated time to complete ideal outcome: 5 min

Estimated time to complete typical outcome given that the learner will encounter an undesirable outcome(s) and restart: 10 min

Estimated time to complete all outcomes: 30 min

References

Clark, R. C. (2013). Scenario-based e-learning: Evidence-based guidelines for online workforce learning. (Links to an external site.) San Francisco, CA: John Wiley & Sons.

Adapted from Giacumo, L. eLearning High-Level Design Document.

Searching for Models and Storyboard Options

Below are the storyboard templates that I have worked with in the past. As you can see, both are geared towards a more visual storyboard.

The first example here was used in a large team where the ID would pass off a storyboard to graphic designers and programmers. Therefore it includes a lot of details for the interactivity, programming notes, and graphic notes. This is not the document that would be shared with stakeholders because of its level of detail. We would share with them the text and source graphics that would inspire the final version and ask for their feedback on that. The modules we created were custom eLearning built from scratch, so it was necessary to have as much detail as possible included. We could include everything from video, games, PDFs, etc. and needed a versatile storyboard that could communicate all the different media.

---

This example was used on a much smaller team where each individual ID was expected to be both designer and developer. There are fewer notes about interactivity and visuals because it is mainly used to communicate with SMEs and clients, and the IDs would add in the details as they developed the final modules. We mainly developed in Captivate and Storyline, so there is a limit to how much customization we could use.

---

Both of these storyboard templates were used by organizations that I joined. While I trust the expertise of the teams I joined, I recognize the importance of being open to improvements. I am a fan of a storyboard that includes more rather than less detail. We typically update our modules every year so often a new developer will take on the updates from an old developer. Documenting as many details as possible helps the new developer understand the decisions made and the programming specifics.

As far as providing wireframes to our clients, I am a fan of using Adobe XD. With this tool, you can easily and quickly program custom buttons and actions that branch to other screens. Check out this article for more information about how XD can be easily used for storyboarding and UX design. I have also used LucidChart to map out wireframes live with stakeholders. This saves time by gaining immediate feedback.

Relating to and Supporting Clients

No matter how skilled an ID is, if they don’t practice the skill of communicating and working with clients and SMEs, they are limited in what results they can produce. I would argue that establishing a relationship with collaborators is one of the most important aspects of working as an ID.

“You should always welcome new SMEs to the process of instructional design. Ensure they know how much they are appreciated and how important they are to the process of curriculum design. ”

— Chuck Hodell

While I am new to working with clients as an Instructional Designer, I have spent my career serving clients and their needs. As an artist, I was tasked with bringing a client’s vision to life while directing them with my expertise. As an ID, I am learning that the communication skills are the same and I am grateful to already have the practice.

I am fortunate that the manager that supported my transition into ID was incredibly adept at navigating team and client relations and I learned a lot from her. My team is currently managing several high-priority clients under her oversight. Some of her practices that I try to incorporate include:

• Over-communicating: Our clients are typically doctors or heads of departments that are very busy and very hard to pin down. It is a delicate balance of giving the client all the information they need but not too much that they gloss over the details. It helps to be concise, highlight or bold the important takeaway, and suggest action steps to them. If we have questions, we schedule meetings for only as long as we need and respect their time by starting and ending on time. Providing agendas prior to the meetings helps the client prepare for the meeting and keep on task.

• Document everything: Because we have to work within our clients’ availability and workloads, we are always sure to document all communications and decisions. This is a way to not only protect ourselves but also help to remind clients of past decisions and requests.

• Bring them into the process: We have the most success on a project when we describe our process and decision-making to the client. We keep it high-level but allow them to see the “why” behind what we do. This helps establish our credibility and cements us as learning experts. Along these same lines…

• State your value: As a part of our introduction process to new clients, we make sure to mention what we can do for the client based on what we have done for clients in the past. We focus on the results we have achieved so that our clients can trust us to do the same for them. It also helps up establish and maintain our value to the enterprise.

• Encourage feedback: We want our clients to want to work with us again. So making them feel comfortable and heard is of the utmost importance. We explicitly ask them how they prefer to work, what communication channels they use, when and how they want to give feedback. This way we can adjust our methods and communications to serve the client. We make sure to ask at every meeting if there is something we can do better or if we are not meeting expectations in some way. We also want them to share with us what they appreciate and would do again.

I also find that these suggestions can go a long way in supporting a team. Admittedly, I tend to be shy when meeting new people and usually try to respect their ideas and expertise by staying silent. But I have learned that doing so is a disservice to my teammates and myself. Raising questions, however small, is the best way to strengthen a current solution or to redirect to a better solution. Either way, questions are welcomed! And as I grow in my career, I learn to trust my instincts and expertise more as well which makes me a more confident teammate. I look forward to growing in my knowledge and confidence as I progress.

References

Hodell, C. (2013). SME From the Ground Up: A No-nonsense Approach to Trainer-expert Collaboration.

eLearning Object Review #3: Analysis, assessment, and ID review

A Support Net

I found this object by googling elearning scenario branching example "storyline". I found a few that were over 7 years old and were no longer supported by flash, so I decided to search for something more recent. I specified that I wanted to only view results from the last five years. I liked this article because it included links to examples, including one of my favorite examples, Haji Kamal. The description of “A Support Net” specified that it was a “choose your own adventure” style scenario eLearning, which interested me.

---

Analysis

What workplace performance does this scenario-based e-learning support? (Clark 2013, ch 1)

• Accelerate expertise

• Build critical thinking skills

• Build skills impossible/impractical to gain on the job performance

• Promote learning transfer

• Gain a return-on-investment

• Motivate learning

• Exploit technological resources effectively

• Engage a target audience that already has basic job familiarity

What are the instructional goals? (Clark 2013, ch 1)

This course is part of a larger curriculum, “Making Sense of Mental Health Problems,” designed to help social workers engage and understand mental health problems. These modules encourage learners to explore all possible causes and influences in a mental health diagnosis and allow learners to practice some challenges they may encounter in the field.

Who are the learners? (Clark 2013, ch 4)

• Novice

• Some experience

• Apprentice

• Experienced

• Mixed

• Other

What are the scenario-learning domain(s)? (Clark 2013, ch 2)

• Interpersonal skills

• Compliance

• Diagnosis and repair

• Research, analysis, and rationale

• Tradeoffs

• Operations

• Design

• Team coordination

• Other

What are the terminal learning objectives? (Clark 2013, ch 4 & 7)

The terminal learning objectives of “Making Sense of Mental Health Problems” are:

• describe key theories and concepts that have informed debates about mental health diagnosis

• outline how diagnostic systems have been developed and implemented

• explain why diagnostic systems are challenged in the mental health field

(from https://www.open.edu/openlearn/health-sports-psychology/making-sense-mental-health-problems/content-section---learningoutcomes)

For this specific module, the terminal learning objective is to respond to mental health problems in impactful and optimal ways.

What are the enabling learning objectives? (Clark 2013, ch 4 & 7)

• Review patient history and personal situations

• Respond to questions, actions, or objections from patients in the appropriate manner that impacts patient

Complexity of responses (Clark 2013, ch 4)

• Number of outcomes

  • One outcome

  • Multiple outcomes: each scenario has four different outcomes of varying patient impact based on the decisions made

• Outcome precision

  • High solution precision: there are right and wrong answers that impact how the patient feels

  • Low solution precision

• Interface response options

  • Limited interface response options: typically two multiple choice response options

  • Multiple interface response options

• Social presence

  • High social presence

  • Medium social presence

  • Low social presence: self-paced, selt-study environment

Scenario settings (Clark 2013, ch 5)

• Office, meeting room

• Computer

• Technical shop, laboratory

• Clinic, hospital, surgical suite

• Equipment and instrument panels

• Factory

• Field site: each scene takes place in a different setting where individuals encounter those in need

• Other

Trigger event (Clark 2013, ch 5)

• Phone call

• E-mail, text message

• Interview

• Failure or crisis

• Murphy’s Law scenario

• Other: each scene shows a person in crisis and the triggering event is them acting out their anger in an inappropriate way

Does your scenario outcome require identification and analysis of data? (Clark 2013, ch 5)

• No

• Yes

Types of guidance provided (Clark 2013, ch 6)

• Faded support

• Simple to complex scenarios

• Open vs. closed response option: response options are limited

• Interface navigation options: there are very few options at any time

• Training wheels

• Coaching and advisors

• Worksheets

• Feedback: feedback about the choices made are given at the end of the scenario where learners can assess their choices by comparing to other potential paths

• Collaboration

Instructional approaches (Clark 2013, ch 7)

• Tutorials

• Expert solution demonstrations

• Questions in demonstrations to promote engagement

• Cognitive modeling examples to illustrate tacit knowledge

• Example repositories linked to organizational knowledge base

• Traditional instructor

• Socratic instructor

• Scenario facilitator

• Other: from what I can gather, this module did not offer any instructional approaches. Because this module does not apply to just experts and can be offered to a wider audience, I don’t think the designers included any instruction and allows the learner to use their own personal experiences.

Feedback features (Clark 2013, ch 8)

• Specificity

  • Specific: this course gives learners specific feedback about their decisions while also allowing them to see other possible outcomes

  • General

• Type

  • Instructional

  • Intrinsic: this module is built around giving intrinsic feedback to learners’ choices. Media in the form of video allows learners an immersive environment to see with effects of their choices.

• Frequency

  • Immediate: this module contains both types of feedback frequency. The immediate feedback is given in the form of an impact meter that always shows above a patient video.

  • Delayed: the module gives details and personal feedback at the end of the module so that the learner can see the overall impact of their choices.

• Focus

  • Solution: while the module recognizes that solutions to mental health crises are varied, it does base its feedback on the ideal response.

  • Process: even though there is an ideal solution, this module allows learners to make mistakes during the process and practice asking the right questions.

  • Learning

---

Assessment Rubric

Criteria	Exemplary	Minor Concerns	Serious Concerns	Score
	3 points	2 points	1 point	Enter
Use of scenario-based eLearning	Scenario-based e-learning content is for learners with some prior experience and supports one or more of the following: rare occurrence tasks, critical thinking skills training, strategic tasks, compliance-mandates, to compress time, or manage risk	e-Learning content may support learners with no prior experience but does support at least one of the following: rare occurrence tasks, critical thinking skills training, strategic tasks, compliance-mandates, compresses time, manages risk.	It is unclear why a scenario-based e-learning design was chosen.	3
Complexity of responses	The complexity of responses are appropriate for the learning goal, learners expertise, and motivation levels	The complexity of responses are on target for the learning goal but not for the learners expertise and motivation levels	The complexity of responses are not appropriate for the learning goal, learners expertise, or motivation levels	2
Interface response options	The interface response options are appropriate for the learners’ expertise level and learning objectives	The interface response options are a bit of a stretch for the learners’ expertise level and learning objectives	The interface response options are inappropriate for the learners’ expertise level and learning objectives	3
Scenario settings	The scenario setting(s) is/are appropriate for the scenario-learning domains, learners, and learning goals.	The scenario setting(s) is/are a bit of a stretch for the scenario-learning domains, learners, and learning goals.	The scenario setting(s) are inappropriate for the scenario-learning domains, learners, and learning goals.	3
Trigger event	The trigger event is appropriate for the scenario-learning domains and goals	The trigger event is a bit of a stretch for the scenario-learning domains and goals	The trigger event is missing or inappropriate	3
Types of guidance	The types of guidance are varied and appropriate for the learners’ expertise levels, scenario-learning domains and goals	The guidance is appropriate for the learners’ expertise levels, scenario-learning domains, and goals	The guidance is not the best match for the learners’ expertise levels, scenario-learning domains, and goals	1
Instructional approaches	The instructional approaches are appropriate and varied for learners’ expertise levels, motivation, prior knowledge, scenario settings, domains, learning goals, objectives	The instructional approaches are appropriate for learners’ expertise levels, motivation, prior knowledge, scenario settings, domains, learning goals, objectives	The instructional approaches are not the best for learners’ expertise levels, motivation, prior knowledge, scenario settings, domains, learning goals, objectives	1
Critical thinking	Actions taken, decisions made, cues used, rationale, rules of thumb, and monitoring, are used throughout the e-learning to support learners’ critical thinking.	Multiple different content-sensitive learner actions, decisions, or rationale, are required throughout the e-learning.	Content-sensitive learner actions or decisions are only required in one or two spots in the e-learning.	3
Feedback	All feedback (i.e., Intrinsic, instructional, delayed, immediate, specific, general, solution, process, learning, reflection, checklists, rubrics) designs are all appropriately provided for learner actions and feedback is integrated throughout the scenario.	A variety of feedback types are provided and appropriate for learner actions.	Feedback is limited or not appropriate.	3
Interface	Navigation is intuitive	Navigation instructions are clearly explained	Navigation is difficult	3
Interactions	All function properly.	---	Do not all function properly.	3
Chucks	Content is chunked into small enough pieces that you can easily follow but doesn’t interrupt the flow.	Chunks are large but you can easily navigate to where you left off.	Chunks are large and there is no way to get back to where you left off or so small the flow suffers.	3
Progression	Is logical and elegant throughout the object	Is logical throughout the object	Seems disjointed or does not build on previous screens	3
Engagement	Multiple motivational engagement elements are used (e.g., stories, images, examples, narration)	Only one or two cases or story is/are used but it/they include(s) multiple relevant images.	Stories or cases are not used, only brief examples. Images may or may not be relevant.	3
Images or video	Good quality (e.g., focus, lighting, background)	Mediocre quality; you can generally tell what they are but one or more is/are difficult to see or interpret	Poor quality; at least one image or video is too small or very blurry.	3
Audio	Good quality (e.g., volume, tone, pace, inflection, no distractions)	Mediocre quality; you can make adjustments that allow you to access the information	Poor quality; you can’t hear some or all of the audio	3
Length	Module(s) is/are 6 - 15 minutes	Module(s) is/are 15:01 - 20 minutes	Module(s) is/are longer than 20 minutes	3
Accessibility minimums	Screen descriptions, closed captions, image alt tags, are provided and logical.	Closed captions, image alt tags, are provided.	No clear evidence of accessibility considerations in e-learning object.	3
			Total point score	49 (Better, or best)

Qualitative scoring guide

Better, or best = 47 - 54 points

This module is an e-learning exemplar demonstrating significant evidence of effective instructional design.

Good, accomplished = 38 - 46 points

This module meets the basic criteria for e-learning instructional design.

Needs work = less than 38 points

I bet you could offer some suggestions to help improve the instructional design significantly for this e-learning. 

Based on Clark, R. C. (2013). Scenario-based e-learning: Evidence-based guidelines for online workforce learning. San Francisco, CA: John Wiley & Sons.

---

ID Review

Below, I commented on my review of each of the criteria that were determined to be below Exemplary.

• Complexity of Responses

  Done well: The responses were easy to understand and matched the tone of the scenarios.

  Improvements to consider: While it complicates the branching, I think the responses could include more choices to cover the gambit of expected responses.

  Why you think these improvements are needed: It makes the responses more believable and realistic to a learner in the field.

  Guidance on how to make the improvements: Ask novice learners what their responses would be to the scenario and use the popular answers as potential responses.

• Type of Guidance

  Done well: The interface is intuitive and simple.

  Improvements to consider: Offer an option that explains how to navigate through the course.

  Why you think these improvements are needed: Some learners may not be familiar enough with eLearning to comfortably navigate.

  Guidance on how to make the improvements: Include a “help” button (question mark or information icon) or navigation directions at the beginning that displays navigation directions.

• Instructional Approaches

  Done well: The feedback given at the end of the scenario is detailed and actionable.

  Improvements to consider: Consider adding in additional instructional approaches.

  Why you think these improvements are needed: Learners could learn from instruction and comparison to how experts would handle patients.

  Guidance on how to make the improvements: Add a section in the feedback that describes how an expert would navigate through the scenarios.

References

Clark, R. C. (2013). Scenario-based e-learning: Evidence-based guidelines for online workforce learning. (Links to an external site.) San Francisco, CA: John Wiley & Sons.

Adapted from Giacumo, L. Template for your analysis, critique, and assessment.

eLearning Object Review #2: Linear vs Branching Scenario e-learning

Reflecting on my past work, I realize that almost everything I have developed that called itself "branching" was not true branching. At most, we would create what I now know as parallel content which shares a common prerequisite that then divides users at a decision point, each leading to different content. Currently, we are building a module to test compliance understanding that asks users to select their area of occupation: clinical, academic, or corporate. We then lead learners through a series of "scenarios" based on the context of their area. I use the word scenario in quotes there to signify that they are not true scenarios where learners have different decisions that can lead to different content. They are descriptions of situations that ask learners "What would you do?" Learners are presented with two choices, one right and one wrong. Whichever they choose leads to the same feedback for both. The paths converge towards the end to continue delivering the same content regardless of previous choices.

Parallel Content

Example of linear and parallel content used in compliance training

The opportunity to use true scenario-based eLearning is not lost on me. According to Cathy Moore (n. d.), branching scenarios can help learners practice skills such as: challenging their own assumptions; recovering from mistakes in a long complex process; navigating extended and ambiguous situations; deciding when to stop gathering information and act. Practicing how to stay compliant to a corporation's ethics standards and when to act fully lends itself to a scenario-based eLearning to support these goals.

My favorite example of a true scenario-based eLearning is the Haji Kamal module. This was the first time I saw what scenario-based eLearning could truly be. Not only does the existence of realistic options pull learners into the world, but experiencing consequences allows users to learn from their choices. Truly an exemplary example.

---

This week, I reviewed Conversations that Work!

This module is an example of a branching scenario because the learner has the potential to view different content to make a path to the end. There are dozens of ways to navigate through this course by giving different answers to the posed questions. Branching scenarios are ideal for this learning module because the learner is honing a skill that is applicable to ambiguous situations. It is more like real life than a straightforward linear object would be because it allows you to answer incorrectly, try and do better, maybe correct your actions, and overall succeed or fail. I specifically appreciated the access to the STATE Resource to give some guidance to the learner as they move along. The module directs learners to this specific button so they can access the resource when they need it.

Conversations that Work is suitable for use by learners on computers ideally. and mobile devices. I suggest computer because the graphics and text weren’t responsive or scaleable and learners could potentially spend a long time working through the scenarios. Mobile delivery is ideal for small chinks of information spanning 3-7 minutes, meaning this scenario would have to be shortened significantly (Giacumo, 2022). The gestures utilized by the module are only click-based so a user could navigate through it via mobile if they were in a pinch. I discuss more about computer versus mobile design here.

References

Giacumo, L. A. Week 4 Lecture Notes.

Moore, C. (2013, July 7). 4 reasons to use branching scenarios in training. Training Design - Cathy Moore. https://blog.cathy-moore.com/2013/07/when-do-you-need-a-branching-scenario/