Our team partnered with a Fortune 500 educational publishing and assessment services company on a 7-month project, aimed at informing features for the next iteration of their reading intervention software program. The goal is to improve student motivation, accountability and enabling a personalized learning experience through the software. We created Endeavor, an adventure-style interactive narrative game to help students become competent and confident in reading.
Jan 2018 – Aug 2018
Lu Yang, Zach Mineroff, Roger Strang, Julia Ridley, Chenxin Wang
Learning Experience Design; Web Design; Mobile Design; Game Design
As the sole designer of the team, I collaborated with project manager, researcher, content director, and developer to build Endeavor. I drove the end-to-end UX design process from research, ideation to implementation. I was responsible for building models, user flows, analyzing competitor products, creating wireframes and designing UI screens. I also designed our spring and summer reports. Please feel free to contact me if you are interested in seeing the reports.
LOW STUDENT ENGAGEMENT WITH THE SOFTWARE
For beginning decoders, reading takes a lot of mental energy. Reading longer passages can be stressful and tiring, leading to demotivation. The reading intervention software supports struggling readers and follows a natural progression of instruction through the skills required to read full paragraphs. However, some students have to force themselves to pay attention to the software. These students are at-risk for never learning to read because they are not enjoying what they are learning.
IMPROVING EXPERIENCES FOR KEY STAKEHOLDERS
We understand from the stakeholder map that the school board are HMH’s client, who will renew and purchase licenses from HMH if the softwares they provide improve student learning outcomes. The account executives and software coaches help HMH sell the product and train teachers to use it in the classroom. They key influencers in our map are teachers and students as they use the software on a daily basis. Because these students are from low-income families, parents tend not to play a key role in the student’s learning experiences from our research.
Provide students with stories of various reading levels
By providing students with a choice of stories, Endeavor enables learners to engage with content that appeals to their interests. The available stories are based on the students’ current reading level, ensuring they will not get discouraged by overly-demanding text.
Empower students with narrative choices
Students make choices in the narrative, giving them a sense of control over their learning experience. A progress bar at the bottom lets students know how much they have read and how much longer they have to go.
Check students’ reading skills through embedded comprehension questions
Comprehension questions are embedded in the narrative, making them seem more like a game than a test of their reading skills. The choices that students make have consequences in the narrative, where incorrect answers lead to poorer outcomes.
Assess students’ weak skills through embedded word challenges
Students who answer a comprehension question incorrectly either do not understand the target word or they could not decode it. Embedded word challenges help make this distinction, and can provide teachers with data on how to personalize future lessons.
Pinpoint the exact confusion through easier word challenges
If a student fails an initial word challenge, we can hypothesize that it was due to a confusion between one or more sounds. To pinpoint the exact confusion, we can examine their answer choices on subsequent word challenges.
Present achievements and performance students care about
At the end of the activity, Endeavor shows students how well they did throughout the story. Metrics like stars, coins, and items encourage students to think about their choices next time.
With the companion mobile app, students can read and interact with stories anywhere and anytime.
The Product Development Process
REVEALING AREAS FOR IMPROVEMENT
To reveal areas for improvement of the software, we conducted contextual inquiries to understand how teachers and students are using the software in the classroom.
Through affinity diagramming, we learned from the data collected in our interviews and observations.
PERSONAS & LEARNING JOURNEY
We created teacher and student personas based on data, and a learning journey to better understand students’ pain points for each of the key instructional elements in the software.
UNCOVERING STUDENTS’ NEEDS
The followings are selected insights, tied to our areas of focus: engagement, personalization, and accountability.
Students may lose confidence in their ability to complete certain activities when the task is clearly too difficult.
A student like Sarah spent well over ten minutes on one trial of a word activity. Without additional scaffolds, she felt as if she would never get it right. A student like Daniel had been stuck on the same spelling module for over two weeks and expressed frustrations and disbelief in himself.
Feedback in the form of direct instruction is less effective if it’s repeated several times after the same mistakes.
A student like Sarah kept spelling the words “tunnel”, “channel” and ”kennel” as “tunnle”, “channle”, “kennle.” Instead of providing feedback on the common error (flipping “el”), the software simply offered the same direct instruction after every mistake.
Teachers like Mrs. Grant would often remedy this issue by employing their own, personalized instructional techniques.
Some students have significant trouble combining low-level skills in more complex tasks. Proper scaffolding is needed to bridge the gap between learning phonemes or word strategies and applying them in the context of a sentence/paragraph.
During a sentence-recording activity, a student like Sarah stopped reading at the first word she didn’t recognize. Instead of trying to apply a word strategy she learned earlier, she just stopped and put her head down. Another student like Daniel, experienced this kind of frustration so often that he tried to “glitch out” the software by furiously clicking on the record button.
While students value being able to track their progress and see evidence of their own learning, they feel as if the progress tracking screens are irrelevant.
Students like Sarah, Daniel, and Devon all seemed to have an automatic reflex to click the continue button as soon as a progress-tracking screen appeared, even if just a few minutes earlier they told us that they care about their scores and progress.
Teachers like Ms. Grant have created their own forms of progress tracking that their students respond well to.
IDEATION OF PRODUCT CONCEPT
How might we support students’ intrinsic motivation for reading while giving them appropriate practice?
How might we design personalized instruction that is appropriate for students’ reading levels and provide teachers with useful data?
We came up with different ideas, which are a combination of differentiated feedback for students’ errors, enabling teacher input, integrating game choices, and creating a storytelling chatbot.
DESIGNING A STORYTELLING GAME
We presented our pitch ideas to our client at the spring meeting. We evaluated the impact for users and feasibility of each idea with our client, and were able to prioritize features to build. We were all excited to design an adventure-style interactive narrative game for our struggling readers. Our goal through the game is to motivate students to practice skills they struggle with the most, enabling them to become competent and confident in reading long passages.
First, I conducted competitive analysis of the following six products to understand best practices for integrating gamification elements into skill training and interactive storytelling. The analysis is based on the gamification model we created from literature review.
We made a functioning prototype in a week in order to quickly test our branching narrative concept. We used user story to help define what our minimal viable product features should involve.
While I’m reading an interactive story, I want to be engaged by the story and the choices so that I feel motivated practicing words and understand how the same activities in the software help me learn to read.
MVP FEATURE LIST
1. Write a narrative story with multiple paths for students to choose from.
2. Embed word challenge activities from the software into the narrative.
3. Create a helpful learning buddy in the form of a Robot.
PROOF OF CONCEPT
We went to a school district and tested 5 students currently using the software.
We found that…
1. Students like the story and are excited about choosing their own path.
2. They like “Robby”, the learning buddy.
Evidence of engagement
1. Students choose to repeat story with different choices.
2. Students put themselves into the story when considering different choices. For example, one student chose “boat” over “rocket ship” because boat was safer to her. Another student didn’t choose to “press the buttons“ because he thought “It’a really bad idea to start pressing buttons when the ship is going to crash.”
3. Mouthing words and using mouse to follow text as they read.
EMBED PERSONALIZED INSTRUCTION & GAMIFICATION
For our mid-fidelity prototype, we added spelling and comprehension questions as students wanted multiple types of challenges. We provided personalized Instruction in the form of videos when students get questions wrong. We also embedded gamification elements to help students stay motivated.
REVISED USER STORY
While I’m reading the adventure story, I want feel engaged and have personalized experiences practicing activities, so that I can read longer passages and increase my confidence in reading.
MVP FEATURE LIST
1. Create personalized instruction on spelling activity and word challenge.
2. Make comprehension questions to test engagement.
3. Use gamification elements (hearts, points, collectibles, and customization of the learning buddy Robby) to hold students accountable for the practice activities.
DESIGNING USER FLOWS
USER TESTING ON ACTIVITIES & GAMIFICATION
We tested the new prototype on 6 students.
We wanted to understand…
1. How do students feel about the gamification elements?
2. Do the activities feel more integrated or more separate to the story?
We found that…
1. Students were positively engaged by hearts and points. Students prefer to exchange points for items after reading the story because they could get better items with more points. Some students prefer customizing Robby to collecting items.
2. Two out of six students said the activities felt separate. Others feel activities went along with the story. .
THE DANGER OF CHOCOLATE-COVERED BROCCOLI
As we set out to develop a high-fidelity prototype, we received feedback from our client and faculties. Both raised concern on “The activities felt separate from the story.”
In the previous prototypes, we interleaved existing practice activities with the story.
We made this decision because…
1.Our goal was to encourage students to see the value in existing exercises in the reading software, so they would be more likely to engage with these activities outside of the game.
2. Having a set number of practice activities at fixed points is easy to scale. Content writers could roll out stories on their own timelines and just indicate where to embed the challenges.
However, though students enjoyed the story, and expressed the activities made sense in the context, we were not sure if they enjoyed the activities. Gamification elements held students accountable for the activities, but there is no guarantee that the students actually read the text and comprehend it well.
The question for us became “How can we make sure our activities are not chocolate-covered broccoli?”
DEVELOP A STORY WITH NARRATIVE CHOICES & CONSEQUENCES
We shifted our focus to integrating practice activities into narrative as a whole. The choices reflect mastery of reading skills and have narrative consequences.
We made this decision because…
1.This will make sure the core feedback loop is about reading the text.
2. Using existing practice activities is less focused on engaging with the narrative and feels disconnected with the story.
3. Wedding the assessment to the narrative will limit the practice activities to those who need it.
REVISED USER STORY
STUDENTS: I want to have an immersive experience reading the story and making my choices. I want to feel supported and hold myself accountable in order to go down a good path in the story.
TEACHER: While I’m teaching, I want to use data to inform my instruction, so that I can tailor my instruction to the words my students are struggling with.
MVP FEATURE LIST
1. Narrative consequences that motivate students to read.
2. Comprehension and word assessments that hold students accountable to read the texts.
3. “Read it to me” function that helps students decode words when they need support.
CREATING A STORY ARC WITH SCENES
We took a modular design approach and conducted a co-design session to brainstorm on scenes in the story arc that can serve as assessments across stories.
A sample story arc with different scenes.
Co-designing story scenes
USER FLOW WITH NARRATIVE CONSEQUENCES
The following user flow shows a students’ journey going through each story scene. The goal of having integrated comprehension questions in the flow is to keep students accountable for reading the text, and the three levels of word challenge questions will help provide data on the specific skill students struggle with. The flow also diverts students into various story branching based on their decoding skills, enabling narrative consequences that are personalized for each student.
DESIGNING STORY SCENES
We created an island treasure hunting adventure story and came up with four story scenes that work together to guide students find the treasure. The story scenes are “Siren’s song”, “Natives”, “Treasure” and “Danger (pirate attack)”. Based on our new user flow, I made the high-fidelity screens for “Treasure” and “Danger”. Then we conducted another round of user testing before committing to designing the four scenes.
The following user flow with hi-fi screens show how the “Treasure” scene would work in the story.
After we validated the story scenes with students through user testing, I continued to design the rest of the scenes. The following shows the other three scenes we weaved into the story as well as items students can collect on the way home.
Credits: The robot, room, island, space rocket, and zoo illustrations are designed by Freepik.