IDeATe Gallery Redesign

In this project, we explored the potential of open portfolio-making in the learning space of Carnegie Mellon University’s Integrative Design, Arts and Technologies (IDeATe) program. We redesigned the IDeATe Gallery, an online portfolio platform for IDeATe students, to scaffold the feedback process and create a supportive community where novices and experts can combine their expertise and build projects together through progressive and targeted feedback.
Date
Sep – Dec 2017
Team
Lu Yang, Ye Jin Lee, Mengxin Yu
Domain
Learning Media Design

MY ROLE

I spearheaded research and user testing, as well as synthesized insights from each round of research. I designed user flow, high-fidelity UI screens, and made a prototype through Invision.

THE CHALLENGE

IDeATe Gallery is an online platform where CMU students can share their work and practice documenting projects. Students thought of IDeATe Gallery as a space for showcasing polished work, and they were intimidated by this idea and were not participating in the community. However, Gallery is intended to be a learning tool where students can learn to document and curate while building portfolios, as well as understand the iterative nature of design process. We aim to explore ways to engage students and help them develop the skills to build professional portfolios that are important parts of initiating their career.

INTRODUCE THE NEW GALLERY

The new gallery helps novice students invite expert students in the community to give feedback on their work-in-progress. We imagined multiple feedback loops in the process to help students learn the iterative nature of portfolio-making and build their projects with progressive and constructive feedback from experts. Experts can be any member of the IDeAte community because IDeATe students come from different disciplines and therefore are experts in their own domain.

Katherine

A novice in physical computing

Katherine is working on a project that uses outdoor sensors. She starts asking her question in physical computing by inputing her project hashtags. The system displays the relevant IDeAte area, tools, skills and projects. For her hashtags, she selects physical computing and sensor from the suggestions, and types in “park walk.” Then she checks if there are similar questions that have been asked. If not, she adds a project description and uploads project images in the next step.
Katherine clicks on the images and adds questions there. Then she invites experts to answer her question. 
IDeATe students who have answered ten questions with the physical computing hashtags will gain a crown in the area. The expert list is ordered by the number of questions answered, crown status, and social distance (classmates will be shown first).

James 

An expert in physical computing

After receiving Katherine’s invitation, James decides to respond to her question. James are encouraged to give Katherine actionable solutions. If he feels stuck, he could check common issues in physical computing to scaffold his feedback.
Katherine receives James’ feedback, and she likes it and replies to it. She can also see who else has answered her questions, and clicks on the orange bar to see feedback. Prompts are given to help her construct replies. At last, Katherine is invited to answer Emily’s question on laser cutting to pay it forward, as Katherine is an expert in this area.

THE PROCESS

UNDERSTANDING PORTFOLIO MAKING

To better understand portfolio-making process, we initially interviewed four students with varied experience in portfolio making and conducted think-alouds with them while going through their websites. We synthesized data using affinity diagram and  found that portfolio-making, in its simplest form, entails documentation, curation, and presentation. We used the synthesis model below to represent conceptual, dispositional and motivational needs of portfolio makers. 

UNDERSTANDING STAKEHOLDER VALUES

We met with Kelly Deleany, Assistant Director of the IDeATe program, to gather a better understanding of IDeATe. Kelly explained IDeAte is a space where different departments from around the university converge to learn from one another through collaboration. We then interviewed Daragh Byrne, who heads up the IDeATe Gallery, to learn how Gallery was intended to help students practice portfolios. We understood that the two drivers of Gallery were documentation function and comment function, and currently the comment function was not active. We also learned that Daragh hoped Gallery to be a space where students learn portfolio-making process, and transfer what they learned to make highly-curated professional portfolios.

The primary goals of IDeATE Gallery are to let students practice documentation and get feedback and visibility on the work. Gallery is ultimately a learning tool.

Kelly Deleany

IDeATe is about learning through collaboration. Students are encouraged to try something new, producing work they would not have in their more formalized primary areas of expertise.​

Daragh Byrne

PICKING UP THE PIECES

After having an understanding of the portfolio-making process and students’ motivations behind it, we started our site study in the IDeATe maker spaces. In order to learn how Gallery was being used by students and professors, we conducted guerrilla interviews with three students and one professor. To our surprise, only one of the students heard of Gallery and the student believed Gallery was used for showcasing the best students’ projects. The professor was not using Gallery for his classes because he already built a class blog to collect students’ work. 
We wondered, “How might we help students become more engaged in the IDeATe Gallery?”
Students in physical computing lab making a robot
The Recently Discussed section in Gallery was not active.
Interviewing with an IDeATe student

INTERVIEWING IDeATe STUDENTS

We contacted nine IDeATe students and alumni who had their own portfolio site or had projects showcased in Gallery. We invited them to tell their experiences with Gallery and portfolio-making process, as well as the challenges they faced building different projects.
We found students we interviewed fall under two user groups. The pink group represents novices, who are constantly seeking feedback on their projects and less worried about site customization. The blue group represents experts in portfolio-making, who values customizing their sites and self-evaluating their process. We made two personas, Katherine and James, to represent the novice and expert user in IDeATe. 
One of the novice users asked for feedback on his project on Gallery. Unfortunately, the student has not received any feedback yet after eight months. 

FRAMING THE PROBLEM

From the interviews with stakeholders and IDeATe students, we learned that students believed that Gallery is for showcasing polished work. However, Gallery is intended to be a learning tool where students can learn to document and curate while building projects. Novice students strongly desire feedback on their projects, but the feedback function was not working. Therefore, we decided to explore how we could build a supportive community through the right touch points that bring people back to participate. 

How might we build a supportive community where feedback and opinions can be given in a way that is helpful?

We visioned multiple feedback loops in order to help students learn the iterative nature of portfolio making, and create a supportive community where novices and experts can contribute their expertise and build projects together. 

MAPPING THE LEARNING EXPERIENCE

We layered the cycle of engagement — aware, join, use, develop and leave — onto the portfolio documentation phases as shown below. Through tracing two personas (expert/novice) across the experience with a set of storyboard panels and touch points in the channels, we narrated how our users could all evolve in the iterative portfolio-making process. 
The circular elements are suggestive of the feedback loops. We see the whole process as multiple iterations that are growing in scale. There are three rounds of feedback given 
to novice at different stages in their portfolio-making process: 1) After documenting an idea,  2) After building one project with multiple documentation, 3) After composing different projects into one cohesive narrative.

Evidence from Literature Review

  • Breaking up review tasks in stages could help broaden focus.
  • Showing drafting process could deepen feedback. 

DESIGN PROBE IN IDeATe Space

We focused on the first feedback loop in the journey map, where students get the “Aha” moment and need feedback on their initial ideas. We adopted a design probe method to understand the subtle qualities of feedback experience. We wondered,  “How are IDeAte students going to engage in feedback?”
We designed two posters to probe students’ interactions with image-based prompts using a laser-cutting project as a prompt to gather feedback on our first poster. Our idea was to build a squirrel den for the coming winter, and we designed a second poster in order to enable students to put their own sketches up there and gather feedback using our poster. We hung up the posters in the IDeAte space where students frequently visited (beside the bathrooms), and drew conversation bubbles of different shapes to make it easier for students to give quick feedback.
Sketches of the Squirrel den idea for the poster
Setting up posters beside the restroom in IDeAte space.
Final installation 
After a week, we collected the posters, and were happy to see there were three pieces of feedback. The community did want to help! The feedback was very targeted towards the questions we asked and used emojis. The feedback post-its were put right on the parts of the sketches where question were asked. Therefore, we decided to continue on this path and design image-based interactions.

PROTOTYPING IDEAS

Using paper prototype, we wanted to understand:
  • Do novices feel comfortable uploading early sketches for feedback? 
  • What kind of experts do novices want to invite? 
  • What keep the experts motivated to give feedback? 
We used building a laser-cutting cat house as a prompt for the learning experience. Our first user struggled to sketch a cat house and pose questions, so we adjusted the approach, asking students to think of their passionate projects and invite people on Gallery for feedback. This time, we were able to see real questions students would ask. By giving users some voice, we let them draw sketches they can relate to and ask questions that truly matter to them.
We found:
  • Novice students were comfortable uploading early work as long as they could get feedback. 
  • Novice students want to invite students with related IDeATe projects or with relevant credentials for feedback. 
  • Expert students are motivated to give feedback for topics they are passionate about. They want to get inspired and pay back the community. They also expect follow-up from novice students.
Novice student sketching ideas
Novice student posing questions
Novice student inputing hashtags
Novice student inviting experts

DESIGNING A NEW GALLERY

I incorporated feedback gathered from low-fidelity mockup and insights gained from more literature review in designing the high-fidelity mockup. We conducted usability test with three IDeATe students using InVision and further improved the on-boarding experience in terms of navigation, feedback management and language used.
A user was typing feedback using our mockup.

STAKEHOLDERS LIKE THE NEW EXPERIENCE

I love image-based interactions and the hashtag sorting components. I see great potential in hashtag to code the type of feedback desired and the type of documentation existing in the portfolio.

A stakeholder invested in portfolio practice

It is effective how you are customizing the experience for novices and experts. I like that it serves as a scaffolding tool and encouraged reflection from the beginning, awkward phases of a project.

A professor from the School of Design in CMU

FINAL TAKEAWAYS

Our hashtag ideas and image-based interactions were received very well by educators who attended our final presentation, and I really had fun exploring the peer-to-peer feedback process. Going forward, here are the areas I would love to learn more about.

Justify the expert status

Our current mechanism gives users expert status once they answer five questions with the same hashtag. However, users may answer questions out of interest rather than expertise, and one’s interest in a field does not necessarily equate with expert status. Possible ways to justify the expert status include counting the number of “likes” in the hashtag system toward expert status to ensure their answers are useful.

Change the mindset around portfolio making

Helping students change their perception of portfolio making from a polished and finished product to an iterative and messy process is challenging. The IDeAte Gallery itself is not sufficient to change this mindset — the conceptual change needs the support of the IDeATe class instructors. We could introduce the new Gallery in the classrooms so the instructors could model the process by giving feedback on the Gallery projects during class critiques. Another idea is to put the projects back to the maker spaces, such as digitally showcasing the ongoing projects from the Gallery and inviting students in the same spaces to interact with the digital screen.