Deliverables: Full presentation of my internship works here, ACM Learning@Scale Paper here

Background & Motivation

AlgorithmLABS is a programming education startup that employs tutors to assist students with online algorithm problems. However, hiring tutors can get quite expensive and their qualities can also be inconsistent. During this internship, I worked with another graduate student to develop AlgoPlan, a tool that simulates & automates parts of the tutoring process, with the ultimate goal of helping AlgorithmLABS to mitigate this problem.

Solving algorithmic problems is a complex task that requires programmers to go through multiple stages. Beginners often experience many unique challenges in each stage, from interpreting the problem, planning the solution, to implementing the solution, as well as navigating smoothly between these stages.

My research is focused on the planning phase, where many beginners tend to immediately start writing code without thinking about the high level solution plan (e.g. diagrams or notes).

Studies regarding the lack of planning in novices

Current State

Of course, there exist various tools for providing programmers more support. Some relevant ones and their benefits & drawbacks include:
1. Flowchart generator that automatically creates responsive flowcharts from code. It’s ideal for visualizing the code structure, but it’s also easy to get carried away in the abundance of shapes and colors.
2. Algorithm visualizer that displays the execution of selected algorithms in text and animated forms. This is helps users comprehend the algorithm visual and code wise, however it’s more of an educational tool as learners are users to
3. Compiler that executes and shows variable values at each line of the code, however it’s more of a debugging tool for users to find errors in their code

From left to right: flowchart generator, algorithm visualizer, code compiler
Opportunity areas - enable a smooth transition between each stage

Problem & Goal

All these tools above focus on one specific stage of problem solving. There exist opportunity areas in developing intermediate scaffolds to enable a smooth transition between all stages. We hope to supporting problem-solving with representations that encompasses the whole process, not limited to a specific stage of problem-solving.

The problem statement that naturally follows is…

How might we support novices to properly build high level solution plans and connect them with code?

This encompasses developing representations for high level structures of their codes, mapping these representation to codes, as well as using representations to guide them through the following stages.

Preliminary works

To get get started on this process, I conducted a formative study to observe behaviors when approaching a programming problem for different levels of programmers. My overall vision was to have a single coding environment consisting of the problem statement, representation, and text editor. Based on three takeaways, I formulated three initial ideas for what the high level representation could look like.

Feedback & Plans

I presented the above proposals to my lab, and received initial feedbacks that provided new directions for my next steps. After analyzing the feedbacks, I also came up with a timeline (which ended up being very different from reality haha, and I will get to that in a bit!).


After synthesizing feedbacks, I proposed a new workflow that combines my previous ideas to support this transition between stages surrounding planning.
First is helping learners to decompose problems, which helps externalize their thoughts- addresses some feedbacks from my proposal.
Second is helping learners to use these plans, which helps them a better sense of how their plan relates to code and thus cycle between task and code level thinking.

More concretely, I visualized the ideas in the sketches then proposed a new user flow, with the two colors indicated the two phases, and grey indicating buttons.

Iterative Design Process

My journey up to the current prototype is shown below, with each version evaluated and improved based on pilot studies with various levels of programmers. Please refer to my slide for more details of each version, including a video demo & evaluation results.

Due to time constraints, I finished designing the current version based on the last iteration's results but could not test it. However, I was able to devise plans for future directions of this project. Snapshots of the current version are shown below:

Planning phase
Coding phase

Implementation & Evaluation

We hope to first make this an independent system, with front end in react and backend in firebase, and potentially integrate into the algorithm labs software later on. We will collect problems from algorithm labs, and collect sample plans through user testing then have them verified by algorithm lab tutors.
After the implementation, high level evaluation plans and measurement metrics are shown below, with studies measuring both qualitative and quantitative aspects of participants' coding behaviors.


This was my first self-driven research experience and I experienced a huge learning curve. Some things I realized about research are: 1. It's slow, 2. Things Don't go as expected, and 3. It's easy to get lost

However, the process of iterations allowed me to develop solid understandings. It seemed like I experienced frequent bumps and failures, but each time I learned something from it. Getting lost taught me to always keep my goal in mind, while maintaining a more flexible mindset.

I also learned some good practices regarding research:
1. If you have an idea, it’s really important to test it. I realized many problems and new improvements I can make after expressing my idea visually and testing it with real users, and also how some of my ideas might work in my head but not practically.
2. Read papers to make sure your ideas are grounded in scientific theories and research, but don’t get too caught up with reading PAPER AFTER PAPERS. I experienced a phase when I was aimlessly reading too many papers and instead ended up really confused.
3.This was my first real taste of HCI and as cliche as it sounds, I was amazed at how diverse and broad this field is. Even at KIXLAB, everyone is working on diverse projects with a wide range of skillset. This really taught me the importance of knowing various fields not only within cs but also in areas like psychology, design, and statistics, because they can all but put into use.

Thank you KIXLAB for a great time :)