Problem

Is concrete instructions beneficial for learning abstract things?

In a class of Educational Technology, our professor Erik Anderson introduced us the concept of Concreteness Fading and showed us a game -- Dragon Box. In Dragon Box, user starts learning algebra from pairing cartoon pictures on two sides of the screen. Gradually, the pictures will fade into variables and numbers, and the whole screen will fade into algebra equations. The game was reported to be engaging , but the effectiveness of learning is not so optimistic.

We wonder if this result applies to other subjects and different contexts. Chemistry is one field we can think of.

Ideation

Combine kitchen game with learning balancing chemical equations.

In the game, food ingredients stand for atoms in a chemical equation and a bundle of ingredients becomes a molecule. The users' goal is to use provided bundles of food to make new bundles, which is, in the chemistry world, assign coefficients to reactions and products.

Three Significant Problems in the First Iteration

How to let chemistry make sense in a cooking game?

We gave a lot of consideration into the user interaction. First of all, we looked into several kitchen games, and find out its layout normally has three parts: Food area, cooking area and recipe or customer area. I used this for reference and created our first interface.

We had a critique session inside our group and pointed out some flaws of this design:

1. User can only have limited number of empty plates since the space of the work table is limited
2. Put the products on the top of the screen and reactants at the bottom does not look like an equation.
3. When people are balancing chemical equations, they change the coefficient on both sides, not only the reactants side, but a cooking game usually only allows users to operate on the material food side.
   
4. If more than one products have a same element, for example, an "O", we cannot decide which plates should the "O" go to.
   

To solve these problems, we made the following changes on user interaction:

1. Instead of having empty plates, when users click on a provided bundle, the ingredients in it becomes "free food" on the left side of the table.
2. When users click on the target bundle, it creates empty plates on the right side that “absorbs” the applicable free food automatically.
3. Put the provided bundles on the left side and the target bundles on the right side, so it looks like an equation.

How to do concreteness fading?

The final goal of this game is to teach chemistry, thus, all the food metaphors needed to be faded into real chemistry in the end. But to make the process less obvious and painful, we designed the fading route to be: all food -> partial atoms -> all atoms -> all text.

User Testing

The current gameplay facilitates understanding but not learning of balancing chemical equations.

We tested the demo on some of our friends, and we observed the following behaviors:

1. They think the worktable helps them to understand what happens after seeing the atoms level.
2. They played the game by keeping adding 1 to a bundle each time based on their observation of the ingredients in the bundles. As a result, they did not think mathematically during the process.

We considered the observation to be a helpful behavior because it is required when learning balancing chemical equations. Also, it helps beginners understand atoms and molecule. So it is appropriate to keep the gameplay for beginning levels. However, to teach users to hit the right coefficients directly, new gameplay need to be introduced.

Three significant problems in the second iteration

Design the progression of the gameplay to help users think mathematically and hit the correct coefficient at one time.

The major problem of the former gameplay is that it allows users to try out by adding 1 to the coefficient. We thought of eliminating the clicking interaction by having users to type in to coefficient directly. Thus, we designed typing levels as the second phase after the clicking levels, it works like this:

However, users can still try out because all the process are visualized on the worktable. To help users move another step forward, we eliminated the worktable in the third phase, we named it “checking levels”. In checking levels, users need to finish all the coefficients of reactants and products and then click on the “check” button to watch the animation and see if they did it right.

Finally, to make the game more like a paper test at the very end, we eliminate the animation after checking and only tell them if they get it right or not after checking and this will only applies to text levels.

Arrange the order of the levels with the concept of reinforcement

Now, the game became complicated because it has two progression going on, which means during the game, we need to introduce new user interactions, new methods of balancing chemical equations as well as concreteness fading.

Because  of so many new elements to introduce, we decided to reinforce new concepts regularly as users move forward in the game. This is how we arranged the order of the levels.

Add a story line to bridge chemistry and cooking

As we brought new concepts to teach chemistry, the conflicts between a cooking game and a educational chemistry games shows up again, the game does not look fun and even not make sense to some extent. We add a story line to let the game make more sense.

Experiment Design

How engaging is our game? Do users really learn from it?

We collect data from the following two aspects:

1. Pretest and posttest logging - We put 6 equations in the pretest and the posttest：
   For those who have finished both pre  and post test, was their performance improved after playing the game?
2. Level logging:
   - How many levels did they stayed in the game? - The engagement
   - How much time they spent in each level? - The rationality of the order of the level

We put the game on reddit, and got over 1000 players in two days.  But since we are not allowed to share the data collected from the experiment outside of the class publicly. Please email me if you feel like seeing the detailed data.

Conclusion

The feedback we got is positive, but a formal user test needed to see if this game really helps users learn.

The feedback we got from reddit are very encouraging. People helped us debugging the game, chemistry teachers showed interest to use this as a instructional material, some users said they could have done better in chemistry if they had this game while they learning balancing chemical equations.

However, since we never had the chance to test it on real middle schoolers, it is too early to say that this game helps users to learn.