Users of my iOS Game Teach Me a Lesson MIT Didn’t
In the game of Monorail, it is a more valuable to be skilled at fixing broken solutions than skilled at deducing correct solutions.
I made the iOS puzzle game Monorail. I figured that I, the author of the game, with my degree in mathematics from MIT, and years of puzzle-solving experience, would be much better at solving my own puzzles than random users downloading my app would be. Users surprised me with a new strategy superior to mine, and this experience has changed the way I write computer programs and think about the business of startups.
The object of Monorail puzzles is to complete a closed-circuit loop through all the stations (dots) by drawing rails. The loop must pass through each station exactly once and close back on itself, like an actual monorail system might in a city.
My Strategy: Careful-Logical-Deduction
My strategy for solving these puzzles was to reason carefully about every rail before putting it down. The game was specifically designed to be amenable to this strategy: each puzzle has only one possible solution-path given the starting conditions, which makes it possible to deduce each rail placement until you have solved a puzzle.
For example, you can observe that in a solution path, each station connects to exactly two neighboring stations. (One rail in, one rail out). Thus, since a corner station has only two neighbors, you know that it must connect to those two.
I developed layers upon layers of elaborate methods to deduce what rails must be placed on the board. I solved puzzles by using logical reasoning, placing rails one-by-one until I have completed the path.
Better Strategy: Iterate-and-Repair
When I shipped the game, I used Flurry to track a bunch of statistics about puzzle-completion time because I was worried the puzzles would be too difficult for average users. After looking at the stats, however, I thought there was a mistake because some users were solving puzzles faster than I was! Were they cheating? Was there a bug in the code? No, they had discovered a superior strategy.
I found some of the best monorail solvers and observed them. They would:
- Start by guessing random paths and just drawing whatever comes to mind.
- Fix mistakes.
- Repeat.
- If things ever look hopelessly messed up, then press the CLEAR button and start over.
I call this the Iterate-and-Repair strategy.
It turns out to be easier and faster to iterate from an existing but wrong solution, than to deduce a correct solution from scratch. Even if you have to occassionally press the “clear” button to start over.
This realization inspired me to be more headstrong when writing computer programs. I now put more emphasis on iteration and refactoring from a reasonable starting point, rather than trying to figure out the best possible design ahead of time. If the code gets totally screwed up, I press the metaphorical clear button and use what I’ve learned to devise a new and better starting point.
I even used Iterate-and-Repair to write this blog post! “Good writing is rewriting.”
If this whole post is obvious to you, then congratulations, you are already an Iterate-and-Repair kind of problem-solver. To me, this was kind of a revelation. I started out as more of a Careful-Logical-Deduction kind of guy. I think this was influenced by my undergraduate education in mathematics at MIT, where so much emphasis is placed on logical reasoning and certitude. Indeed, I’ve observed that many of my MIT friends who play monorail tend toward Careful-Logical-Deduction.
Now I’m more of an Iterate-and-Repair kind of guy, and I think I’m a better puzzle-solver, better programmer, and a better startup founder for being so.
I wish I was more a Careful-Logical-Deduction kind of guy, I have zero logical deduction skills :( for what is worth I don’t think all problems can be easier to solve using deduction just as not all problems can be easier to solve using trial and error, I believe it’s best to kind of have a mix of both.
rsm
July 6, 2011 at 3:53 pm
Yes, for sure you want to use the appropriate problem-solving strategy for the task at hand. I was just blown away that I created an artificial game specifically to be solved with careful logical deduction, then found out that iterate-and-repair STILL worked better.
Logical deduction is probably still better for situations where mistakes are irreparable. I can’t think of a great application of iterate-and-repair to chess, for example, where you can’t take back moves.
Aaron Iba
July 6, 2011 at 4:53 pm
I actually wish the opposite. It seems like time and time again I see people getting ahead just because they are more comfortable jumping in and failing. This discomfort I have with a lack of tactical strategy seems to hold me back from getting it done quicker. For example, I will read documentation when I need to program with a new language or framework, spending up to 200% of the implementation time just constructing the conceptual models for myself.
aoberoi
July 7, 2011 at 8:52 am
I second that first comment, I probably do a lot more guess and mess (even with simple scripts) but your way is much better for things like nuclear reactors etc.
Scott
July 6, 2011 at 4:34 pm
I’d say iterative design would be great for those instances as well with our computer simulation abilities. Some type of automated genetic design sequence combined with simulation might work very well.
bmc1313
July 7, 2011 at 12:42 pm
I would argue that we should be very careful to treat this as a useful tool, not the only tool, and be aware of its limitations. Everyone’s certainly talking about the “fail fast” mentality right now, and I wouldn’t say that it’s always the perfect strategy.
However, in general I absolutely agree with you, and this is a brilliant case study that illustrates the concept :-) Nice!
joethephish
July 6, 2011 at 4:44 pm
MIT’s approach to AI seems to have a similar bent; traditionally I think it has been focused on logical deduction, theorem proving, etc. Would this change your viewpoint on that as well?
The classic iterate-and-repair algorithm is natural evolution; would you consider an evolutionary approach to AI? I ask because most logical deductive-types seem to consider EAs with contempt, although that is a broad generalization.
Joel Lehman
July 6, 2011 at 5:06 pm
Despite my initials, I don’t have much experience with AI :-). To answer your question: yes, I would consider an evolutionary approach, especially if you think puzzles are a good metaphor/playground for studying problem-solving and intelligence.
Aaron Iba
July 6, 2011 at 7:03 pm
I suppose MIT students don’t learn guess and check.
What a cocky twat you are, sir.
Gus
July 6, 2011 at 5:47 pm
You could change the game play to enforce your original intent. Make it so that games are based on being fast but also on accuracy. Every time a piece is selected that isn’t right they get marked as a mistake. Mind you I haven’t played your game so I may be assuming your existing game play incorrectly. But this way they have to think it through like you originally intended. This of course assumes that you do have only one way to finish it(Which it sounds like you do.)
Francois Laberge
July 6, 2011 at 7:54 pm
Goddamn, you are full of yourself.
john
July 6, 2011 at 8:50 pm
Great post about iterating and repairing
Andrew Mager
July 6, 2011 at 9:46 pm
Awesome app :D
Alina Libova
July 6, 2011 at 9:51 pm
[...] It turns out to be easier and faster to iterate from an existing but wrong solution, than to deduce a correct solution from scratch. Even if you have to occassionally press the “clear” button to start over. via Users of my iOS Game Teach Me a Lesson MIT Didn’t – blog.aaroniba.net [...]
Iterate and repair | halfblog.net
July 6, 2011 at 9:55 pm
The two methods “Iterate-and-Repair” and “Careful-Logical-Deduction” each have their own merit. For web applications “Iterate-and-Repair” works better as you can launch quickly, get feedback on what’s important and improve that. For certain applications like “Mars Rovers” or “Telecom Switches” it is imperative to have “Careful-Logical-Deduction” to visualize all possible scenarios, as there will be very few (and costly) opportunities to iterate.
Anay Tamhankar
July 6, 2011 at 10:23 pm
[...] Users of my iOS Game Teach Me a Lesson MIT Didn’t « Aaron Iba’s Blog [...]
My daily readings 07/07/2011 « Strange Kite
July 7, 2011 at 4:30 am
I used to play a two person version of monorail. We would make our grid with pencil dots and then use red,blue pens to draw lines. Game play was exciting as you had to essentially block opponents moves. Given I had one sibling, we’d play for hours. I felt like writing to you because of similarity of grid, who knows I can relive an electronic version some day!
gsvolt
July 7, 2011 at 5:00 am
Hmm. As a novelist I’ve always been what they call a pantser, flying by the seat of my pants and fixing the mess later. Iterate, repair, blow up, whatever. But as a programmer or problem solver, I’ve never done that. I’ve always planned and plotted. I’m going to have to think about that. I’m planning my ninth novel in advance. Why not try “switching sides” on programming or problem solving too? I won’t tackle this tax paperwork by the seat of my pants, but otherwise…
MichaelEdits
July 7, 2011 at 5:24 am
Thanks for this post. At first I thought this was a really obvious insight that I had already internalized long ago, but when applied to actual feature-level coding I realize that I have been erring too far on the side of CLD. I definitely have bouts of spinning in analysis paralysis mode when starting on a large, ambiguous problem with many possible solutions, and I&R is probably the most effective way to at least start building momentum.
BTW for the logical conclusion of CLD, see Feynman Algorithm[1]:
1. Write down the problem.
2. Think real hard.
3. Write down the solution.
[1] http://c2.com/cgi/wiki?FeynmanAlgorithm
Joe Cheng
July 7, 2011 at 12:44 pm
Thanks for sharing your experience. In this world there are three kinds of people – the logical-mathematical kinds (also called the left-brained), the intuitive-creative kinds (the right-brained), and those who are on the continuum that connects these two kinds. It appears that you were the first kind but now you are exploring the right side. I am an artist and a graphic designer and so I am more intuitive than I am logical – yet for some funny reason, I like to solve puzzles – I do them exactly the way you’ve described that many in your audience do it – but I assure you that there’s no substitute for logic.
We may be able to solve some puzzles intuitively (and decide not to bother ourselves with the ones that we can’t solve) but we can’t program them using our intuition… so, let us not short-sell logic :)
Nifty
July 7, 2011 at 10:30 pm
Downloaded Monorail and played it. I must be thoroughly right-brained to begin clicking the “Clear” button on the second puzzle of 5×5 (easy – it said!) But I completed it :) I like the game and its going to stay in my iPad for a while…but I am confident that I won’t ever complete those 50 puzzles in your lite version.
As a normal, non-deductive type user, I loved the 4x4s (I must be smarter than I thought, I told myself, I could sail through those puzzles without losing any time,) and I was feeling so proud of myself when I clicked the 5×5 button…the first puzzle was simple, the second took time, the third…well, it was challenging – I’ll come back to it later:)
So…I asked myself – does the casual gamer really wants to be challenged?
BTW, I loved the puzzle, the simplicity of the idea, and the reward (the train.) Thank you.
Nifty
July 7, 2011 at 11:20 pm
Great blog Aaron. You have made my day. I have been trying to make an argument for this approach to software development to the developers on my team without much success. This blog is going to be a great way for me to put it in much clearer terms. For what it’s worth, I completely agree with the “Iterate and repair” concept and have found it very successful in my own coding endeavors.
dwyerz
July 23, 2011 at 1:46 pm
Can’t your game be thought of as an MDP? I would think that when you have Markov holding, the iterative approach might be effective. Haven’t given it much thought thought, maybe take a look at some thing like TD-Learning (eg. TD-Gammon)
Matt Gershoff (@mgershoff)
October 26, 2011 at 11:01 am