Users of my iOS Game Teach Me a Lesson MIT Didn’t

In the game of Monorail, it is a more valuable to be skilled at fixing broken solutions than skilled at deducing correct solutions.

I made the iOS puzzle game Monorail.  Before publishing the game, I thought I had figured out the best way to solve the puzzles. Users surprised me with a better strategy, and this experience has changed the way I write computer programs and think about the business of startups.

The object of Monorail puzzles is to complete a closed-circuit loop through all the stations (dots) by drawing rails.  The loop must pass through each station exactly once and close back on itself, like an actual monorail system might in a city.

My Strategy:  Careful-Logical-Deduction

My strategy for solving these puzzles was to reason carefully about every rail before putting it down.  The game was specifically designed to be amenable to this strategy: each puzzle has only one possible solution-path given the starting conditions, which makes it possible to deduce each rail placement until you have solved a puzzle.

For example, you can observe that in a solution path, each station connects to exactly two neighboring stations.  (One rail in, one rail out).  Thus, since a corner station has only two neighbors, you know that it must connect to those two.

I developed layers upon layers of elaborate methods to deduce what rails must be placed on the board.  I solved puzzles by using logical reasoning, placing rails one-by-one until I have completed the path.

Better Strategy: Iterate-and-Repair

When I shipped the game, I used Flurry to track a bunch of statistics about puzzle-completion time because I was worried the puzzles would be too difficult.  After looking at the stats, however, I found that many users were whipping through puzzles I thought I had designed to be super difficult. How? They found a superior strategy.

I found some of the best monorail solvers and observed them.  They would:

  • Start by guessing random paths and just drawing whatever comes to mind.
  • Fix mistakes.
  • Repeat.
  • If things ever look hopelessly messed up, then press the CLEAR button and start over.

I call this the Iterate-and-Repair strategy.

It turns out to be easier and faster to iterate from an existing but wrong solution, than to deduce a correct solution from scratch.  Even if you have to occassionally press the “clear” button to start over.

This realization inspired me to be more headstrong when writing computer programs.  I now put more emphasis on iteration and refactoring from a reasonable starting point, rather than trying to figure out the best possible design ahead of time.  If the code gets totally screwed up, I press the metaphorical clear button and use what I’ve learned to devise a new and better starting point.

I even used Iterate-and-Repair to write this blog post!  “Good writing is rewriting.”

If this whole post is obvious to you, then congratulations, you are already an Iterate-and-Repair kind of problem-solver.  To me, this was kind of a revelation.  I started out as more of a Careful-Logical-Deduction kind of guy.  I think this was influenced by my undergraduate education in mathematics at MIT, where so much emphasis is placed on logical reasoning and certitude.  Indeed, I’ve observed that many of my MIT friends who play monorail tend toward Careful-Logical-Deduction.

Now I’m more of an Iterate-and-Repair kind of guy, and I think I’m a better puzzle-solver, better programmer, and a better startup founder for being so.

36 Comments

  1. rsm
    Posted July 6, 2011 at 3:53 pm | Permalink | Reply

    I wish I was more a Careful-Logical-Deduction kind of guy, I have zero logical deduction skills :( for what is worth I don’t think all problems can be easier to solve using deduction just as not all problems can be easier to solve using trial and error, I believe it’s best to kind of have a mix of both.

    • Posted July 6, 2011 at 4:53 pm | Permalink | Reply

      Yes, for sure you want to use the appropriate problem-solving strategy for the task at hand. I was just blown away that I created an artificial game specifically to be solved with careful logical deduction, then found out that iterate-and-repair STILL worked better.

      Logical deduction is probably still better for situations where mistakes are irreparable. I can’t think of a great application of iterate-and-repair to chess, for example, where you can’t take back moves.

      • Posted March 28, 2013 at 12:39 am | Permalink

        Suppose you could to the iterate and repair in your head for chess but for small subsets of moves to gain favourable position on the main board.

        Also, this reminds me of genetic algorithms where lots of solutions (potentially random incomplete) solutions when combined over generations can solve certain classes of problems.

        Sarwar

    • aoberoi
      Posted July 7, 2011 at 8:52 am | Permalink | Reply

      I actually wish the opposite. It seems like time and time again I see people getting ahead just because they are more comfortable jumping in and failing. This discomfort I have with a lack of tactical strategy seems to hold me back from getting it done quicker. For example, I will read documentation when I need to program with a new language or framework, spending up to 200% of the implementation time just constructing the conceptual models for myself.

  2. Posted July 6, 2011 at 4:34 pm | Permalink | Reply

    I second that first comment, I probably do a lot more guess and mess (even with simple scripts) but your way is much better for things like nuclear reactors etc.

    • Posted July 7, 2011 at 12:42 pm | Permalink | Reply

      I’d say iterative design would be great for those instances as well with our computer simulation abilities. Some type of automated genetic design sequence combined with simulation might work very well.

  3. Posted July 6, 2011 at 4:44 pm | Permalink | Reply

    I would argue that we should be very careful to treat this as a useful tool, not the only tool, and be aware of its limitations. Everyone’s certainly talking about the “fail fast” mentality right now, and I wouldn’t say that it’s always the perfect strategy.

    However, in general I absolutely agree with you, and this is a brilliant case study that illustrates the concept :-) Nice!

  4. Posted July 6, 2011 at 5:06 pm | Permalink | Reply

    MIT’s approach to AI seems to have a similar bent; traditionally I think it has been focused on logical deduction, theorem proving, etc. Would this change your viewpoint on that as well?

    The classic iterate-and-repair algorithm is natural evolution; would you consider an evolutionary approach to AI? I ask because most logical deductive-types seem to consider EAs with contempt, although that is a broad generalization.

    • Posted July 6, 2011 at 7:03 pm | Permalink | Reply

      Despite my initials, I don’t have much experience with AI :-). To answer your question: yes, I would consider an evolutionary approach, especially if you think puzzles are a good metaphor/playground for studying problem-solving and intelligence.

  5. Gus
    Posted July 6, 2011 at 5:47 pm | Permalink | Reply

    I suppose MIT students don’t learn guess and check.

    What a cocky twat you are, sir.

  6. Posted July 6, 2011 at 7:54 pm | Permalink | Reply

    You could change the game play to enforce your original intent. Make it so that games are based on being fast but also on accuracy. Every time a piece is selected that isn’t right they get marked as a mistake. Mind you I haven’t played your game so I may be assuming your existing game play incorrectly. But this way they have to think it through like you originally intended. This of course assumes that you do have only one way to finish it(Which it sounds like you do.)

  7. john
    Posted July 6, 2011 at 8:50 pm | Permalink | Reply

    Goddamn, you are full of yourself.

  8. Posted July 6, 2011 at 9:46 pm | Permalink | Reply

    Great post about iterating and repairing

  9. Posted July 6, 2011 at 9:51 pm | Permalink | Reply

    Awesome app :D

  10. Posted July 6, 2011 at 10:23 pm | Permalink | Reply

    The two methods “Iterate-and-Repair” and “Careful-Logical-Deduction” each have their own merit. For web applications “Iterate-and-Repair” works better as you can launch quickly, get feedback on what’s important and improve that. For certain applications like “Mars Rovers” or “Telecom Switches” it is imperative to have “Careful-Logical-Deduction” to visualize all possible scenarios, as there will be very few (and costly) opportunities to iterate.

  11. gsvolt
    Posted July 7, 2011 at 5:00 am | Permalink | Reply

    I used to play a two person version of monorail. We would make our grid with pencil dots and then use red,blue pens to draw lines. Game play was exciting as you had to essentially block opponents moves. Given I had one sibling, we’d play for hours. I felt like writing to you because of similarity of grid, who knows I can relive an electronic version some day!

  12. Posted July 7, 2011 at 5:24 am | Permalink | Reply

    Hmm. As a novelist I’ve always been what they call a pantser, flying by the seat of my pants and fixing the mess later. Iterate, repair, blow up, whatever. But as a programmer or problem solver, I’ve never done that. I’ve always planned and plotted. I’m going to have to think about that. I’m planning my ninth novel in advance. Why not try “switching sides” on programming or problem solving too? I won’t tackle this tax paperwork by the seat of my pants, but otherwise…

  13. Posted July 7, 2011 at 12:44 pm | Permalink | Reply

    Thanks for this post. At first I thought this was a really obvious insight that I had already internalized long ago, but when applied to actual feature-level coding I realize that I have been erring too far on the side of CLD. I definitely have bouts of spinning in analysis paralysis mode when starting on a large, ambiguous problem with many possible solutions, and I&R is probably the most effective way to at least start building momentum.

    BTW for the logical conclusion of CLD, see Feynman Algorithm[1]:

    1. Write down the problem.
    2. Think real hard.
    3. Write down the solution.

    [1] http://c2.com/cgi/wiki?FeynmanAlgorithm

  14. Posted July 7, 2011 at 10:30 pm | Permalink | Reply

    Thanks for sharing your experience. In this world there are three kinds of people – the logical-mathematical kinds (also called the left-brained), the intuitive-creative kinds (the right-brained), and those who are on the continuum that connects these two kinds. It appears that you were the first kind but now you are exploring the right side. I am an artist and a graphic designer and so I am more intuitive than I am logical – yet for some funny reason, I like to solve puzzles – I do them exactly the way you’ve described that many in your audience do it – but I assure you that there’s no substitute for logic.

    We may be able to solve some puzzles intuitively (and decide not to bother ourselves with the ones that we can’t solve) but we can’t program them using our intuition… so, let us not short-sell logic :)

  15. Posted July 7, 2011 at 11:20 pm | Permalink | Reply

    Downloaded Monorail and played it. I must be thoroughly right-brained to begin clicking the “Clear” button on the second puzzle of 5×5 (easy – it said!) But I completed it :) I like the game and its going to stay in my iPad for a while…but I am confident that I won’t ever complete those 50 puzzles in your lite version.

    As a normal, non-deductive type user, I loved the 4x4s (I must be smarter than I thought, I told myself, I could sail through those puzzles without losing any time,) and I was feeling so proud of myself when I clicked the 5×5 button…the first puzzle was simple, the second took time, the third…well, it was challenging – I’ll come back to it later:)

    So…I asked myself – does the casual gamer really wants to be challenged?

    BTW, I loved the puzzle, the simplicity of the idea, and the reward (the train.) Thank you.

  16. Posted July 23, 2011 at 1:46 pm | Permalink | Reply

    Great blog Aaron. You have made my day. I have been trying to make an argument for this approach to software development to the developers on my team without much success. This blog is going to be a great way for me to put it in much clearer terms. For what it’s worth, I completely agree with the “Iterate and repair” concept and have found it very successful in my own coding endeavors.

  17. Posted October 26, 2011 at 11:01 am | Permalink | Reply

    Can’t your game be thought of as an MDP? I would think that when you have Markov holding, the iterative approach might be effective. Haven’t given it much thought thought, maybe take a look at some thing like TD-Learning (eg. TD-Gammon)

  18. abhinavlal
    Posted March 14, 2012 at 12:47 am | Permalink | Reply

    Reblogged this on Chronicles of a Web NiNja.

  19. Posted June 18, 2012 at 2:40 am | Permalink | Reply

    I am not an expert in any of the fields. But, I guess this is a constraint programming problem – where random solution + iterations are suggested to get optimal solution. Having said that, I thought it should take more time to arrive at the optimal solution. In that way, I am still surprised.

  20. shastra
    Posted March 27, 2013 at 11:18 pm | Permalink | Reply

    Reblogged this on Siliconomics and commented:
    Careful deduction Vs Iterate, Repair strategy.

  21. Posted March 28, 2013 at 12:27 am | Permalink | Reply

    Like Taleb’s argument – “we are quite certain that the Romans, admirable engineers, built aqueducts without mathematics (Roman numerals did not make quantitative analysis very easy)”

    http://statspotting.com/what-are-you-breaking-today/

  22. Posted March 28, 2013 at 1:39 am | Permalink | Reply

    i guess, this is already somewhere in the comments, but i am not the “read all the f***n comments in a thread” kind of guy.

    from my personal view of solving problems, the key skill is to know when to apply which kind of problem solving strategy. like within your puzzels. when you know that at the corners, there is only one possible solution, don’t try experimenting!

  23. Posted March 28, 2013 at 7:04 am | Permalink | Reply

    The trick is not being afraid of the clear button. Product departments do a great job of telling us there is no clear button.

  24. johnicholas
    Posted March 28, 2013 at 7:42 am | Permalink | Reply

    This is actually a standard strategy in programming computers to solve problems too, called rapid restarts: http://ftp.cs.rochester.edu/~kautz/papers/jar.pdf

  25. Posted March 28, 2013 at 8:40 am | Permalink | Reply

    Their way of solving the problem is essentially analogous to stochastic algorithms for optimization problems (see http://en.wikipedia.org/wiki/Simulated_annealing ).

    For good measure, here’s a recent algorithm from MIT that could be used to solve your game, and which works in an essentially similar way: http://web.mit.edu/newsoffice/2013/short-algorithm-long-range-consequences-0301.html

  26. Posted March 28, 2013 at 11:22 am | Permalink | Reply

    Around the time when Sudoku came out I was in and out of hospital routinely, and it proved to be a good way of keeping my brain active. I got to be quite good at it.
    A colleague fo mine also picked up the sudoku bug, and after a while we started challenging each other with how long it would take us to solve the more complex Sudoku puzzles.
    I would almost always win, and usually by a sizeable margin. I’d quickly look for obvious numbers, then step through and scribble down possible numbers. With the complex puzzles that’s usually about as far as you can get without reaching a decision point. From there you’ve got (so far as I’ve ever figured) two options. Either go with it, or logically deduce the permutations of various moves.

    I went with the former every time, trusting my instincts to guide me through. He went for the latter. I wouldn’t even necessarily complete the possible number list, if I came across a square with just two options in what looked to be a strategic position. Occasionally I’d get it wrong and have to erase, but even when I did I’d gained enough of a feel for the particular puzzle that the real answer would now be obvious.

  27. Posted March 28, 2013 at 1:28 pm | Permalink | Reply

    Evolution bitches, it works.

  28. Posted March 28, 2013 at 6:14 pm | Permalink | Reply

    Great article.
    This is why the most ‘productive’ languages have a REPL: Hack, Hack, Hack, Win.
    Though to make this work (well) at a project level you need unit tests. In the case of your monorail puzzle a player can reason about the whole problem easily because they can view the entire problem space at a glance… When dealing with software that spans multiple files and thousands (or tens of thousands) of lines you need to be sure that the change you made in the part of the problem you’re looking at hasn’t broken something in the part you’re not.

  29. Posted March 29, 2013 at 5:32 am | Permalink | Reply

    The relevant question here is “what’s the penalty for a failure”?

    For instance, I would expect strategy to change if in the game you only had so many ‘lives.’

    Here is where I speculate:
    Now, in your own life, I would venture to say that because you were probably one of the ‘smart kids’ in your peer group, that the ‘penalty’ for getting a low grade on an assignment was high- you may have had pressures to perform from parents, expectations from your peers, and yourself, where avoiding embarrassment lead you to develop a more careful, calculating approach to things – whereas (some) of your peers would just do an assignment, hand it in, and try to do better next time, motivated moreso by the “need to complete”. These underlying motivations are reflected in your disjoint strategies in the monorail task.

  30. Posted October 25, 2013 at 1:12 pm | Permalink | Reply

    Even though I’m merely so-so at logical deduction, I have to say it’s the only way I would enjoy a puzzle like this. I can’t really picture having “fun” by generating random noise, even if that noise hones in on a solution.

    Of course, maybe I already play other puzzles that way without considering it. I do enjoy my newspaper’s Jumble word puzzle, and to an extent, anagram solving always requires some randomness. But there’s a logical component as well; only certain combinations of adjacent letters are valid in English, and most words have a predictable vowel-consonant pattern.

    I suppose that Monorail players aren’t guilty of literally choosing nothing but random paths; they would know not to try one that caused a third connection to a point, for example. Extending from this, it seems reasonable to suppose that an Iterate-and-Repair solver, looking to improve her speed, would still benefit from the practice of Careful-Logical-Deduction, because then she has it as second nature, another tool in her belt.

4 Trackbacks

  1. By Iterate and repair | halfblog.net on July 6, 2011 at 9:55 pm

    [...] It turns out to be easier and faster to iterate from an existing but wrong solution, than to deduce a correct solution from scratch.  Even if you have to occassionally press the “clear” button to start over. via Users of my iOS Game Teach Me a Lesson MIT Didn’t – blog.aaroniba.net [...]

  2. By My daily readings 07/07/2011 « Strange Kite on July 7, 2011 at 4:30 am

    [...] Users of my iOS Game Teach Me a Lesson MIT Didn’t « Aaron Iba’s Blog [...]

  3. [...] came across an article by Aaron Iba about the use of logical deductions in puzzles. He came to the realization that the fastest puzzle solvers solved the puzzle by making guesses. [...]

  4. [...] came across this blog post where a game developer was introduced to an optimal strategy compared to the one he originally [...]

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: