Development Timeline

A couple months ago, I, Chris, like many others, watched the code bullet video that highlighted the terminal competition. I was interested in the competition, spent a weekend looking into the game, learning existing strategies, and finally building an algo to compete. It did ok but not great, as one would expect with an algorithm coded over one weekend. During Thanksgiving break, my cousin Steve and I started to discuss the game. Both of us had nothing to do over Christmas break and thought the terminal coding challenge would be a fun and educational experience. Gradually over the coming week, we collaborated remotely and developed a plan to create an algo for the competition, an algo that could win. This is the story of how two programmers, having 16daystocode, created a top 10 algo and how we could have won it all. Hopefully the information here will help other players to improve their programming knowledge and help C1 Games to improve the terminal competition for future seasons.



Imagine if you used this strategy for all your games; making a giant database and pulling from it to adopt the strategy of whichever algo you think has most successfully beaten the algo you are currently playing against. It’d be a bit like that villain from Star Trek that’s always threatening to “incorporate your biological distinctiveness”...


Day 1:
Travelling from out of state, we arrived separately at around 6PM, ate dinner, and set up computers. Using a legal pad and a sharpie, we stated to refine our plan. The following day, December 16th, was the deadline for the 2018 finale competition. This is where we would first test our idea. We made an account, downloaded the starter kit and wrote some python scripts testing out the API. After a couple hours of work we went to bed.

Day 2:
We woke up early on day 2 and started coding. Steve worked on database creation while I worked on building the algo. Our first database would only use the players list for the 2018 finale competition. We used the API to request all of the bot ids participating in the competition and, if the user hadn’t submitted an algo, we got their highest ranked algo using the leaderboard. Once we had a list of bot ids, we used the API to request the recent matches for that algo and got the game id corresponding to the most recent loss. With the list of bot ids and game ids we pulled the extended replay for that game and saved the first turn played by the losing player and the game id of that game to a database.json file. The first revision of our algo was completed in 6 hours of work and was 200 lines long. It read in a database.json file on startup from the local directory. Then it determined its opponent’s layout from the gamestate object on turn 1. After this, it looked through the database using a sequence matcher to find the closest match of the first turn in the database. After it found the best match, it then used the API to request the extended replay file. It parsed the replay file and extracted the sequence of moves played by the winning algo. It then proceeded to play the units for the first and second turns. Every turn after, it played the units corresponding the that turn of the extended replay file. After a test upload of an algo named “resistance is futile”, we did some tweaking and uploaded an algo named “resistance is still futile” and entered that into the 2018 finale.


Of course, just uploading such a “mimic bot” and a database of its own probably won’t help you much. That database will go out of date pretty quickly, especially when people are uploading new bots constantly, or when games are being played after the bot upload deadline. So, you’d want some way to update that database. A script running on your own computer, constantly adding new matches. But if you submit your database when you submit your bot, you won’t be able to update it past the deadline. What you really want is to be able to pull that database from somewhere else. Fortunately, Python is a powerful tool, much like magic in the Harry Potter series. And just like Harry was “Allowed A Wand” at the Triwizard tournament and used it to summon his broom, there’s no rule that says you couldn’t summon a database file from, say, github.


Day 3:
On the third day we worked towards the goal of a remote database. We created another github account called 13daystocode and learned how to use git on our computers. We locally tested whether an algo could pull a file using python’s requests library and found that this worked. We then uploaded algos testing whether this would work on the C1 servers. It did. We could pull a database from an outside source and use it in our bot. This means we could now update the database on github from our computers and our algo would pull that new database from github and use it. At the end of day three we noticed that the first three rounds of the 2018 finale had been run and that our algo surprisingly made it into the top 8. I was about to go to sleep when I realized that maybe we could see the results of the whole tournament using the API. Steve and I got the last game id of the games displayed in the matches section of the competition and started to increment game numbers. After a few games, we ran across a game by two from the top 8. This is what we were after. After 15 more minutes we had written down the full results of the competition. We went to bed eagerly awaiting the next day’s live stream to see if we were proven correct.

Day 4:
As we had predicted, we lost our first game in the top 8 and Forceo won the tournament. We were encouraged by this since we were by far the lowest seeded opponent in the top 8 and had created a bot in about 12 hours that had gotten into the top 8 of a terminal competition. Its funny to look back at the elos of our opponents in the top 8 for this competition. Everyone else was 1800 or higher and we were somewhere around 1350. Because we created a database that targeted our opponents in the competition, our bot was really bad on the general leaderboard. We sketched some ideas about what our next revision of the database and algo would look like. We needed scripts to automatically gather game ids, to turn the game ids into layouts, and to upload the database.

Days 5-7:
We wrote and debugged the various scripts and uploaded another “resistance is futile” algo. It climbed into the top 30 but didn’t get much higher. This algo had a database of all algos’ starting layouts and the associated game ids. Each newly created database took 11 hours to build, parse, and push to github. We also noticed our algo randomly performing poorly on certain matches, but we didn’t know why. This mimic strategy wasn’t nearly as simple as we thought it was going to be. We needed to think a lot harder about how we got feedback from our algos and how we managed our database.


Implementing this strategy won’t be easy. You’ll still make some mistakes that could plague anyone, like mixing up scramblers and EMPs, and find some improvements you could make, like waiting to identify your opponent’s layout until the 2nd frame so that you see the moving game pieces and not just the static ones. Maybe you’ll spend 3 days rewriting your database script so that instead of pulling every game every time you rewrite the database, you only pull the new games and append your file, dropping your database refresh time from 11 hours to 3.5 minutes. You’ll fool around with different ways to decide which algorithm you’re facing when several of them start with the same layout - picking the one closest to your ELO would be a good estimate; another thing you could do is eliminate anybody you know you’ve already played against, since outside of competitions, you only play each algorithm once. Still you’ll lose some games, either because of misidentification or because that first unplayed turn is just too much of a disadvantage. Maybe you’ll realize you need feedback on which of these two factors are the problems, so you’ll again use the power of Python to have your bots email you which algorithm they think they’re playing against every time they play a game.


Day 8:
We faced the problem of getting feedback from our algos head on. We wrote a short function to email us about what strategy our algo was playing, what algo it thought it was playing against, and how confident it was that it was playing that algo. This ability of email allowed us to learn even more information. We emailed our file path on C1’s servers and the names of files in our directory. We began to wonder how much access we had. Our conclusions by the end of the day were the following: Read access to our bot’s folder, yes; write access to our bot’s folder, no; read access to our opponent’s folder, no; read access to the temporary directory used to run matches, yes. Read access to the root directory, yes. When we found out that it was not possible to find the id of our opponent within our temporary folder we didn’t explore any more. There were many interesting files in the root directory such as ‘adminalgokey’ but we did not attempt to access them because we felt it would be against the spirit of the competition. We also developed a plan to make our database more dynamic including considering the elo of our bot since this would dictate the elo of our opponents.

Day 9:
We added elo to our gathered data and used it to create databases for specific elos. We then developed scripts to pull our bots’ elos and to filter a large database into smaller, elo specific, databases to push to our bots. Our bots improved, getting into the top 20. We still had a problem though, our database was taking 11 hours to update for both opponent games and opponent elos. We discussed and planned a pipeline strategy that would first look only at active bots and get losing game ids. The next script would take these game ids and pull the extended replays. It saved multiple factors such as the layout of the opponent, the elo of the opponent, and a ‘score’ measure which was calculated based on multiple factors that are beneficial the the mimic strategy such as the bot we’re mimicking not playing anything on turn 0, health at the end of the game, and health at the end of turn 0. Each of these factors had weights associated with them when calculating the final score for a game. The next script in the pipeline would be solely focused on updating bot elos. This improved our elo update time from 11 hours to 3.5 minutes. The final script would chose a single game for each unique layout within a specified elo range around our bot’s elo such as 100 elo above and 50 elo below.

Day 10:
Christmas Day we decided to take a break from coding.

Day 11-14:
We wrote and debugged the code for the new database pipeline. We also added maintaining defenses to the algo code. This would, after playing the spawns for the turn, check to see if the all units matched the units in the original game. If they did not, the algo would try to spawn in the units to match the original game. The database pipeline along with this strategy was very successful and one of our bots made it up to 5th. Then, on the 14th day, we made a small modification to our pipeline and went to bed.

Day 15:
This small modification inadvertently introduced nulls into the database down the pipeline and, when we woke up the next morning, all of our algos had tanked because they crashed on reading the database. We deleted all of them and started fresh. We completed the database pipeline and prepped for the ‘final’ day of coding.

Day 16:
On the final day, we tried different pipeline settings until we found a set of 6 settings for our 6 algos that we felt would perform best. We implemented a window-within-a-window strategy for some of our algos so that it would preferentially choose a layout near its elo but if there wasn’t one near its elo, it would still have a layout that could be played. We uploaded 3 of our final algos the night of the last day. This is where development would normally end because you can’t upload another zip file after the deadline, but for us, the development continued since we could pull a remote database.

Day 17:
We monitored our algos’ performances and tweaked pipeline values appropriately. We noticed that our algos more or less had a random walk pattern in elo but drifted in one direction, most of the time up. Normally, a top tier algo crushes lower level opponents 9 times out of 10 and moves its way up quickly to the top. Our algos performance though, was more elo independant. Meaning we won about 75% of games, on average, no matter where we were in elo, with the exception of very near the top where strategies are more dynamic. Our performance there stabilized to 50% as expected. During the climbs of our algos, we averaged 50% correct identification of opponent’s bot ids and a 97% win rate against those correctly identified algos. In the other 50% that we did not correctly identify, we had a 50% win rate.

Day 18:
Our algos were solidly in the top 20 at the start of day 18 and we prepared for the move back to our home cities. I would run the database while Steve was travelling and then he would run it while I was travelling. By the end of the 18th day, our top algo was in 12th and we felt we had a good chance to make it into the top 10. We then noticed something odd. Our old test bots began emailing us again. Starting from our first email tests and then working its way up to the server scanning bots. Our first thought was that someone else might have downloaded and run our algos but that was determined not to be true when the file directory was the same as C1’s file directory. This was right around the time when C1 doubled matchmaking frequency so that may have something to do with it. Either way, this was pretty confusing for us since we didn’t know which emails were real and which were fake. The fake emails continued past the end of matchmaking on January 4th.

Day 19:
We traveled back to our respective homes and swapped the database. We were bobbing in and out of the top 10 at this point so we started work on a top 10 database for the final round robin competition. We knew that anyone could submit an algo different than what was displayed on the leaderboard so we gathered a list of all algos for each user and got to work on this difficult problem. Eventually, after many hours of work, we had made the first draft of our top 10 database. The game ids were selected specifically from matches that the winner didn’t play anything on the first turn so as to perfectly mirror any action and not take the game off script. This is also when we noticed that Voice of Cthaeh was also a mimic algo or, at least, a half mimic algo. Many of the games in the top 10 used his games and the others used Track X’s, since Track doesn’t play anything on the first turn either. We also referenced some of our own games where we beat top 10 opponents since these games would be perfect for our strategy. 2 of our bots were in top 10 range while the other 4 were further behind. We uploaded the top 10 database to the 4 bots that were lower, more or less dooming them. However, this would get of valuable data if any of them faced a top 10 opponent.

Day 20: We reviewed the matches from the previous night and swapped out game ids for Free Oracle. We counted down the last hours of the season constantly monitoring our databases for our two bots WE_ARE_THE_BORG_5 and YOURE_ALLOWED_A_WAND. At 4 PM we uploaded our top 10 database to WE_ARE_THE_BORG_5, our top bot at the time. So now all but YOURE_ALLOWED_A WAND had the top 10 database. We did this because matchmaking was still continuing and we wanted a bot to remain live with a database that would be useful outside of the finals matches. Matchmaking continued after 4pm and YOURE_ALLOWED_A_WAND overtook WE_ARE_THE_BORG_5. The round robin matches were played right after match making ended and C1 used the highest bot at the end of matchmaking not at the 4 PM deadline. The database for YOURE_ALLOWED_A_WAND didn’t have any of the top 10 games in it because the bot had already faced all of the top 10 algos. It played incorrect games and lost every match. So in a cruel fate of irony, the only bot we had not uploaded our top 10 database to played in the top 10 round robin competition. Our other bots had confirmed some of our choices for the top 10 database and it’s likely that we would have done very well in the final round robin if not for the unfortunate circumstances.



All in all, Steve and I had an enjoyable and valuable experience creating a Terminal algo. The 20 days were basically a python bootcamp and it taught us a great deal in terms of python syntax, database management, and full stack development. We may participate in future seasons and are excited to see what Terminal grows to become!

Chris and Steve