LuigI/O FAQ

LuigI/O FAQ
We get lots of questions about LuigI/O, what it is, how the AI works, and what different things are, so here is a compilation of the most frequently asked questions. I'd suggest you read this if you have anything you want to know, or before trying to make a suggestion about something you don't fully understand, as it most likely has the answers you are looking for. If you're left still curious about things, feel free to ask us on the stream or on discord.

** Questions about the stream in general **

Q: What is this? / What's the point of this?
A: It's a self-learning AI (Artificial Intelligence) that is emulating evolution to get through Super Mario Bros 3 one level at a time. The point is to see just how far this AI can make it in the game, because self learning AIs are cool. The stream also has a huge playlist of music you can listen to, and we always will welcome you to chat with other viewers in the live chat (so long as you follow the rules)

Q: Who made this?
A: LuigI/O is streamed by Akisame and developed by Akisame and Electra.

Q: When did the stream begin?
A: LuigI/O started on May 18, 2018, playing Super Mario Bros. On Dec 5, 2018 we began Hard Mode of Super Mario Bros. On Mar 8, 2019, we began the Lost Levels. On Apr 11, 2020, we began Super Mario Bros 3.

Q: Didn't Sethbling make this?
A: Sethbling created MarI/O for SMW and later made a modification to make it work for SMB. LuigI/O is a far, far improved version of that, small bits of MarI/O still remain but the code is almost completely new and rewritten.

Q: How/why did LuigI/O start?
A: MarI/O was streamed by Einfach Nerdig, with a slightly modified version of MarI/O by Akisame. It got stuck multiple times, and was eventually terminated due to Einfach moving and and not having enough internet to stream. Near the end of MarI/O, Akisame, one of the mods, created his own stream with an improved AI to see if it could get past the parts where MarI/O got stuck.

Q: What will happen when we beat Super Mario Bros 3?
A: We don't know exactly what we'll do after we beat SMB3 just yet. Some ideas we are considering is to play Super Mario World or SMB/SMB3 rom hacks. We will start thinking about it once LuigI/O gets far enough into the game.

Q: How many levels have we beaten? Can I see the stream VODs?
A: We have beaten all levels of Super Mario Bros, almost all levels of Super Mario Bros: Lost Levels up to 8-4, and now we are beating SMB3 levels. On the LuigI/O channel are videos of every single one of his previous level clears. We do not save recordings of previous full streams though as Youtube only will let you save the most recent 13 hours. You can see all the beaten levels here: https://www.youtube.com/watch?v=CowWsz9ZJjk&list=PL-Wqn40FTp6x3o2_SktOTi_TIr1sYr4pZ

Q: I really love watching LuigI/O's breakthroughs and clears live. Can I get notified when he makes progress?
A: If you join our discord there is a channel that will say when he makes a breakthrough. You can give yourself a role to have this automatically @mention you when he beats his previous best, or just when he beats a level. Please note that this notification is given out with a slight delay so as to not spoil it for those who are actually watching the stream live.

Q: Where did you get your chatbot?
A: I wrote it myself. I was unsatisfied with the chatbot solutions available online so I decided to write my own from scratch in Python. I didn't use any other chatbot codes for inspiration so the features may be different than what you are used to from other streams.

Q: I have a suggestion for something to change! How can I reach you?
A: To suggest things, use the !suggestion command on stream, post in one of the suggestions channels on our discord, or just ask a mod. However we highly recommend reading through this FAQ first before making any suggestions as there may be features you don't understand yet.

Q: I really like the stream! Can I donate to you?
A: We have set up a Patreon and a streamlabs. If you really enjoy this stream you can donate here: https://www.patreon.com/Akisame or here: https://streamlabs.com/luigio We appreciate donations, as it helps us rent the server that LuigI/O runs on. However, please make sure only donate to us if you know you can miss the money.

** Questions about things on the stream **

Q: What is fitness?
A: Fitness is a numerical measure of how well the AI did, based on how far it got in the level.

Q: What is the Gen / Species / Genome?
A: Gen is the current generation, Species is the current species, and Genome is the current attempt # of that species. More info about these in the AI questions section.

Q: What is the 1st and 2nd below Gen, Species and Genome?
A: 1st an 2nd shows the best and second best attempts done so far.

Q: What is "Pop" and the numbers next to that?
A: Pop is population. The top number represents how many genomes/attempts it has taken so far in the current generation while the bottom number shows the total number of genomes in that current generation.

Q: What is the "#MAX" on the bottom left?
A: That is the number of times that the AI has gotten close to its best fitness this generation.

Q: What time zone is the clock in?
A: It is the time zone where LuigI/O is located, UTC+01:00

Q: What does the yellow "T=" on the top right mean?
A: Occasionally we will turn turbo on to speed up the AI, when it is stuck. The number in front of T= is how far in the level (in pixels) it will go before it returns to normal speed. Sometimes it is set to a range when there is a specific part of the level that is slow.

Q: What is the "Best idea" on the top?
A: About every hour a message from chat will be put up on the top of the screen. When a ! appears in a random location on the stream, the first person to type "!idea" will get to place their idea, they will then have 5 minutes to put "!idea [their message]" to change the idea to what they want. The timer of the idea can range between 50 and 70 minutes so don't expect it to always take the same amount of time

Q: What song is playing next? How is it chosen?
A: The current and next song are displayed in the AkiAmp box. They are chosen randomly from our playlist https://pastebin.com/W4jpYCkZ Songs can also be queued by viewers.

Q: I have a song that I think would fit well on the stream. Can you add it to the playlist?
A: If you have song requests you can use the "!request [YTID]" command to request them. The YTID is the part of the youtube URL that comes after "?v=". Please be patient with your requests as we must review the requests for copyright before they are added to the stream. Also, please don't request meme songs, extended versions of songs, copyrighted music (including music from recently released games), or music that is very off of the stream music's theme and feel. If these rules are broken the request will be denied, if it is clear that you are breaking these rules on purpose you may be banned from requesting music, or banned altogether.

Q: How do I get a song to play next in the queue?
A: You can use "!playnext [name of song]" to get a song from the playlist to play next. To ensure that the same songs aren't played repeatedly and the music is generally random, you have a limited number of uses of this command, called charges. You get a maximum of 3 charges to use daily.

Q: What are the red and blue bars on the minimap?
A: The red bar is the point that the best attempt at this level has reached, and the blue bar is the point that the best attempt from the currently running species has reached.

Q: Where did you find the minimaps?
A: We got the maps we use from https://nesmaps.com/maps/SuperMarioBrothers3/SuperMarioBrothers3.html

Q: What is the "Gen Spec Gnm GSID Fit Time" table?
A: That displays the last 5 improvements that the AI has made. Gen stands for generation, Spec stands for species, Gnm stands for genome, GSID stands for global species ID, Fit stands for fitness, and Time is the time the breakthrough happened relative to the start of the level.

Q: What is global species ID / GSID?
A: This is a value for each species that is unchanging. 2 breakthroughs on the table with the same GSID were made by the same species, and if the currently running species has the same GSID as one in the table then they are the same species. This can not be said for the number under "Spec"

Q: How long has it been on this level? Is that timer for the level or the whole game?
A: The timer and try counter under the game screen shows how long we have been on this level and how many times we have attempted it.

Q: What is the information at the bottom of the screen - GSID, SMax, Stale, Nick?
A: That is information about the current species. GSID is the global species ID, SMax is the fitness of that species' best attempt, Stale is the number of genomes since that species made an improvement, and Nick is the name of the species given to it by the chat.

Q: How do I give a species a nickname?
A: Only users with special permission are given the permission to nickname species, so that we don't get people spamming our species nicknames. Be around for a while and eventually we may give you this permission. Once you get it you can use "!nick [species] [name]" to give a species a nickname. Add a g before "nick" to do this based on GSID rather than normal species #, and add an r before "nick" to change a nickname rather than to add a new one.

Q: Where do you get your background?
A: The background is from http://www.misucell.com/WDF-7037.html and slightly modified to make Luigi into an evil cyborg.

** Questions about how the AI works **

Q: What is a genome?
A: A genome is a single attempt at the level, which uses one or more neural networks to try to beat it.

Q: What is a species?
A: A species is a collection of genomes that have been automatically grouped to be similar - generally genomes with very similar strategies will appear in the same species.

Q: What is a generation?
A: Once all species have run through all their genomes, then new genomes will be bred based on the old genomes that did well. This is called a generation. The number of attempts in a generation is it's population.

Q: How is fitness calculated?
A: Fitness is the number of pixels forward in the level that Luigi has moved - this counts both going up and going to the right. In levels with mazes or other weird paths the fitness function will be modified to increase along that path. If Luigi doesn't increase his position for a certain period of time, then his fitness will decrease once he starts moving again, the longer the pause the more he will lose.

Q: Does LuigI/O gain any fitness for powerups?
A: He does not. However, if he happens to find a mushroom or fire flower, it will let him damage boost and often get further in the level than he could without it, which can encourage him to get these powerups. Generally he ignores them as it is too complex for him to have to wait and hit a specific block and grab the powerup, but in a few levels he has taken advantage of them.

Q: The AI is getting worse - it was just getting to the end and now it's not doing anything!
A: Since genomes are grouped by their tactics into species, better genomes will generally end up together. Some species are much better than other species, if you wait a generation for the same species to come around again it will most likely do as well if not better than as you saw it do last generation.

Q: Why does the AI just stop sometimes / why is it so stupid?
A: The networks are evolved randomly and are very chaotic - bad changes can lead to it doing stupid things or not doing anything. The ones that do this will not perform as well and will be bred less, so over time it will improve. Evolution takes a lot of time for it to get the proper changes, and often times genomes will get worse with higher rates than they get better. Over time, however, natural selection will always win out.

Q: Why does the AI reset even though it didn't die?
A: If it has not moved forward for a while it will be killed, this is to prevent us for having to wait for the in-game 5 minute timeout as this will take a while. When it is close to the end of the level it will take longer before it is automatically killed as networks of that complexity will sometimes stop for a while yet still decide to do something later after something has changed.

Q: If the AI was left for long enough would it speedrun?
A: Once LuigI/O has found a strategy that works it generally sticks to it, if you left it for a long time it would get more consistent at it's strategy but it wouldn't change it up to try to win any faster.

Q: It skips genomes sometimes, why is that?
A: Genome #1 of each species is always an exact copy of the best attempt that species has ever done. This attempt is not shown in later generations as we have already seen it.

Q: Is the AI completely reset between levels?
A: No, it will start with 1 genome that beat each of the previous levels in the game. Sometimes these will help it get a headstart on this level, sometimes they won't.

Q: Could the AI be programmed to play all levels with the same network?
A: No, there is too much variety between levels for it to do it all with one brain. Some situations look the exact same to the AI but require different solutions.

Q: Could the AI be programmed to play other games?
A: With some modifications, it could be changed to play similar games, primarily by changing RAM addresses. 3D games are not anywhere near the realm of possibility at this point.

Q: What is the deed?
A: Tell LuigI/O "I have done the deed my lord" and then he will respond with a word. Make sure your next response contains the matching word to do the deed, this will allow you to use the suggestion command. Here are the matching words: Salad / Rubber Duck. Gun / Butter. Heavens / Toilet. Bread / Books. Music / Balloon. Air / Chicken. Video / Rats. Elder god cthulu / Wings. Monkeys / Timemachine. Bomb / Glitter. Rope / Bizon. 1000000 volt powersupply / prisoner. Please don't share this secret with anyone, it is to make sure that people read the FAQ before suggesting stuff, and is hidden in the middle of the FAQ for a reason.

Q: What is the jumbled mess of lines and boxes overlayed on the game screen?
A: That is a visual representation of the AI's brain.

Q: What does the AI see?
A: On the view of the AI's brain, the large 13x13 box is what the AI sees - it sees all blocks (even coins) as white, and all object hitboxes as black.

Q: Why is the AI's vision so simplified?
A: It is true that the AI's sight is simplified. This is because though giving it a better vision of it's surroundings might increase it's maximum potential, it would also take a lot longer to be able to comprehend it's surroundings at all, so in the end would end up taking longer by confusing it with a lot of information at once.

Q: Does the AI get any other information?
A: The nodes below the AI's vision are other information it gets. The first two are scaled based on Luigi's X and Y velocity, the next 3 are timers that switch between white and black (the frequency can evolve) and the last one is always white to allow constant terms to be added to the neural network's calculations. These exist to give the AI a sense of movement and time, as it only ever sees based on 1 frame of the game at a time.

Q: Can the AI press left and right / up and down at once?
A: No, it cannot. If it attempts to press both buttons at once you will see the squares next to the button list turn red, this indicates that it is trying to press those buttons but is blocked from doing so.

Q: How does the neural network do it's math?
A: A fully white square indicates a 1, and a fully black square indicates a -1. The AI uses a feed forward neural network to figure out what buttons to press at the end. Each square in the middle of the network diagram takes a weighted sum of it's inputs (green inputs are positive multipliers, while red ones are negative multipliers) and then uses a sigmoid function (2/(1+e^(-4.9*x))-1) to normalize the result between 0 and 1. The layers of neurons are arranged in the order they are calculated.

Q: How is the network's shape decided?
A: This algorithm uses NEAT, or neuro-evolution of augmented topologies. This means the networks start very simple and will get more complex as time goes on. Mutations can add extra connections between neurons, or split a connection by putting a neuron in the middle, as well as enable and disable current connections. If a loop or dead end structure was to be randomly created it is not shown on the screen, only parts of the network that are able to be evaluated are rendered.

Q: The network sometimes changes mid-run, why is that?
A: Some levels are too long or varied to be able to be completed using a single neural network. If it detects that it has been stuck for a long time, it will create a new network that takes over during a later part of the level. This allows us to be able to beat much more complex levels, and means that if the AI gets stuck doing a suboptimal strategy it will be able to evolve out of it's rut by changing it's strategy altogether.

Q: What determines the population?
A: The population starts at 1000 and quickly drops to 300. When the AI has not made progress for a while it slowly rises to 500, drops back down to 300, goes up to 600, etc. This oscillating nature is meant to allow it to produce more varied genomes to give a better chance of a necessary mutation, only to then cut down on the genomes that weren't necessary by reducing the population back down. Some other statistics such as the percentage of a species that is unfit for breeding each generation is based on a similar oscillating pattern.

Q: How are new genomes bred?
A: After a generation, each species will breed genomes from their top ~50% proportional to how well that species did. This is based on it's average rank (1 = worst genome, #genomes = best genome) compared to the entire pool, and can get a multiplier based on that species' max fitness. In addition to these genomes, a few genomes are generated crossing two different species together, and any remaining genomes necessary to fill the population are created from a random species.

Q: How are species defined / created?
A: Each genome is added to a species by comparing it to the top genomes of all currently existing species. If it's networks are similar enough in structure, then those genomes are said to be similar. The genome will be added to the species it is similar to, if there are multiples it will be added to a random one, if there are none a new species will be created. Usually, genomes bred from a species stay in that same species, but occasionally one species can feed into another as their strategies change.

Q: How are species killed?
A: A species is killed when it doesn't produce enough genomes in a generation. It is also killed if it has gone 25 generations without improving once. The top 2 species are immortal until another species is able to surpass them, to prevent permanent loss of progress.

Q: Why can't the AI see what is needed to progress?
A: All this AI knows is to make random modifications to a big group of lines and boxes, and is then given a number that is the score of how well that network did. Calculating how good a network is is extremely hard. Look at the current network shown on screen, can you tell me a specific addition that would make it improve, without screwing up any of the already existing calculations? The best way to do this is often just by trying things at random until they work, just how complex life was able to evolve from simple natural rules on Earth.

Q: Why can't we tell the AI what is better?
A: We could tell the AI to always press Right + B, and to primarily use the jump button. However, the goal of LuigI/O is to be completely self-learning, all we give it is a function to calculate how far it got in the level, and it does the rest. Any further input than that would be kind of cheating.

Q: Can I see graphs of the AI's progression, or detailed statistics?
A: On our discord is a channel that will print the stats of each gen, the stats of each species with a nickname, and a graph of the AI's progress on the current level.

Q: Can I run the AI for myself?
A: Yes, you can. Install FCEUX and then download this code:
SMB2J: https://cdn.discordapp.com/attachments/459694956886622219/549282621427810320/LuigIO-v4.2.zip
SMB3: https://bit.ly/34FNXix
This link may not be updated all the time, the most recent available version is always on Discord. Also, always feel free to ask for help in setting up the AI on your computer.