DS Emulation vs Console Timing Issues

DS Emulation vs Console Timing Issues

Report by Vulajin (http://twitch.tv/vulajin)

Major contributions by (alphabetical order):
Abitalive (http://twitch.tv/bitalive)
Dabomstew (http://twitch.tv/dabomstew)
Sinstar20 (http://twitch.tv/sinstar20)

For all references to "emulator" in this document, you can substitute DeSmuME, which was specifically investigated. Versions 0.9.8 and 0.9.9 are both confirmed to be affected.



Abstract

Investigation has determined that DeSmuME has significant speed advantages over DS consoles running the same games. This applies to all models of DS, including DS Phat, DS Lite, DSi, 3DS, and 3DS XL. DeSmuME appears to be much faster when it comes to loading data "from the cart," meaning that area, screen, or menu transitions are substantially faster than on consoles. For several games, the in-game timers used for speedrun timing are also affected, as it appears that the game synchronizes the in-game clock to real time regardless of load times.



Background

Several days ago, a runner named Abitalive contacted me on SRL IRC to talk about speedrunning Pokémon Black. As part of our discussions, he shared the times he attained on the first two gyms during an emulator practice run: 22:35 to Cress, and 43:56 to Lenora. For comparison, my best splits on console are 23:26 and 44:49, respectively. Given his indication that he was still learning the route, he and I were both skeptical that he would be surpassing my best splits by this much time already.

Using several fixed points in the run, I compared his timing to mine. He completed the introduction sequence (which is a fixed length, as long as you button mash quickly enough) at 0:44.xx, where my console completes it at 0:47.xx. He saw the "I received Tepig!" message for choosing your starter at 1:43.xx, where I receive it on my console at 1:49.xx. This seemed to indicate a speed advantage for the emulator.

Further, Pokémon Black displays the DS system clock in-game during battles (and, once you acquire the C-Gear, outside of battles as well). The time displayed by the DS clock in Abitalive's emulator advanced at the same rate as the real time on his WSplit. This raised fears for both Abitalive and me that the in-game clock may be affected by the emulator speedup.



Testing

First I wanted to verify that the speed difference was not simply caused by me being bad. I accomplished this by simply playing the early portions of the game in the emulator. I completed the introduction sequence at 0:44.xx, the same as in Abitalive's run. I saw "I received Tepig!" at 1:43.xx, the same as in Abitalive's run. I received control after defeating Bianca and Cheren at 3:53.xx, just over 7 seconds faster than I normally receive control. This indicates that the speedup was not simply caused by Abitalive mashing more effectively than me.

I did some additional tests to determine whether the in-game timer was affected by the emulator speedup. Upon creating a new game, the game plays an introduction in which you choose your name and gender, then creates a save file. After you appear in your house, the in-game timer begins counting. In many older consoles, these timers are based on frames advanced, and therefore disregard lag, load times, and emulation problems. For such a timer, if the emulator runs faster than console, the in-game clock should advance commensurately faster.

After Cress, I compared my real time to my game time. The real time was 24:00.xx, but the game time was 0:22 (seconds are tracked, but they do not display in the game). I continued to Nacrene City and checked again. The real time was 36:22.xx, but the game time was 0:35. This indicates that the in-game timer also benefited from the emulator speedup.



Further Analysis

So far our testing confirmed a difference between emulator and 3DS specifically for Pokémon Black. Next we wanted to determine whether the effect did actually extend to other games.

Sinstar performed a substantial amount of testing to compare emulator, 3DS XL, and DS Phat. First he confirmed emulator speedup on Pokémon White 2. Then, he confirmed that DS Phat is also faster than 3DS, but still slower than emulator. Performing further tests, he isolated the source of the speedup to area transitions. This explains why the in-game timer does not advance faster commensurate with the emulator speedup. Finally, Sinstar confirmed emulator speedup in Pokémon Heart Gold as well.

I used Final Fantasy 4 DS as my test game for "games that are not Pokémon." I confirmed emulator speedup during the opening sequence. I then performed a timing test of walking through a door 20 times. On emulator, I found that this took about 47 seconds; on both DSi and 3DS, I found that it took about 53.5 seconds.

Dabomstew and Abitalive assisted with further testing of console differences using Pokémon Black. Each of us measured the time to perform a standardized set of steps on emulator and on each DS console we owned. We then used these times to draw conclusions about the relative speed of various DS models.

Using the data we gathered and the data provided by Sinstar, we drew the following conclusions:

- Emulator runs substantially faster than all DS and 3DS consoles.
- DS Phat runs substantially faster than other DS and 3DS models, but still slower than emulator.
- DS Lite runs somewhat faster than DSi and 3DS models, but cannot conclusively be compared to DS Phat yet.
- DSi and 3DS models run slower than all other DS models.



What's Next?

There are still some unanswered questions. The main ones are:

- Do all games experience this load time speedup on emulator? Many more diverse games need to be tested to determine how widespread the issue is. However, it clearly affects DS Pokémon games.

- How do DS Phat and DS Lite compare? Someone who has both of these consoles needs to test them. I, unfortunately, do not have a DS Phat or DS Lite.

- Is the difference between DS Lite and DSi consistent?

- Is DSi consistently the same speed as 3DS?

- Are there differences from individual console to console?

- Are there differences between emulator versions?

- Does the emulator time depends on HDD/SSD speed, RAM, CPU, anything else?



Testing Methodology and Data

For my initial information gathering, I simply compared WSplit timestamps observed in my Pokémon Black WR run (3:30) with Abitalive's practice run of the first two gyms (using Twitch highlights). My second round of information gathering involved simply playing the game in the emulator and comparing with my experience playing on console.

Sinstar performed similar tests on his own. Once he determined that the speedup resulted from loading time, he performed a specific test in Pokémon White 2. Starting inside the Pokémon Center in Aspertia City, he measured the time to exit the center, run through the gate to Route 19, and reach the grass. This path included 3 area transitions. His measurements indicated approximately 16 seconds for emulator, 17.5 seconds for DS Phat, and 19 seconds for 3DS XL.

In testing Final Fantasy 4 DS, I simply began the game and mashed through all dialogue and cutscenes as quickly as possible. I timed from selecting "New Game" until gaining control of Cecil outside the throne room. On emulator, this took 2:27.24; on DSi it took 2:33.64; and on 3DS it took 2:32.20. The "20 door" test was performed using the door just north of where you gain control of Cecil. I entered the door once to set my starting position, then timed myself walking through the door 20 times. Timing ended upon regaining control after the 20th door transition.

To compare console models using Pokémon Black, the test steps were as follows: enter your room in Nuvema Town from downstairs. Start the timer, then walk downstairs and back upstairs a total of 20 times (10 times going down the stairs and 10 times coming back up). After the final stair climb, wait for your character to walk off the stairs automatically, and stop the timer as soon as s/he stops moving.

Using this methodology, I measured emulator, DSi, 3DS with capture card, and 3DS without capture card. My times were as follows:

Emulator: 34.56s avg
DSi: 46.06s avg
Capture 3DS: 46.06s avg
Non-capture 3DS: 46.07s avg

Based on these times, it appears that my DSi and both 3DSes run at the same speed as one another. The emulator is approximately 33.3% faster than any of the consoles.

Dabomstew measured emulator, DS Lite, and 3DS without capture card. His times were as follows:

Emulator: 33.90s avg
DS Lite: 44.44s avg
3DS: 46.35s avg

Based on these times, it appears that Dabomstew's emulator runs approximately 31.1% faster than his DS Lite, and approximately 36.7% faster than his 3DS.

Abitalive measured emulator, DS Lite, and DSi. His times were as follows:

Emulator: 36.68s avg
DS Lite: 45.17s avg
DSi: 47.30s avg

Based on these times, it appears that Abitalive's emulator runs approximately 23.2% faster than his DS Lite, and approximately 29.0% faster than his DSi.