Mirishita 2019 CEDEC conference slides

This is a rough translation of Mirishita's CEDEC conference presentation slides. This should give you a good understanding of how Mirishita's design philosophy and software design/architecture works.

While I do have some background on software engineering, I admit that technical terms are not my strong suit, so I apologize in advance for any errors. Regardless, do enjoy!

Also, be sure to refer to the slides when reading this as well.



_______________________________________________________________________________________________________________________________________
Mirishita Management Techniques!!!


- About this Session
 - Contents:
  - Introducing the Mirishita Engineer Team's management guidelines and examples of practical application
 - Session Targets:
  - Engineers with experience developing, operating management titles (games which are constantly updated rather than a single release)
  - Engineers involved with management titles
  - People interested in Mirishita's Engineer team


- Session Contents:
 - Engineers' initiatives
 - Server-side initiatives
 - Engineers' challenges


_______________________________________________________________________________________________________________________________________
Engineers' Initiatives
For the sake of stable operations


- Self-introduction:
 - Sasaki Katsushi (佐々木 克)
 - Bandai Namco Studio - Programmer
 - Mirishita Client-side Engineer


- Management team structure:
 - Total team size: >100 people
 - Engineers: ~30 people


- Our mission:
 - Implement the specifications of functions and ideas based on commercial strategies that are always aware of user needs via engineering, and release them with stable quality and in proper cycles.


- Definition of Stability:
 - Don't stop service
 - No bugs in the updates
 - Progression according to schedule


- How long does it take for a song to be released?
 - About 6 months for a new song!

- 6 months before:
 - Unit (and song) concept is decided.
- 4 months before:
 - Production plan decided
 - Progamming begins
 - Performance data is prepared
- 3 months before:
 - Motion capture
- 2 months before:
 - Finish programming
 - Production work begins
- 3 weeks before:
 - FIX
-  2 weeks before:
 - Begin testing
- Release


- About 4-5 songs release every month
 - Platinum Star Theater
 - Main Scenario (Commu)
 - Platinum Star Tour
 - Main Scenario (Commu)
 - ?


- The Premise
 - Client updates take time
 - Systematic progress is essential


- Engineer Guidelines 1:
 - 80% Rule: If you have 7.5 hours, use 6 hours to program


- Reasons for 80%
 - Emergency support
  - Investigation of errors
  - Responding to requests of users
 - Others
  - Refresh (they need to take breaks too lol)
  - Technical research


- Engineer Guidelines 2:
 - The schedule is of the utmost priority


- For the Schedule's sake
 - Grasp the mid-to-long term plans before determining the points that absolutely cannot be left out
 - Proposal of a test-driven schedule that is reliable and sustainable
  - Must have a thorough understanding of the test schedules' evaluation criteria


- What if we can't ensure testing time?
 - Send it off unsullied
  - Don't include any riskier updates
 - Volume adjustments
  - Whole team will help to shrink features


- What if, after all that, we can't do it and it's no good?
 - We do our best and implement it
 - Use a Boost (lol)
 - Cannot be conservative with updates

_______________________________________________________________________________________________________________________________________
Server-side Initiatives
For the sake of stable operations


- Self-introduction
 - Hoshina Kazunari (保科 一成)
 - Bandai Namco Studio - Programmer
 - Mirishita Server-side Engineer

- Server-side Guideline
 - Ensure the maximum possible time for gameplay


- For the sake of maximizing gameplay time
 - In order to never stop:
  - Ensure sufficient capacity
  - Scaling
  - Do not create even a single point for malfunctions
  - Never cause a problem that requires emergency maintenance
 - In order to reduce the amount of time needed to stop:
  - Shorten the regaular maintenance period
  - Resolve issues without stopping the game
  - Be able to switch back on the spot


- Types of Maintenance
 - Regular maintenance
  - Backups
  - Changing of database schema
  - OS and middleware update
  - Waiting for client updates
 - Emergency maintenance
  - Infrastructure failures
  - Fatal bugs

- Mirishita and Maintenance
 - Ever since we opened on June 2017, the game has never stopped even once for maintenance
  - There have been some parts that have stopped due to failure
  - Can do maintenance on certain functions

- Mirishita's Server
 - Google Cloud Platform: We employ Google App Engine (GAE)
 - For more info, search "The GAE/Go that supports Mirishita"
 *(Is a seperate presentation, https://www.slideshare.net/GoogleCloudPlatformJP/gaego-80349110)


- Characteristics of GAE
 - High availability
  - A standard decentralized configuration, so not likely for trouble to occur
 - Scaling efficiency
  - The servers can be started in seconds, so only need to do scaling at moments with heavy load
 - High capacity
  - Can bear around 10,000 requests per second (in Mirishita's case)
 - Fully managed
  - Can scale automatically according to the number of requests and amount of data


- Back to Types of Maintenance...
 - Regular maintenance
  - Backups (Can run without stopping)
  - Changing of database schema (No need)
  - OS and middleware update (No need)
  - Waiting for client updates
 - Emergency maintenance
  - Infrastructure failures (Reduced)
  - Fatal bugs

- Maintenance for the sake of updating the Client
 - If the server and client versions don't match and it can't operate (Incompatible), must enter maintenance in order to wait for a new client update
 - Even if the store releases a new client, not all users can download it - restriction
 - How does Mirishita solve this?

- Do our best to preserve backwards compatibility (For the sake of Client updates)
 - The server supports both old and new clients
 -  Before a new client is released, update the server first so that you can download it immediately upon entering the store


- Handling bugs
 - When bugs occur in the production environment, in certain cases the whole game needs to be stopped to perform maintenance to prevent the damage from spreading
 - Very important to detect bugs before they occur
 - How does Mirishita solve this?

- Do our absolute best to test (Ensure no bugs come out)
 - Bring our test environment closer to the production environment
  - Move server configuration closer
   - GAE can make test configurations the same as the production one
    - Different security settings
  - Reduce the amount of data
   - Function to create dummy data
  - Move the settings closer
   - The settings differ based on environment, bring it closer step by step


- Production Environment Update Flow (refer to the diagram here for illustration)
 - Update production environment from V1 to V2
 - Test with a combination of the V2 Client and V2 Server (Development)
 - Test with a combination of the V2 Client and V2 Server (Staging)
 - Test with a combination of V1 Client and V2 Server (Staging)
 - Test with a combination of the V2 Client and V2 Server (Production)
 - Since iOS requires a store review, submit V2
 - Now all Clients use the V2 Server


- Test Environment
 - Step-by-step application
  - Test using different Client and Server versions
  - Fill in the gaps from Production-side
 - Test in Production environment
  - A debugging user exists, invisible to other users
  - Testing future features


- Bug handling
 - January 2018: Players' scores were not recorded in the Ranking
 1. Notify users after discovery
 2. Confirm a fix in the Test environment
 3. Apply the modifications to the Production environment while application is still running
 4. While operating, restore the rankings from the Play Log


- July 2017: Setting error that prevented users from returning to the Home screen
 1. Perform an immediate version switch-back
 2. Inform users
 3. Confirm a fix in the Test environment
 4. Apply fix to Production environment

 - Detailed Action Log
  - If we have a detailed log of a user's selections, stats, used and earned items, etc., there is a very high chance that we can restore their state later.
 - Rollback function
  - Most dangerous after new deployment, so be prepared to roll-back to the immediate previous state

- January 2019: A bug where displayed items could only be purchased once in the Item Shop
 1. Perform maintenance on the Shop function, inform users
 2. Confirm a fix in the Test environment
 3. Apply the modifications to the Production environment while application is still running
 4. Re-open the Shop function


- For the sake of maximizing possible play time
 - Server layout
  - High availability architecture
 - Test environment
  - To reduce the gap between Test and Production environment
 -  Handling bugs
  - Prepare and exploring for ways to do it without stopping the game


- This does not mean no maintenance
 - There is no value in no maintenance
  - If so, when maintenance is really required, our judgement would be less capable
  - Is a factor that can hinder challenge
 - Always aim for high user value
  - To continue playing with a peace of mind
  - To always deliver new content

_______________________________________________________________________________________________________________________________________
Engineers' challenge
To surprise the users


- Self-introduction
 - Hayato Ikeda (池田 早人)
 - Bandai Namco Studio - Programmer
 - Mirishita Client Engineer


- Recall Our mission:
 - Implement the specifications of functions and ideas based on commercial strategies that are always aware of user needs via engineering, and release them with stable quality and in proper cycles.

- Further mission:
 - Always with a spirit of challenge, implement and deliver surprises that will exceed a user's expectations
- That is to say:
 - Because there is "stability", the challenges we take on will always be connected to the element of "surprise"


- For the sake of surprising the users
 - Have an environment that can challenge us
 - Constantly pursue intellectual curiosity
 - Share ideas among one another

- What does it mean to surprise the users?
 - Do something you would never think about
 - Even if you did think about it, do what you still haven't done yet


- Case studies:
 1. 13-Man Lives
 2. Mirishita Kanshasai (Thanksgiving Festival)
 3. TEAM_UMIMI


- What is 13-Man Live?
 - A new mode added in April 2018
 - Increasing the number of idols rendered on stage from 5 to 13


- Technical background
 - Before release, the AKANE Great Strategy (refer to my previous translation of the Unite 2018 slides) was performed.
 - Implemented in the Test mode
 - Improved terminal performance


- Surprise from the project members
 - The impact of the test display was very large
 - Increased the unity of all the project members
 - Able to add more function in a short time period

- Surprise from the users
 - The impact of 5 to 13 members on stage during the teaser trailer was very large
 - A short time period of 3 months to add the new mode
 - Added 13 costumes at the same time
 - Now, 13-Man Lives are regularly added


- What is the Mirishita Kanshasai?
 - A special Live Event that took place in October 2018
 - Had a special in-game livestream that many users could enjoy at once

- Technical background
 - Studying of video playback methods and constructing a test environment
 - Cooperated with Bandai Namco Rights Marketing
 - Implementation of the chat, stamps and survey functions
 - Conducted thorough testing for this one-shot feature

- Surprise from the project members
 - Various ideas came out in the first attempt
 - Possible to play live video from within the app
 - Might as well do it in a vociferous manner
 - A positive spiral of ideas and implementation

- Surprise from the users
 - Surprising that real-time livestreaming could be done from within the app
 - Stable servers
 - Various real-time functions implemented:
  - Chat
  - Stamps
  - Surveys
  - The birthday button (remember the Chizuru skit?)

- Feelings of Thanksgiving toward the users
 - Present of the Official Pamphlet
 - Also implemented a stream archive
 - "We should be the ones. Thank you."


- What is TEAM_UMIMI?
 - Technical research project
 - Using Mirishita to perform technical research!
 - Effective use of the 80% rule
 - UMIMI is an acronym

- What's UMIMI?
 - Ultimate MIllion Live wo MIseteyaru!
 - A team to show off the Ultimate Million Live

- Are you surprised?
 - We don't have anything we can show you today, but stay tuned... I think you'll be surprised
 - Look forward to any future updates!
 - Continues Challenging!


- Engineers' challenge
 - Challenges are built on a foundation of stability
 - (Good techniques + Proper measures) = Surprise from the users
 - Aim for that positive spiral
 - Continue to take on challenges


- So what are Mirishita's management techniques?
 - Stability & Surprise

- Thank you very much!