Bringing Alita to Life

													Bringing Alita to Life
Although Alita: Battle Angel will be released at the end of 2018 and the first Avatar film came out in 2009, Lightstorm Entertainment began early development work on both projects at the same time. Each influenced the other, says producer Jon Landau. When we first started doing Avatar, we knew about Alita, and we thought that Avatar would really benefit Alita in the facial performance capture technologies that we were developing for it. Now, having produced Alita, I've learned that the reverse will also happen and the Avatar sequels will benefit from new technologies and processes we and Weta developed to bring Alita to life."
	Even so, the universes of Alita and Avatar are very different. As Landau points out, "On Avatar, so much of what we create with the performances is in a completely virtual world. That's one set of challenges. A whole different level challenge is bringing performance capture characters to life in a live-action set, having them interact with other actors, with animals, and the world. We have to realize their character with CGI that lives up to the photographic reality of a physical set. That's one of the different challenges on Alita that we only had to deal with in small parts on the first Avatar. On the Avatar sequels, we'll deal with that in a much greater percentage of the film. So, what we learned on Alita is going to be a real benefit."
	One thing that makes Alita: Battle Angel unique is that it is the first film to have a completely humanoid but CGI-realized lead interacting with physical surroundings. Alita, played by Rosa Salazar in performance capture translated into CGI, interacts with live-action humans, sets and props. Alita exists in a singular space, Landau explains. " Think of visual effects as a bell curve. On one side of it you have all live-action, then you get to the apex where its visual effects and live-action combined, and then you come back down to all visual effects. Alita is landing on the peak of that bell curve, which Is the point of most complexity. We are right at the peak of integrating live-action and CGI. The biggest technical challenge on Alita is the facial performances. Movies are made in the close-ups. So, what Weta Digital did is create a CGI 1:1 model of Rosa Salazar, so that we can look [at Salazar's live work] side by side [with the digital version] and make sure that we are getting one hundred percent of her performance in the CGI world. Once we knew we had that, that performance could then be translated onto the character of Alita that facially is different, but we knew we were getting  Rosa's performance because we had the 1:1 CGI comparison.
	"When we were designing Alita," Landau adds, "we thought we really had an opportunity for the first time ever to bring to life, off the pages of a manga, a character that would hold true visually to the manga style. That's why we chose to go with the larger eyes. That's why we chose to go with the smaller mouth. This was all done through a very deliberate and time-consuming development art protect where we looked at bigger eyes, smaller eyes, bigger mouths, smaller mouths, bigger head, different hairstyles, all to determine how we could best bring her to life in an emotive and engaging way and still be true to the manga. That was one of our challenges and one of the things that the team of artists that we were lucky enough to have working with us achieved very successfully."
	Weta senior visual effects supervisor Joe Letteri explains, "We preserved Alita's oversized eyes from her manga origins, but without breaking from the realism of the world she is in." As Alita also has a cybernetic body, "This balance of large expressive eyes became a key part of her design that helped her character straddle the line between human and machine."
	Alita's protagonist is different — and arguably more challenging— because unlike, say, Caesar in the Planet of the Apes films, she looks almost entirely human, and unlike the Na'vi characters in the Avatar films, the majority of the time she interacts with a practical rather than CG environment. "Alita is pretty distinct in the sense that this is the only film where the lead is completely computer generated throughout the entire film — and not an ape — appearing almost entirely in live-action environments," visual effects supervisor Richard Hollander points out. "I've developed other characters in the past, and in doing so you go through a lot of questions. As a character moves further away from human, the questions and objections diminish; the viewer is more forgiving and it ultimately works. But when you get to something that's anthropomorphically closer to human, it's more difficult [to satisfy human perception of that]. I loved the process, asking those questions: What do we put into Alita? What don't we put into Alita? You have this character and you think you know what she's supposed to do. A pure CG solution gives you a lot of flexibility, but you can't do all of those things and stay true to the character. You have to narrow it down and make decisions about how she's going to act, what things you can do to her, before you take it out of Rosa's performance."
	Letteri affirms, "Alita was one of the most ambitious characters that Weta Digital has ever created. In 2018, when Al and technology advancements are all over the news, creating a leading character that is equal parts young woman and machine meant that we had to stay true to her emotions as she comes to terms with who she is and begins to forge her own identity. Alita is a multi-dimensional character who is equally at home in a dystopian action scene as she is in a romantic encounter with a new boyfriend."
	Weta Digital visual effects supervisor Eric Saindon agrees. "One of the first and most important things with bringing Alita to life was being able to capture Rosa Salazar's entire performance. Capturing Rosa's basic motion was an easy task and not what we were aiming for with Alita. Our goal for this movie was to gather Rosa's every subtle motion, voluntary or not, to elevate Alita from a CG character to an actor."
	However, Saindon points out, most actors show up at their call time and simply go through wardrobe and makeup before starting the day's shoot. "Rosa's daily routine required her to be with Weta Digital's performance capture team two hours before filming started every day. For this film, we built an all-new performance capture suit specially designed for Rosa. It had all the wires and white balls, commonly seen on motion capture suits, built into the garment. With this suit we were able to capture all of her movement, even her breathing while she performed.
	"Before filming started," Saindon explains, "we performed a 3D facial scan of Rosa and printed a fitted mask to help us with the capture dots we applied to her face. By putting holes in the mask, we were able to place the dots in the same place on Rosa's face every day. Connected to her helmet were two small cameras positioned in front of her face, so we could record the change in the dots and her facial performance for every setup while filming. In post-production we would use this footage to determine the exact muscles that were firing in Rosa's face for each frame of the film."
	"Actress Rosa Salazar's rich performance gave us a glimpse into Alita's inner feelings, and capturing the emotions in her face was crucial to making Alita come alive," adds Letteri.
	Making Alita come alive required not only the all-encompassing performance capture, Letteri continues, but changing the way the effects team approached some aspects of the digital program. "We introduced a new workflow into our facial system to make sure that all of the details of Rosa's feelings and expressiveness translated to Alita. We built an exact digital replica of Rosa as a validation step between the captured performance and the digital character, so that we could confirm the digital translation of Rosa's facial performance to Alita was exact. This was a level of complexity beyond what we had previously done with Neytiri from Avatar or Caesar from Planet of the Apes."
	As for Lightstorm and Weta joining forces to create Alita, "Who better to pair to make this film, to jump to this level," Hollander says. "Weta and Joe Letteri and the gang already had a lot of experience of what it was like on the Apes films. They just kept doing better and better work as time went on. You've got this perfect set-up, and then you have this unique environment. I knew about the technology, but just being around it is different. There's the day-to-day aspect of it, like the facial camera sitting right in front of Rosa." Hollander points out that Salazar quickly adapted to the close proximity of multiple lenses. "We're very interested in shooting with reference cams, pointing five or six cameras at her body and face, that are used as reference for Weta and for the editors."
	This, Hollander explains, is to make sure that there is coverage of the performance from all angles, so that the filmmakers can see all possibilities. "The other cameras get that for us. We'd always place one off to the side." Still, Alita has been a steep learning curve. "As I proceeded to he involved in this, you learn what to do, and now I want to make the whole film again," Hollander says. "But that whole process of understanding what you're capturing is incredibly important for doing the edit of the film. You need something there to cut to. This is all practiced behavior in Lightstorm's case. They've been doing this since Avatar and are currently producing the next Avatar.
	"You can't conceptualize your movie unless you have all these parts, including the parts that don't exist [on set] and are going to be CG," Hollander continues. "We used actual elements of Rosa Salazar in her performance capture and that was cut into the film. So we began making the film without our computer-generated Alita. Rosa then begins to get replaced by these different forms. But if you don't have that basis, you can't edit it, and if you don't have some of the reference cameras it makes it really difficult to understand what kind of performance you're going to get. So you do picture-in-picture with her face, to understand how to creatively put together your film. These are really impressive practices. I had not worked on a film where that was so necessary, and I learned a lot. The [practical] shooting went really well."
	"When creating Alita from Rosa's performance, we pay special attention to her eyes and mouth, especially when she is speaking," Letteri says. "The facial model built for Alita had nearly 2,500 facial action shapes — three times the number used for the groundbreaking work on Neytiri. This gave animators more precise control over the movement that creates her expressions than previously possible. We did extensive work integrating specific muscle information and tissue data to validate the proper placement of her digital skull relative to the surrounding structures. This type of precision gave us more refined expressions and fine-tuning of the micro-movements around her mouth and eyes."
	"On Alita." Landau says, "we have really refined the science of the facial performance capture to a level that Weta has never done before. We're doing it in a humanoid character, in a photographic world, on Earth — that is a much higher standard to realize. Weta is using two hi-def cameras for the performance capture, and we're running a series of sessions that Weta calls 'FACS sessions,' where we put the performer inside of a very-confined space, and they repeat the dialogue so that we get all of the nuances of their performance with not just these two cameras, but a multitude of cameras that help us to be more authentic to the actor's performance.
	"From an acting standpoint," Landau continues. "doing performance capture is no different than doing live-action. We need the actors to give us all the subtleties and the nuances of their performance. We then work with Weta to bring that performance to the screen. We shoot a tremendous amount of reference of the performance, so not just the cameras that are recording the face, but off-board reference cameras. Those reference cameras stay with the shot throughout all of the post-production visual effects process. It gives us a continual reference to the performance that the actor gave on the set. We line up daily every shot that we get back from Weta, whether it be at an animation level or a lighting level, and we compare picture and picture - what did Rosa do, what did Jackie Earle Haley do, what are the characters doing?"
	"One thing that helps Alita truly transcend other CG, characters is her interactions with the rest of her world," Saindon points out. "We were encouraged by Robert Rodriguez and Jon Landau to allow Rosa to interact with and touch the other actors as much as possible. This adds a whole other level of integration, especially in a native 3D format, but it's quite difficult to remove the actor from the plate, so it is often avoided. Rather than shy away, we doubled down by building CG versions of all the actors in the film and tracked 3D movement so that we could always be sure Alita fit in the correct 3D space and we got the correct collisions with her hair, clothing and other characters.
	"To ensure we always had the camera coverage we needed on the different sets and locations," Saindon continues, "the Weta crew would go in a few days in advance and wire up fifty to sixty cameras pointing to every part of a film set. This meant, wherever Rosa was, we would always have multiple cameras to use to calculate her movement. As added support, we had six manually operated reference cameras on Rosa at all times. Depending on the scene, these would focus on different elements. In a dialogue moment, the extra cameras would be tight on Rosa's face. In an action scene, they would be set wider to capture more angles that would help us when we were working out her movements and the interaction points with the plate elements."
	While capturing the face is essential, even more is required. "You need to get the upper body and some of the movement," Hollander explains. "Even if you're just doing a talking head, you still have to capture the body to get it really going. Because Weta has been doing this so long, when it gets down to movement, they know that the concept work [prior to animation] will never address that. You get these concepts [for the] face, and some side angles and so on, and it's not what it's [ultimately] going to be; it's a suggestion. So that step is a huge step once you build in CG. A lot of conceptual artists now do the concepts based in 3D geometry, so when they rotate it, at least the proportions are the same. But before, they used to draw it and the proportions would change as they drew it, because they couldn't get it exactly right. We're getting better that way to close that gap. But even then, when they paint lighting on, you have to make that jump early and then do your design in a repeatable environment. You have to re-render it [multiple times]."
	Alita has several bodies during the course of the film: Cybergirl, Berserker, and Motorballer. "Each of these bodies had a different movement style, and therefore meant a different way of thinking about Rosa's performance," Letteri says, describing the process. "When creating Alita's body, we knew developing a visual language of mannerisms and posture based on Rosa's movements was essential to maintaining Alita's character as she changed physical form throughout the film. Her Cybergirl body alone had 7,700 pieces, including her intricate mechanical internal elements and clothing. Alita's Cybergirl body is made from distinct material types comprised of different metals and featuring an alabaster skin with unique light scattering properties. Despite this mix of materials, we always made sure that each of Alita's bodies could move fluidly to capture the gracefulness or Rosa's performance. As Alita starts to connect with her former life, we see her start to own her own body as she rediscovers her knowledge of her unique cyborg fighting style, Panzer-Kunst."
	When Alita decides to compete in the lethal Motorball game, Letteri says, "Animators added fast and fluid acrobatic movements that demonstrated the strength and training that builds her confidence. In her Motorball form, Alita's movements were animated to look effortless; she is clearly in her element."
	However, these were simply initial stages in realizing Alita. "Establishing the design and movement of Alita was just the first step to bringing her into the world of film, "Letteri continues. "Her hair and clothes, as well as the different materials of her body and her skin, all played vital roles in establishing her believability. There were over 150 different hair grooms and clothing items created for Alita to fully realize her different incarnations and style variations. All of these forms had specific versions for dry, underwater, transitioning out of the water, ambient wind, motorbike speed, and Motorballer action. These were all managed with carefully art-directed simulations."
	In terms of new programming, Letteri explains, "We were able to take full advantage of our new Physlight addition to our proprietary renderer, Manuka, which allowed us to add an even greater level of photographic realism to Alita. This can be seen most clearly in her skin. Being a teenage woman, she doesn't have many wrinkles or blemishes that help convey the natural properties of skin. But the subsurface scattering of light in Alita's skin, combined with a true spectral response from the renderer, gave us the ability to create a future-tech synthetic skin that still allows audiences to connect to her as fundamentally human."
	Hollander points out that, as the Alita character is a Total Replacement cyborg, "That face is not real skin, but it emulates real skin. So you start to feel comfortable watching her. And she's an advanced TR, as well. I think that is the factor where, if this works, all of a sudden you forget that she's got the alabaster body, gears on the inside, or the Berserker body that's got this other, unknown technology underneath it. You should just be focusing on that performance and her issues in the movie." There are other significant characters besides the main protagonist in Alita who are completely or substantially realized in CC, These include Jackie Earle Haley's tormented hulking former Motorballer Grewishka and Ed Skrein's arrogant Hunter 'Warrior Zapan. They went through the same design process its would fully live-action characters.
	"Oftentimes, people get confused when they're dealing with something that's computer generated." Landau says, "We had an incredible team of artists who worked to design Grewishka, Zapan, all of these characters. We would then look at the final designs we had, and we then had to determine how we were going to realize them. Just like we did when we were making Titanic. We knew we needed the ship, but we had to work out how much of the ship we needed to build practically and how much we could do digitally."
	Landau delineates what was needed. "With Zapan, we said. 'We're going to use Ed Skrein's whole face photographically. But Zapan's body is a computer-generated body that replaces Ed Skrein's body. Jackie Earle Haley, who plays Grewishka, is this powerhouse of an actor in a small frame and here he is [giving] a powerhouse performance in a very large character. As with Alita, we need to use computer-generated technology to create his performance on screen, and we do that with performance capture We don't call it motion capture, because we don't just want the motion of a character, we want the emotion of character - and that is the performance we get. That's what we do by simultaneously capturing the facial performance through a head rig that is worn by the actor and through body markers."
	In the case of the gigantic Grewishka, Landau continues. "One of the unique challenges was his height. We needed to give actors the eye-line, where to look when he was on the set, so that they would know where they were reacting to. For that, we found a stuntman and put him on stilts so that he could be at the right eve level. He would walk the performance, Jackie Earle Haley would be right there giving the vocal performance and the other hand gestures that the stunt person effectively mimicked. The head height gave us the right eye-line for all the caller actors in the scenes."
	Hollander recalls another on-set solution," A good, simple example is shooting the scene in Chiren's lab when Grewishka's in Chiren's repair rig. We built a chair for Jackie that he had to crawl up on that put him up high, so his head was where Grewishka's head was going to be. So there he sits, and his eye-line down to Jennifer Connelly is perfect."
	A difference in size between performance-capture actors and the characters they're playing is not uncommon. "Any time you have a CG character, its almost harder when you don't have the actual thing in its actual size [on set]." Hollander explains. "That changes your production quite a bit. Grewishka has three different phases. He goes from eight feet to nine feet tall. His size is enough to use lots of problems. You want people's eye-lines in the right place." So, in order to help the live-action performers react properly, "We built a big picture of him, from the artwork, full size. And everybody said. 'Oh, I get it.' That stuff helps. For Grewishka, there's a lot of stand-and-talk, so it wasn't so hard to do. When he moves around, most of the time he's fighting Alita, and that's all in CG."
	A process called Simulcam was also beneficial. "Simulcam - meaning simultaneously capture - via a gaming engine and pipeline, generates an image that looks like Grewishka, and animates in real time, and mixes it in with the frames that are coming off the camera [into the Video Village monitors]," Hollander says, "Weta had the ability to take the performance capture of Jackie — not his face, just his body and head movement -- and put that into our model of Grewishka, in real time, where we're shooting, and composite that image of the Grewishka character back into the camera's view. So, the camera moves around and there's Grewishka, right there. On the set, there is a monitor that is the result of the camera taking [the live-action footage] and the Grewishka CG being generated live. We did that almost every time we saw Grewishka in the film. We fed the Simulcam image back to the camera operator, so he knew where Grewishka was, how much he had grown."
	Hollander gives an example of how Simulcam helped during shooting. "There's a scene where Grewishka walks into Vector's office. [Five-and-a-half-feet-tall] Jackie was walking across the floor, so Jackie's head was nowhere near where eight-or-nine-feet-tall Grewishka's head would be. [Cinematographer] Bill Pope and his team pointed the camera relative to the Simulcam not to where Jackie was. Now we've got Grewishka's head in the right place in the upper portion."
	Even a few years back, Hollander observes, it would not have been possible to make Alita as it is being done now. "We've had different incarnations of the technology and it only improves. I shot pickups for crouch references using a new technology called N-Cam. Lightstorm owns one and Weta owns one. It's a wonderful way to understand where the camera is moving in 3D space. So, I had crowds I could match tip to the CG shots we had already built."
	For all of Alita's groundbreaking CGI advances, Landau wants people to remember that it is primarily a Iive-action movie. "We have certain scenes that are completely computer-generated. But at the heart of a it,we went out and built a 97,000-square-foot set in Austin, Texas." Landau emphasizes. "We had our actors walking on real streets interacting with each other, interacting with the environment they are on.
	"If we're on a virtual stage, like we are with Avatar, you can set up a permanent facility within which you can do all of your capture," Landau continues. "On Alita, for the majority of the shoot, we were going from one physical set to another and to another. The team at Weta Digital had to come in and put up performance capture cameras all around these live-action sets in a way that then could capture the performances, but not be overtly seen in the scenes. Big challenge."
	For the live-action, Landau notes, "We approached it as a very efficient production in Austin, where we had Bill Pope, the cinematographer from The Matrix, as our director of photography. We had Garrett Warren, our stunt coordinator from Avatar, and other movies like Logan, working with us, and treated it really like a live-action film that, in the center, had this very unique challenge of taking Rosa Salazar's performance out of photography and creating her character through computer-generated imagery."
	All of the technical elements also had to Serve the distinctive vision and style the of director Robert Rodriguez. "We're accommodating Robert's fast and furious style of shooting as much as we can," says Hollander. "He's got his own style of making decisions and coming to conclusions and asking for different things. Weta did a lot of design work so they could do similar things to their works on Apes. They had a large crew on set at all times, large amounts of equipment, cameras by every single set, performance capture cameras being moved around all the time, because we almost had a performance by Rosa. We wanted to make that whole environment as seamless as possible for Robert. It was pretty amazing."
	"There are certain action sequences that are all virtual," Landau adds, "and for those sequences, we turned to the process that we used on Avatar, which I call virtual production. When an actor goes onto a performance [capture] stage, we have a performance capture system and a virtual camera, and we realize the sequences virtually."
	Integrating CGI into a live-action shot and creating a completely CG shot for a film that is largely live-action both present challenges. "Implementing anything CG in a live-action frame in 3D is challenging because there's an integration issue between the two," explains Hollander. "This film was shot in native 3D with two cameras. When you try to integrate things that aren't real with everything else, it adds a whole other dimension in energy and technical expertise. But on the other hand, the challenge with an all-CG scene is to make it look like its not CG. Today, its easier than it was five years ago, with better tools for rendering, but you have to work at it and the difficulty level depends on the environment that you're trying to duplicate."
	Of the work Lightstorm and Weta are doing together on Alita, Landau concludes. "We have been working with Weta for more than ten years now, and we have a great shorthand to get through this process. We have a team of people here [at Lightstorm's headquarters in Southern California]. They work daily, hours upon hours, with Weta. It would not be unusual for us to have six-, seven-hour reviews with Weta looking at these shots on a frame-by-frame basis. The relationship with Weta is, first, a friendship, and second, this incredibly rewarding collaboration."