It Was No VFX Snap Introducing Smart Hulk to ‘Avengers: Endgame’
The introduction of Mark Ruffalo’s Smart Hulk was the great surprise and the visual effects highlight of “Avengers: Endgame.” As Bruce Banner explained in the delightful diner scene before dabing for his selfie: “For years, I treated The Hulk like he’s some kind of disease, something to get rid of. Then I started looking at him as the cure. Eighteen months in the gamma lab, I put the brains and the brawn together. And now look at me: Best of both worlds.”
Thus, it was imperative to pull off the right combination of brains and brawn through the performance-captured animation. The task fell primarily to go-to Hulk specialists Industrial Light & Magic, with Framestore handling the Avengers hangar sequence for the time travel testing. Going off of Marvel concept art, a new twist on the Professor Hulk hybrid from the comics, ILM took some online footage of Ruffalo and made a keyframe test to show off the way they could handle the nuanced emotional moments.
“The range of performance that Ruffalo gave [in ‘Endgame’] was goofy, serious, sad, happy. But he was in control,” said Russell Earl, ILM’s VFX supervisor. “And, with the man and creature, we had to figure out where on this dial to portray that physically. Obviously, we knew the force was going to be Bruce Banner, but then how to have that come through the creature and find out what that balance was. Because you go too human, it’s not Hulk anymore, and you go too creature, the voice may not fit, so there was a lot of careful consideration given.”
For Smart Hulk, ILM rebuilt its toolset from scratch, upping its facial performance-capture system (which won an Academy Sci-Tech award this year) to handle the necessary fidelity and detail of Ruffalo’s face as the central component of the new CG character. “We went through three different iterations of software so shots were redone,” added Kevin Martel, ILM’s animation supervisor. “It was like building a car as you’re driving on the highway. We were fortunate to incorporate Disney Research’s Medusa [performance capture] solver for the first time with head-mounted cameras, so we could take our footage that we shot on set or in the volume with Mark, and then tweak the solver [an Academy Sci-Tech winner from 2017] to work with that. Prior to that, it was more like an ADR setup.”
The high-res Medusa data formed the basis of the Banner computer model, which was re-targeted onto the new Smart Hulk model. On top of that, ILM rewrote its animation software to provide more controls for the artists to alter Ruffalo’s performance. It was about getting as much of Ruffalo as possible into Smart Hulk, while, at the same time, staying true to the look of Hulk (whose green color was subdued). The task was made more difficult by Ruffalo’s asymmetrical face. Expressions didn’t always match and they had to find a Smart Hulk solution without diminishing what Ruffalo provided. In addition, the latest set of muscle tools were applied to accelerate or suppress certain areas of the face, depending on the emotional requirements of the scene.
“Now that we see it on the character, is it reading the same way as did on Mark Ruffalo, and does it feel the same?,” said Martel. “If it’s too broad or too subdued, we could carefully reintroduce those elements that Mark has to make them into Smart Hulk.” They could play with tone as well, if Ruffalo came off as too cocky or too angry. The narrative obviously required more brains than brawn, more talking and less fighting. He works with Robert Downey Jr.’s Tony Stark to build a time machine to retrieve all of the Infinity Stones, and then crucially convinces The Ancient One (Tilda Swinton) to give him The Time Stone. He can’t overpower her because she separates his astral form from his body, so Banner must prevail.
Yet ILM also had to deliver the proper Hulk body, even though the dapper Banner wears stylish clothes and gargantuan glasses. “We didn’t want to have a totally Hulked out body,” said Earl. “But, at the same time, he’s a big dude so we had to keep some of that Hulk. Even through the clothes we needed to show that he’s still pretty buff. We rebuilt and re-sculpted from the previous films with all of his muscles firing away in the anatomy.”
The key moment occurs when Smart Hulk puts on the Infinity Gauntlet and suffers significant damage to his arm. “We filmed that early on where he walks up and says he’s the one to wear it because it’s mostly gamma rays,” Earl says. “It’s an iconic moment that shows off the different layers of effects, burning up his skin, his suit, and the difficulty of doing the Snap. In early tests, we tried to capture the subtle performance from Mark and now we’re pushing right up to the max. How much do the veins in his body start to pop out? How does his face hold up for that amount of intensity and pain to survive? It had to be believable without stepping over the line and becoming too fantastical.”
