The Last of Us turned in another hot button season in its second year. Audiences buzzed over an unexpected death (to those unfamiliar with the video game, anyway) and a shift in perspective to Bella Ramsey’s Ellie. Yet, the series also shocked and delighted with its accomplished and frequently groteseque special effects.
VFX supervisor Nick Epstein and animation supervisor Dennis Yoo were a huge part of creating the epic horde attack on the town of Jackson in the season’s second episode. Here, in an interview with The Contending, they talk about creating the look of the entire horde. That included varied running looks as well as variety in tattered clothing. Epstein and Yoo were involved in every VFX aspect of the season, including the horrifying but epically cool Bloater.
The Contending: One of the biggest moments this season was seeing the huge Bloater attacking Jackson. What went into creating that horrible looking creature?
Nick Epstein: A grotesque beauty! [Laughs] The creatives this time wanted something bigger and more imposing. I think we already did a really good job in season 1 and this time we just wanted to up the ante. Part of that is the way he is introduced, things are already going bad for Jackson, the whole town is under siege from this infected army, and then all the sudden you hear the Bloater. I think the visuals needed to match up with this “oh, shit” moment. You needed to see that it’s a bigger Bloater than season 1, making you think, how can this get any worse? We looked for a lot of references, and the creatives directed us towards wrestlers, not professional wrestlers but just really solid guys. We experimented with a few different things, we even had some comically large Bloaters and some even more bloated Bloaters. It was really cool to go through the creative process with Craig (Mazin) and Neil (Druckmann), especially since Neil knows everything there is to know about Bloaters. So showing him a variation of something he is the expert on was really fun. Our art director Gino did some really great concepts and it worked on the first season Bloater.
For our final Bloater we added more Cordyceps detailing on him, which was derived from the fungus called chicken of the woods that one of our people actually went out and scanned and then we melded that onto his body. Then we added a lot of snow and ice geometry all over him and then Dennis, there were a few considerations on the movement from season 1 right?
Dennis Yoo: The biggest thing was the weight. Even if we motion captured someone, nobody is going to convey that kind of weight on a creature. Adding that weight was something we did after the motion capture. We started keyframing it on top and changing the motion there. It was similar to season 1 but he was bigger and there were also no places to hide from it. In season 1 the lighting helped, and here the lighting revealed everything, including the tissues in the muscles. I was originally worried about that and talked to Nick but the lighting helped us there, so for this season I wanted to make sure when he is stepping there is a muscle structure and that you can see the ripples. Luckily we had some tech from other shows so that you could actually see the muscle and the fatty tissue, especially in the legs.
Nick Epstein: It’s that reveal shot where you realize oh, shit, there is this huge monster that is not only bigger but has the proper muscles as he’s running towards us.
Dennis Yoo: One thing I wanted to make sure of was that we were not hindering ourselves. In season 1 you had a lot of Cordyceps where they should not have been, especially on the hip, which was constricting the leg. I wanted to make sure that wasn’t a hindrance at all, and originally we had a lot of Cordyceps on his shoulder and it looked great and was very large, but I realized he couldn’t turn his head to the right. So we had to go back on that design. Overall I thought we did a fantastic job this season.
The Contending: I thought it created the terror quite nicely. But what I loved though, even before the Bloater appeared, was Abby comes across the infected in the snow and first we’re, like, okay, did they die and they are frozen here? Then we see no, no, they’re not, they’re hibernating, and just the way they slowly emerge and then merge into that horde. What went into creating that motion there to release them all like that?
Dennis Yoo: I think that particular shot was technically our hardest shot. Initially we saw these plates of just a dolly up and looking down, and we’re just trying to figure out what was supposed to be in there. And it took a while to figure out what Craig actually wanted, a layer of dead bodies on top and then those infected to crawl through those dead bodies. So we ended up using Ragdoll, which is an animation simulation software, and we dropped a bunch of bodies into a pit. Then from there we got our motion to get those guys crawling out, and then on top of that we had to simulate it again to get them to react to each other. Then we had the Keyframe on top, and motion edit to make sure all those contacts are working and that the performances are right. Just because you motion capture something doesn’t mean that that’s the performance Craig wants to see. So a lot of times we have to change the performances, and then after that there is cloth stimulation because we don’t see cloth in animation, so we had to grip as closely as possible to the actual model itself. The cloth is added and then we do a little tweaking on top of that so it’s all this layered process, and then Nick, you want to get to the effects of that?
Nick Epstein: You have the couple nature of the motion edits, and the Ragdoll simulation, on top of that everything has to go to cloth and hair simulation. With all the contact, that was quite a lengthy simulation for that sort of sieve of bodies, and disturbed a hive of infected crawling up through them. And then obviously you need to pass this off to the effects department, who do all the snow simulations on top, and they wanted a more realistic cadence through the snow. These infected, although they’re frantic, they need to struggle a bit because they’re trying to get purchase, so we had to take into account the forces generated from the snow simulation that impacts on the cloth. Dennis has to anticipate that and go okay, we need to put in a bit of a struggle here so we can’t just play back our motion at 100%. Essentially there’s all of these simulations and processes that have to come together in a shot like that, which makes it really really complex.
Then of course you’ve got your base snow simulations and snow effects on top of the surface as well, like the rolling blowing snow, which has to also interact and affect the bodies crawling out. It’s a fairly long shot, but there was so much work that went into those four or five seconds of screen time that was pretty tricky. We also had the build up to it, which I really love. It’s sort of eerie. Are they alive? Are they not? Like, what the hell’s going on here when she first rolls down. That clicker head motion was a pretty cool process to arrive at that. Simulating the snow underneath the clicker, which in turn basically causes the clicker to rotate in this eerie way, and also start descending at the same time. I think the end result looks like it was almost effortless but it took a lot of exploration to figure out the right rotation and the right descension as well. That was quite a process, and I think quite a cool approach to a shot like that.
Dennis Yoo: It’s just a clicker’s head turning in the snow and no one’s going to know how much detail went into that. It was incredible. If you think about snow, It actually crumbles differently than, say, sand. Snow has a hard crust on top and different temperatures of snow also look different. So seeing all the notes coming, I don’t know how many references to snow we had in the end. I just realized how hard it was to simulate snow. It’s a very different beast.
The Contending: It was worth it. I loved that clicker scene so much! I read that you guys had to kind of create a new sort of weather control system to show this area and in Wyoming. Can you talk about what went into that?
Nick Epstein: That was born out for two reasons. One, we knew that the storm was going to be a character in this episode. It’s very metaphorical. It also develops at the beginning and it’s got a very clear trajectory through the episode while the horde attack happens and peters out in the aftermath. So we knew that we would have to be able to tell that story as Craig and Mark (Mylod) the director wanted to, by making sure that everything conformed to those blizzard conditions as the storm came through. Then, the other reason was I was on set for the Jackson siege part of the shoot, and I think it was the second or third day, we had blazing sunshine and then we had pouring rain, which created puddles everywhere, and then by the late afternoon when it got a bit colder we had snow. These wildly varying shooting conditions went on throughout, and I knew from those first few days on set that we’d need some very robust processes in order to deal with that, and conform it to the weather conditions that the filmmakers wanted.
So we used a few techniques for that and they also became part of this toolkit that the compositing team used. The basis of it was some machine learning based depth and segmentation extractions to allow us to have a usable depth space that we could then insert, not just the weather, but also our CG infected horde in and amongst plate stunt actors. Then being able to extract those depth and segmentation mats meant that during animation, for example, we could layer in our horde and give Craig and Mark and Alex (Wang), who’s the production effect supervisor, a clear look at what we’re proposing before we go through all of the rendering and shading. Then the weather control bits sits on top of that, and the composing artists, estimating a wind direction based on what was happening in the plate, and they would estimate strength and density and then they would populate a number of particle systems for the blowing snow and a number of volumetric like pre-cash volumetric simulations, but slay them out in the correct deck space and tied into the correct orientation that we needed for the storm and adjust the density for them. That would give our artists a very good starting point with hopefully little effort because we knew this was something we’d have to replicate, these kinds of shots, more than 100 times where we needed to have full control over the weather. So the process needed to be robust, and I think it worked really well.
The Contending: For the siege of Jackson itself one thing that I really liked was the realism of the attack and the way we see the smoke building. Can you just kind of talk about what went into the battle itself?
Nick Epstein: For the battle we got involved early on the previous aspect of it, and it was used as a guide when they were shooting. A lot of the events, like when the barrels explode outside the gates and stuff like that, is very carefully choreographed on set. One of the things I love about the way The Last of Us generally approaches shooting is that it is very grounded. So even if a shot’s going to be 100% or just a large amount of it is going to be computer-generated imagery, they’ll still shoot a plate. For all of the explosions they had stunt actors running through those barrels. The barrels were exploding and guys were running through, catching on fire (obviously being doused with anti-flame afterwards). That gives us a really clear basis for the beginning of our work that is always grounded in reality, which sets that realism bar higher. It makes it harder in some ways but I also love being sent that challenge of, this is what reality looks like, and yours needs to be as good or better than that, so we build around that.
Dennis Yoo: Going back to the realism of it, I was really impressed when Alex would show me the plates of barrels, and you don’t really think much about the barrels but they actually built those ramps and they had the barrels roll off. So, even though they rolled off and they just fell, seeing that weight and what we needed to do better was tremendously helpful. Going back to the preview and the planning that started with the script. Craig is obviously very story-driven because he’s a writer, and I love his planning. It wasn’t just written as battle ensues that I’ve had in other scripts; he was very detailed in what was going on. So we had the script and we did a lot of motion capture for it because we didn’t have many boards, and then we started laying out cameras with the actual actions, then there was a back and forth. Being there for the initial capture was really interesting. We just tried to capture as much as we could, knowing that potentially things wouldn’t be CG but just have a pre-visualization of what might be shot.
Seeing the set coming to life was very cool, and then having the plates coming back and seeing that they looked just like the previews, or seeing that it was totally different. Also what really helped was they had the plans for the set and that was given to us. So we actually got to stick a CG virtual set in there and stick cameras, and that actually helped with where cameras might go on set. We’d stick a camera on a roof and I would get a call back saying there’s no way anyone’s going to get up on the roof. Can you put that camera down on the balcony? It’s obviously easy to put a camera anywhere you want, but having that grounded with an actual visualization of a set and what a camera might be doing, that definitely helped on both ends, making our previews look great, and giving them ideas of where they put their cameras, and what sort of compositions they can get.
The Contending: I read that you had some technology to vary the clothing of the infected and I thought that was really cool, because there’s not supposed to be a uniformed look there. They are all different people. What went into creating that intricate program?
Nick Epstein: After getting over the shock of what the expectations are and seeing the plates come in and knowing that a full screen CG horde goes here, but then thinking about your biggest problems, you realize variation is going to be one of them. We scanned thirty of the stunt actors with prosthetics, but we have a lot of shots where you’re seeing one to 200 of infected pretty close up, and because they’re usually charging, the front line isn’t a constant frontline all the way through the shot. So you can’t just say oh, I’ll have my thirty hero infected at the front and then everyone else behind, in a crowded lower fidelity version. We were dealing with essentially a crowd of hero characters so we had a few systems for dealing with that, like one was that the infected could wear any piece of clothing or inherit any hairstyle or type of grooming. There was a dynamic refitting aspect to that where our creature artist during the pre-roll of the simulation had a system that would expand and then shrinkwrap the clothing back on so that we didn’t have to have models go in and refit every piece of clothing, it was dynamic.
Then the other part of this was the shading offering, which is usually something that’s baked down during look development, and then the asset departments publish that. It’s a very static thing, but between our look depth supervisor and our CG supervisor, we worked on a system to basically expose all of that for the lighting artist so when they’re actually composing the lighting for the shot, making sure it matches what was happening on the set, they are also in control of what the infected are wearing. So when you get those shots where they are running down Main Street our lighting artists really went to town with that. Giving notes like, I think you can only have one pair of orange trousers, because two or three is really going to stand out. Then we were getting notes from Craig that it’d be great to have a red and yellow striped shirt there, and we were able to address those notes because we anticipated the need to be able to vary these things at the very last point of the pipeline. It wasn’t super straightforward to do that, but it was pretty rewarding to anticipate a problem and you already have processes in place to address that problem.
We have artists coming in saying I went overboard and did five or six versions. I would say no, that’s a great problem to have, because there was so much artistry we didn’t think was going to be there initially, where we now had people to almost create a backstory to some of these infected with how they designed the clothing in different ways. The infected were once humans so they did have a backstory and so they should reflect that in what they’re wearing.
Dennis Yoo: Things we had to deal with in animation we tried the mix and match in the low rev version, but we couldn’t see all the extra shading. If you think about the modeling that would be involved for animation it would take an exorbitant amount of time. So what we ended up doing was we had the base of the model still. And we put the crowd together in animation a lot of the time. We had the same base model together, which wasn’t great especially when they were close up. I’d always get calls from Nick saying, “You know those two guys are exactly the same.” So it was almost like a Rubik’s Cube where we only had about thirty base models to fit hundreds of guys. Interestingly enough, our horde wasn’t orcs where they all have similar armor. They had different sizes and skeletons and Craig didn’t want to see all adults; he wanted to see children infected. That didn’t work well with the motion where a man is running full tilt. You put that on the motion of someone half that size, it looks quite comical. So it’s about where to place these people as well so they do not ruin the shot. It was rewarding, but the time constraints were maddening at the same time. Normally we just have a handful, but every other shot was like this.
The Contending: Final thoughts?
Dennis Yoo: I want to delve into the action of the horde, even though you think it’s just a horde of people running, there’s lots of different things happening. For instance, the first shot that you see of them all running together created some problems because we had thirty base motions of people running, but just a handful of actors. So the problem is, even if you get an actor to run differently, they still have the same essence of that person. When you start seeing them all that clearly you start to realize that that is the same motion and it’s not it’s a different take but it is the same person. I realized we need to get a lot of different people running to avoid that same essence, so that was revealing. Another thing was the horde itself, even though you have them running down the stage, they do not interact with each other. So to have that big of a crowd and that frantic, they would start pushing into each other to have that organic sensation. So we had ongoing layering of motion on top just to get that feeling. Then when we had that horde on the wall piling up and falling off each other, and all the dead bodies on the ground where the infected were running over and tripping on, there were constant different motion challenges.
Nick Epstein: Then you set them on fire!
Dennis Yoo: Yes, so then they’re on fire and kicking up snow. I would get a call saying that snow needed to be disturbed before they entered, because there was already a crowd of people inside Jackson, so we needed a hundred frames of more people running. I would say nobody’s going to see it, and they would respond yeah, but they will see the snow.
Nick Epstein: It goes back to that level of difficulty, where you peel back there are just more levels of difficulty there. It was a journey, but we’re both big fans of The Last of Us, so it was very cool to be involved in the series.
Dennis Yoo: Huge pleasure to be involved in an epic project like this. I also love the fact that they are always pushing us just to make it better.
