The effort used about 50 motion capture cameras and eight standard high definition cameras. So many were needed, Letteri explained, because, unlike the previous movie in the series, Rise of the Planet of the Apes, in which most of the scenes show apes individually caged, the apes in this movie are running all over the place, in a visually crowded environment. With multiple actors in action running between trees, markers are frequently blocked from any one camera’s view; 50 cameras meant that no matter what happened at least one camera would be tracking most markers.
The crew set some of the cameras on the ground, camouflaged by moss, put others on 5-meter tall aluminum towers that could each hold several cameras, put single cameras on poles attached to flat bases in places where they needed to blend into their surroundings, and, in some cases, simply tied the cameras directly to tree limbs. They camouflaged them well, said Momcilovic, and in post production carefully looked to remove any evidence of the array of cameras but, he said, it’s likely that an audience member watching the film closely will be able to spot the occasional camera lens peaking out of leaves or moss.
The set up had to be assembled for each day’s shoot and calibrated by a crewmember walking the scene carrying a rod with markers to make sure multiple cameras could spot each marker at every place an actor might go. A good gust of wind, or someone bumping into a camera pole, could disturb the position of a camera enough in order to require recalibration. And the whole setup had to be packed away at the end of the day, even if the crew was planning to come back to the same spot, it couldn’t sit out in the damp forest.
The cameras had to be protected from the weather. And they had, for the first time, to communicate with the local data server wirelessly, adding to the complexity of getting this whole setup to work reliably. The team from Weta designed the outdoor housings and the Wi-Fi add-ons themselves. And it was a crunch. The camera cases, eventually manufactured in Canada, were still in prototype form when the crew arrived on location.
In spite of the special cases, a lot of the equipment had to spend each night surrounded by silica gel in order to dry out. And keeping everything going required the efforts of a software developer, Glenn Anderson, who also knew a lot about electronic.
“He was fixing stuff full-time,” Momcilovic recalls, and along with programming changes in the software, using a custom web site that pushed out updates to the array of cameras while they were in use. (Anderson carries a certain amount of tech-cred outside the computer graphics world: he developed the Eudora Mail Internet Server back in 1993.)
Each actor wore 48 markers on his or her body; faces, filmed with head-mounted cameras, were dabbed with spots of white paint. Because of the lack of control over the lighting in an outdoor environment, the Weta team couldn’t use standard reflective markers. They had previously developed infrared LED markers for a brief outdoor scene in the previous Planet of the Apes movie; these, however, weren’t reliable enough to use for extended filming. So, Momcilovic says, they molded them strands of translucent plastic, strong enough to take a direct hit from a hammer without damage to the LEDs inside. The strands, attached to the actors with Velcro, could be controlled remotely, turned on or off or made brighter to deal with changing light conditions.
As many as 13 actors wearing these active LED markers appeared in a scene at one time. The processors in the cameras, after analyzing the video on the fly to extract the marker locations, transmitted the data to a server on site. As soon as the crew brought the server out of the forest and onto the Internet, it sent all the motion capture data to New Zealand, where Weta’s team of animators could immediately start the process of using it to create digital apes.
Though the weeks shooting in the Canadian forest presented the biggest technical challenge up front, the toughest scene for the computers to process afterwards turned out to be a fully computer-graphic industrial environment, set outside of New Orleans.
“There’s a big ending sequence,” says Letteri, “with a large number of apes on a half-built skyscraper, at night, with lots of lights throughout. Everything you see in the scene is CG — creating the fur, the muscles, and the facial simulation in that environment took an intense level of rendering,” about 550 core-processor-hours per frame. The team, of course, used banks of computers with multiple cores, typically 200 to 1000 processors would be in use at a time, though sometimes as many as 50,000 ran simultaneously. Was it worth it? I intend to find out today.
Get more from IEEE Spectrum
This article originally appeared on IEEE Spectrum; all rights reserved.