On the recurring question of the strange filiation that puts VFX and animation on a par, the answer is provided by a growing number of examples where live action shots are not only confined to visual effects but also include long sequences of animation, or even a large part of the film that contains animated scenes that are equally as good as their live action counterparts. This conference looks over four examples to illustrate this.
The Mill is a British studio founded nearly twenty years ago by brothers Ridley and Tony Scott to work on the visual effects for their own films. The studio has since become a full service structure and is specialised in many fields including advertising. Jordi Bares is the head of this department and chose to present the recent campaign for Orangina, featuring a cast of human-like animals – giraffe, deer, lion, bear, iguana etc. - showing off the benefits of the drink as if it were a shampoo, deodorant or even a spot cream... In all, thirteen ads will be shown across different media around the world. Unlike feature films, one of the main constraints in advertising is the number of decision makers involved in a project. "Between the client, the agency, the director and the artists, it is sometimes difficult to advance," explains Jordi Bares in his introduction. By making this a creative force, he likens commercials to a marathon race where: "You have to be consistent and persevere, always keeping your head up. The key word here is flexibility"
It took two weeks to complete the design of all the characters in the new Orangina campaign. Jordi Bares says that "you shouldn't launch into the 3D too quickly. You need to do a lot of drawings first, advance step by step, get them approved, correct and so on. Creating with ZBrush, for example may give you a sumptuous effect but proves time consuming if the client doesn't like the result. There's nothing better than a pencil and a piece of paper. After all that you can think about the 3D." As a piece of advice he suggests, "Always present a model that suits both the client and the agency but, at the same time, create something that suits you. Wait for the reactions and then present your own creation..." This sort of strategy satisfies everyone.
In this campaign, the shooting was done in motion capture as well as live action to serve as a reference video. "The animatic is also very important for the animators," continues Jordi Bares. "These are the two references they use for studying any unconscious gestures like heavy shoulders, a drooping eyelid or a movement of the hands; every detail counts." Although rendering is the most important stage of the production process, because the client has to approve any eventual modifications, Jordi Bares warns against too much high tech: "too many tools kill the tool. You should never change a detail or an image just for the sake of it and always ask the same question: why do I want to change this? If you can't answer straightaway, then don't bother... Time is precious in advertising and you have to concentrate on priorities." In other words, are you adding value or not?
After this presentation, which included a making of and hints on advertising production, Thomas Duval, VFX Supervisor at Duran Duboi, presented the work done by his different teams on the feature The Round Up, relating the events of the Vél’d’hiv’ Roundup during the Second World War. "We had to create an invisible FX, one that cannot be detected and fits in sealessly with the other shots. Beginning with quite a large handicap (almost nothing to work on) we expected to produce something that was less than realistic. But our initial setback turned out to be an advantage as having no visual references gave us a certain amount of freedom when inserting 10,000 people in the shots filmed with Steadicam. The contribution of animation had to be vital, because of the gravity of the subject matter."
Thomas Duval then went on to give a step by step account of how the studio goes about such an exercise: "After receiving the screenplay, we make up a subjective list of shots we can work on. The director then approves all or part of this list." As mentioned previously, there were very few visual references to reconstruct the "Winter velodrome" cycle track, yet the orders were to be as true to life as possible. "Once the preparations we done, we made up an estimate that was approved by production." Like many studios, Duran Duboi starts working both technically and physically at the shooting stage. "We produce a 3D preview of scenes that include FX, which allows us to prepare for making them, and also gives the director the possibility to work out camera moves and not to feel technically restricted as he knows that everything can be extended digitally if necessary."
The work of Duran Duboi was shared between discoveries and extensions to sets. Thomas Duval explains that, "a quarter of the real set was reconstructed to a scale of 85 %, in a 2,000 m2 studio in Bulgaria with 500 extras." They then had to track the crane back to pan over the full set. To fill out this huge expanse, the studio used the software Massive for crowd management, but without the agents sold with the package as the animators created small simple cycles for the 10,000 digital extras – "there was no need to do more, given the size of the shot." These extras are part of a digital library of characters that the studio clothes and animates depending on the production requirements and the historical periods.
Where the Wild Things Are is a live action film directed by Spike Jonze and adapted from the best-selling children's story by Maurice Sendak that has sold over 20 million copies since it was published in 1963.
Nicolas Scapel, Head of Rigging at London-based studio Framestore, starts off by going over the genesis of the film adaptation of the book, which began as far back as... 1983. "Originally, there was a 2D/3D project that was supposed to be directed by John Lasseter and Glen Keane, but it only remained at the planning stage. Then a full CG project was announced in 2001, which didn't come to anything either. Two years later, a live action feature was finally approved."
Visual performance in this film hangs on the monsters that, if created in CG, would have given greater ease and options of expression and staging. However, Spike Jonze had chosen to rely on life size puppets produced by Jim Henson & Co, creator of the legendary Muppet’s Show. An animatronic system set inside the puppets was supposed to make the facial expressions. After a day of shooting, the system wasn’t working as expected and the actors were finding it hard to cope with, so the decision was made to get rid of the animatronics leaving the director to assume that it would all be done in post-production. "Nevertheless, as everything was shot in 35 mm, with no additional lighting and only basic tracking, this was going to be a tough challenge." The production crew carried out all sorts of tests to animate the monsters’ faces, but it wasn’t till 2008 that Framestore was contacted and agreed to work on the film. The mission was clear: six months of post-production – as the theatre release date had already been planned – six monsters’ faces to digitise and 1,200 FX shots to work on!
Not the easiest of tasks then, to recreate, without any other references but the live action, a whole spectrum of emotions for stuffed animals of over two metres! Framestore decided on the projection mapping solution that generates facial animation in 3D collected from 2D information. "First we model the heads of each monster in 3D then follow on with a basic tracking. We then refine this tracking to get a facial set up to animate the rigging. The fourth step is to extract the textures from the original shot, along with lighting details and then we animate the face, eyes and mouth via specific shaders."
Fur dynamics was also an essential component in this live action film, and Framestore had to create the fur set ups especially. "As fur can’t be animated, we had to get rid of it shot by shot then we reapplied our own 3D hair dynamics to which we applied the original textures." All these procedures were then composited for a final "raw" render, in order to make any necessary adjustments, before blending the different 2D and 3D elements.
To help the animators in their work, the voices of the monsters were recorded with the actors who performed together as well as in different situations from the script like races or doing somersaults so that the video could serve as a reference for the facial animation.
Nicolas Scapel concludes by listing his key ingredients for a successful rigging: "a face with heightened expression, warping based on folds, adjustments to existing models, a good asymmetry and many practise sessions for the facial animation." The recipe for the animation consists of "a chief animator for each character throughout the whole film, video references, maintaining a broad emotional arc, direct feedback from the director for greater reactivity, or better still, his physical presence."
The studio Double Negative, represented by Animation Supervisor, Paul Davies, created 250 of the 1,000 FX shots in Jon Favreau’s Iron Man 2. These include the Monaco Formula 1 Grand Prix, involving 22 vehicles and 100 % CGI, the sequence where Tony Stark dons his red and gold amour which he takes out of a briefcase and the fight with Whiplash.
The studio created a real virtual set of the surrounding town and port of Monaco before working on the circuit itself. The majority of the work for the race consisted in creating a digital version of the riverside zone of the circuit with the main track, the harbour wall, fences and barriers, stands, street lights, a dozen buildings, the headquarters of the race commissioner, crowds of digital extras, a number of yachts and various other vehicles etc. While a small team started work very early on, Double Negative went to Monaco with the director of the second crew just before the Grand Prix in May 2008. They got together a library of HDR and LIDAR pictures, along with reference videos of the whole circuit. The crew then went off to Downey Studios in Los Angeles to construct a 1,500-metre asphalt strip for the battle sequence. "For the race itself, we created 22 CG cars. There were already stationary cars in the shots leading up to the race, but once it gets underway, everything that has wheels goes digital."
Another important moment in this film is the sequence where Tony Stark dons his armour after opening his briefcase and pulling on two gloves that evolve into his all-over body armour suit. "We did a lot of preliminary tests to work out the order of the armour parts, but most of the time we were stuck when we arrived at the gloves. The animation just wasn’t right," explains Paul Davies. "In the end, we completely reversed the process to make the gloves the start off point to Stark’s transformation into Iron Man."
The British studio also worked on the powerful whip effects of Stark’s rival Whiplash, and had to make them look almost organic. The idea was that the whips should contain a plasma core capable of cutting through cars like butter and some of this energy would escape in the form of Tesla coils. To make the animation credible, the crew produced three control levels: "six general controls, six sub-controls for more details and an adjustment layer," explains Paul Davies.
Motion capture was used by Double Negative for the fighting sequences between Iron Man and Whiplash, played by Mickey Rourke, who sometimes found himself on the set in front of an actor covered in sensors. "Mickey was even alone on the set, when he makes Iron Man fly over him for example." However, this was not kept for the animation. A choice that was therefore governed by artistic rather than technical reasons – "It allowed us to have more realistic scenes, but we chose to transcribe everything in keyframe animation, which seemed more appropriate for this type of character."
Thomas Duval goes on to point out that "this combination of techniques helps us towards a more efficient workflow." Nicolas Scapel also confirms "the relevance of such an approach," regarding the work done on Where the Wild Things Are. Finally, Jordi Bares concludes with some tips for any sort of production "You have to be able to listen to your client; listen to the silences to advance one single step. And, equally important, you need to know how to coordinate your crew." As demonstrated here by these four examples.