Key words

hybridization, issues, motion capture, augmented reality, Studio Anim, SolidTrack, Keymotion, Kanaki, Platige, SolidAnim, Autour de Minuit, 2 Minutes, Another Day of Life 

The conference on hybridization challenges is characteristic of the porosity of sectors, between VFX and animation but also between animation and live action, augmented reality, video gaming, etc. After defining the two avenues that the conference would pursue (tools and practices on the one hand, and contents, on the other), René Broca, conference co-organizer, gave the floor to speakers in the first field, asking them to explain to what extent their innovations are "hybrid".

For Emmanuel Linot, of SolidAnim: "Our tools are hybrids because they provide a different way of broaching motion capture." Isaac Partouche, also from SolidAnim, went on to explain that it provides a "means, when shots are done, to previsualize CG animation that's maneuverable in real time." Last of all, speaking for 2 Minutes, Jean-Michel Spiner stated that he sees hybridization in "products derived from TV series' bases using data emanating from these series."

SolidAnim: mocap and augmented reality

The SolidAnim company, founded in 2007, is specialized in motion capture (mocap) with two studios based in Angoulême and in Ivry-sur-Seine. In 2010 the company decided to start up an R&D department focused on perfecting tools which are flexible and adapted to real time. "The idea is to provide innovative solutions so that directors can have more control, and processes can be simplified," explained Isaac Partouche.

SolidAnim decided to invest in optic mocap – since the system "is the most precise, to within a millimeter" – with 48 Vicon MX40+ cameras for capture of several individuals for not only bodies but also faces and fingers (equally distributed over two capture sites). Another 18 OptiTrack cameras were added on. "Today we have much greater signal reliability but the real question is how far do we want to go in realism? We know how to reproduce identical digital clones, including for ground contact and shoulder movements", said SolidAnim's CEO.

Recently the company worked on Pascal Herold's feature, Cinderella: Once Upon a Time in the West, which is a new take on the old tale, with animal protagonists who act like humans. "On this project", states Emmanuel Linot, "we set up a customized animation pipeline with our Keymotion tool which is a mix between mocap and keyframe animation." The technique mix allies motion capture for dance sequences and keyframe for acting sequences. In spite of this, the film was fully shot in mocap to provide animators with the right references. Emmanuel Linot emphasized that this is not performance capture: "We worked with the final voices; during the shoot the actors did more miming than real acting."

The cast included one very big character and another very small one, and the first reflex was to adapt the mocap with different actors. "In the end, we shot it all with the same small actor, but we developed retargeting software to readjust the sizes." At the same time, the actors wore head mounts to capture their gaze, which was then synchronized with the body movements.

Another piece of innovative software from SolidAnim, SolidTrack is a tool intended for easier access to virtual camerawork. "With SolidTrack we can give directors a way to visualize characters or hybrid objects on a shoot, in augmented reality." The workflow is as follows: starting with CG animation, real-time animation, motion capture or keyframe and via a plug-in MotionBuilder, SolidTrack immediately displays the animation on the camera handler's monitor so that shots can be taken which perfectly match the animation. It's a light rig: an IP Ethernet camera and a fisheye on a camera (35 mm or digital). "We focused on Steadicam cameras for flexible outdoor filming," concludes Isaac Partouche.

What are other R&D development routes for SolidAnim? Today the company is concentrating on two subjects: facial animation with EyeTracker, meaning to capture eye movement without markers and with a single camera; and previsualization on set, for the augmented reality.

In answer to a question of how this R&D is financed, Isaac Partouche explained that providing services "helps pay for 30% of our tool development, and 30% comes from company equity. The rest comes from our submissions to calls for projects."

In answer to a question concerning the possible future marketing of Solid Track, he replied that "For the time being, it's just a service; but between now and the end of 2012, it's not impossible."

2 Minutes on pads and tablets

Being a producer of animated series, the 2 Minutes studio has opened its field of investigations to media hybridization, with the intent of "prolonging the televisual experience" as Jean-Michel Spiner put it. "Although the broadcasters intensively demand this, it turns out to be very hard to finance." 2 Minutes has already done much testing in the field: financed by M6, the CNC, MEDIA and the producers, Chico Chica Boumba is the first series to integrate specific Web and iPad apps. "First of all, there's an augmented reality app. The user positions a sensor within the webcam's field and then children can dance alongside the main characters of the series." In addition, there's also a Guitar Hero-type Web application.

Beginning with the premise that mainstream applications are scantily financed, if at all, 2 Minutes opted for a strategy based on "a roll-out of several properties, independent of production techniques, with reuse of production assets and interfaces adapting easily to different IPs."

Encouraged by these first successful trials, 2 Minutes set up Studio Comics, an app available online at the AppStore or on the Gulli website. Launched in May, there have already been 3,500 downloads, without any specific advertising. The idea here is to compose comic strips, using characters and backgrounds from the TV series. Functions dealing with zoom, and managing layering, saving and printing, plus sending of e-mails all give users a possibility for creating their own stories, starting with characters and environments from the series, without the production having to create any additional items.

Going further in hybridization, Studio Anim enables users to make their own cartoons on Web and iPad. "The idea is to juxtapose existing animation sequences with sound that's integrated in two ways: either by keyboard typing or recording with a mike." The assembling is done by bricks of action/sound. SolidAnim has two levels: beginner and expert. In the first case the camera frame is fixed and the player has two characters at most. The positions are set, the periods of action are pre-established, and each action/sound brick is of the same length. In the second case, the users can manage the movement of the characters via ground-based markers, and they can also tweak a depth map (to make the characters smaller if they are moving towards Z). "There's also the possibility of exclusionary zones in the movement, of managing characters' moods and an option to direct their gaze." Here speech is dissociated from action. Studio Anim's version 1 should be out at the end of 2012, for youngsters 5 to 13. A version corresponding to Androids is planned of course.

In answer to a question concerning purchase of the app, Jean-Michel Spiner suggested that it would be: "free at first. Afterwards, each of the license holders would have to choose. Discussions are ongoing with other producers to acquire the software."

Another Day of Life: hybridization serving History

Spanish-based Kanaki Films is specialized in documentary productions and its director, Raul de la Fuente, was nominated for the Goya 2012 awards for his short film Virgen Negra. The Polish company Platige Image was started in 1997, and this Warsaw studio, with its 150 artists, has won a number of prizes in international festivals including Annecy.

So we contacted Raul so that he would shoot the live action sequences – mainly the survivors' testimony – while Damian Nenow, author of the prize-winning short film Paths of Hate (at the Annecy International Animated Film Festival), would manage the animation portion." The split between animation and live action is about 60/40.

"The technique used for the animation part is motion capture, but it turned out to be very difficult to harmonize that with the stylized animation, especially for the close-up facial shots," explained Jarek Sawko, joking slightly: "For sure, I would have loved to be able to have used the SolidAnim tool when we were in production."

At present, the live shoot of the documentary in Angola is finished. "We are now working on the third draft of the script, and still searching to complete financing. If things go well, production should begin in March 2013 and end in fall 2014, with a budget of 3.25 M€."

Speaking specifically of hybridization, the Platige producers noted that "this mix of techniques might lead one to think that we can contact all of the funding entities in Poland for financing. However, strangely enough, some of them reject us, saying that we're producing animation... and the animation funders judge it to be a documentary! Our only perspective, within our country's political funding structure, is to position it in the historical film category, no matter what the technique."

Autour de Minuit: hybridization in line of sight

Directed by Nicolas Schmerkin, Autour de Minuit is a production and distribution company that has about 40 short films to its credit, including the Oscar-winning Logorama, several music videos and a first series, Babioles, to be broadcast unscrambled on CANAL+ in September 2012. It's really an exemplary series since the characters are in CGI and the backgrounds are live action. "Our intent is to try grafting techniques together. Afterwards we'll see if it gels or not."

For the company's founder, "Hybridization dates back to the dawn of time… movie-making time, with more or less success." If we check carefully, there are plenty of examples. First, there's the dancing duet between Gene Kelly… and Jerry (of Tom and Jerry) that Nicolas Schmerkin counts as one approach. "It's possible to view a film like Mary Poppins as cohabitation. The 1980s denoted a period of confrontation because technology and humanity were waging a historical battle. Proof of this is found in Tron or else Who Framed Roger Rabbit. With the onset of CGI in the 1990s, we could say that the grafting – to continue on with our gardening parallel – has thrived. Success of this is evidenced by The Mask or Terminator. 2000 was the beginning of the decade of fusion, of alloys that impact the short film sphere since this remains a more open and experimental field, less subject to constraint. As a matter of fact, these particular films are often self-produced and homemade, so to speak."

Going back to Autour de Minuit's slate, Nicolas Schmerkin recalled that the first hybrid film that he produced was Obras, in 2003, a poetic journey throughout the unfolding destruction of Barcelona which alternates between 2D, CG, camera mapping, photography, live action shots...

"We wanted things to be more out in the open and less homemade than what I just described. But how could we submit a highly-standardized file for a non-standardized film? In the end, we gambled on sending out an extremely-visual file, over 30 seconds of test, to the regional funding structure, the CNC, everywhere where we might hope to secure financing." And the producer recalled his great surprise when: "We got very good press, and noticed that the various commissions had "overlooked" the absence of scenarios and formatted documents. Obras was nominated for the César awards and, still applying this principle of very visual files and hybrid works, the slate has won about thirty prizes in different festivals worldwide."

The conclusion once again raised the issues: "We do believe that there is a demand in this type of hybrid production that opens up new narrative options, but securing financing remains a problem since – paradoxically – it's specifically this hybridization which worries potential backers."

Drafted by Stéphane Malagnac, Prop'Ose, France
Conferences organized by CITIA
under the editorial direction of René Broca and Christian Jacquemart

Translated by Sheila Adrian

Contact: christellerony@citia.org