The document discusses the workflow, rendering, and shading techniques used for the Butterfly Effect, a real-time animated short created through a collaboration between Unity Technologies, Passion Pictures, and NVIDIA. It describes how the project involved over 30GB of source assets organized across 137 shots. The pipeline connected Passion Pictures' existing workflow to Unity's editor to minimize changes for artists. Techniques like point caching, tessellation, mental ray shaders, indirect illumination, shadows, ambient occlusion, and depth of field were used to achieve photorealistic rendering in real-time.

1. Butterfly Effect:
Workflow, Rendering & Shading
Renaldas Zioma
Unity Technologies
@__ReJ__
Monday, April 29, 13

2. Unite template2
What is it?
Real-time animated short
Collaboration between Unity, Passion Pictures and NVIDIA
Monday, April 29, 13

3. Unite template
Opportunity to Learn
• Passion Pictures
• New world of RealTime
• No prior experience with Unity
• Unity Technologies
• CG Pipeline
• Huge amounts of Data
3
Monday, April 29, 13

4. Unite template
Project Outline
• 30GB of source Assets
• 5GB of runtime Data
• 137 shots
• 40 “scenes” in Editor
4
Monday, April 29, 13

5. Unite template
Project Organization
• Team of ~10 - first month
• pre production
• research / tools
• Team of ~30 - next 2 months
• content
• tools / engine modifications
• Team of 5 - last 3.5 months
• polish
• optimizations
5
Monday, April 29, 13

6. Unite template6
Monday, April 29, 13

7. Unite template7
Monday, April 29, 13

8. Unite template
Previz
• One Sequence
• Goals
• Initial timings
• Visualize Space
• Prime lenses
• 10, 14, 18, 21, 24, 27, 35, 40, 50, 75, 100, 200
• Zoom lens
• 18 to 150
8
Monday, April 29, 13

9. Unite template
Pipeline
• Connect Passion pipeline with Unity Editor
• Main Goal
Minimal changes to Artist workflow!
9
Monday, April 29, 13

10. Unite template
Passion Pipeline
• No Change for Artists
• Use Existing Asset Pipeline
• Sequences SB, BU
• Shots SB_01_00, SB_02_00
• Assets [Character/Camera/Layout]
• Export Models and Publish Shots as normal
• Add Unity support
10
Monday, April 29, 13

11. Unite template
Character Rig
• XSI Softimage Rig
• Bones
• Envelope
• Face Shapes
• Controls
• Unity Rig
• Bones
• Blend Shapes
• Materials
11
Monday, April 29, 13

12. Unite template
Facial Rig
• Bones
• Head, neck, jaw, eyes
• Blend shapes
• Face
• Mocap shoot
• Face markers
• Cubic motion
• Retarget facial animation onto XSI controls
12
Monday, April 29, 13

13. Unite template13
Monday, April 29, 13

14. Unite template14
Monday, April 29, 13

15. Unite template
Non-linear time
• Treat Sequences as Live Shoot
• Cover the action from different angles
• Each sequence needs multiple cameras
15
Monday, April 29, 13

16. Unite template
Non-linear time
• Shot Description
• Final Cut edit
• Export EDL as XML file
• Camera in/out points
• Timeline in Unity
• Sequence Preview in RV
• Using EDL from Final Cut
• Play whole preview sequence in RV
• Instant Feedback for Animators
16
Monday, April 29, 13

17. Unite template
Timeline in Unity
17
Monday, April 29, 13

18. Unite template
Timeline in Unity
18
Monday, April 29, 13

19. Unite template
Non-linear Challenges
• Animation
• Particles
• Physics
• Rigid Body simulation
19
Monday, April 29, 13

20. Unite template
Point Cache
• Per-vertex Deformations
• Cloth
• Hair
20
Monday, April 29, 13

21. Unite template21
Point Cache
Monday, April 29, 13

22. Unite template
Tessellation
• CatmullClark
• Now: on CPU
• Future: OpenSubdiv library
22
Monday, April 29, 13

23. Unite template
Hair
• Guide “strands” for modeling
• GPU Tesselation and “Amplification”
•
23
Monday, April 29, 13

24. Unite template24
Hair
Monday, April 29, 13

25. Unite template25
MentalRay Architectural
(MIA) Shaders
• Familiar
• Small set of parameters
• Physically Based
• Micro facet
• Energy conserving
Monday, April 29, 13

26. Unite template26
1 parameter for Diffuse
Monday, April 29, 13

27. Unite template27
2 parameters for Specular
Monday, April 29, 13

28. Unite template28
2 parameters for Reflectivity
Monday, April 29, 13

29. Unite template
MIA Implementation
• Oren-Nayar
• Diffuse
• Cook-Torrance
• Specular
• Geometric + Roughness + Fresnel term
• Ward distribution for Roughness
• I think... ;)
• Direct + Indirect illumination
29
Monday, April 29, 13

30. Unite template
Unity Implementation
• Oren-Nayar
• very expensive
• OK approximations
• Cook-Torrance
• Normalized Blinn-Phong instead of Ward
• Roughness to Phong exponent (n):
• n = 2 / roughness2 - 2
• Tweaked to match MIA look:
• n = 2 / roughness4 - 2
• Schlick approximation for Fresnel
• Only direct illumination at RunTime
30
Monday, April 29, 13

31. Unite template31
MentalRay Unity
Monday, April 29, 13

32. Unite template32
Monday, April 29, 13

33. Unite template33
Indirect Illumination (GI)
• Point Clouds for Dynamic Geometry
• Spherical Harmonics for Diffuse Bounce
• Cube Textures for Reflections
Monday, April 29, 13

34. Unite template34
Indirect Illumination (GI)
• Textures for Static Geometry
• Diffuse Bounce
• AO
Monday, April 29, 13

35. Unite template35
Indirect Illumination (GI)
• Calculated offline with Autodesk Beast
• Path Tracer + Final Gather
• IBL for Sky
• Fake Caustics
Monday, April 29, 13

36. Unite template36
Real Time Shadows
Monday, April 29, 13

37. Unite template37
Baked Shadows
• Of course baked only for static environments
Monday, April 29, 13

38. Unite template38
Ambient Occlusion
• HBAO - Screen Space
Monday, April 29, 13

39. Unite template39
Ambient Occlusion
• Too Dark & Dirty
Monday, April 29, 13

40. Unite template40
Ambient Occlusion
• Applied ONLY to Indirect Illumination
• Colored to match Sky better
Monday, April 29, 13

41. Unite template
Skin
• Texture Space Diffusion
• 3 Layers
41
Monday, April 29, 13

42. Unite template
Back Scattering
• Translucency Map
• Adds intensity to sub-
dermal layer
42
Monday, April 29, 13

43. Unite template
R
43
2 Specular lobes
Monday, April 29, 13

44. Unite template
R
44
Monday, April 29, 13

45. Unite template45
Depth Of Field
• Antialiasing aware
• FXAA based
• With Splatted Bokeh Texture
Monday, April 29, 13

46. Unite template46
Monday, April 29, 13

47. Unite template
Monday, April 29, 13

48. Unite template48
Thank you!
Monday, April 29, 13

49. Unite template
References
49
 [Bavoil08] Image-Space Horizon-Based Ambient Occlusion. L. Bavoil, M. Sainz, 2008
 [Catmull78] Recursively generated B-spline surfaces on arbitrary topological meshes. E. Catmull, J.
Clark, 1978
 [CookTorrance82] A Reflectance Graphics Model for Computer graphic. R. L. Cook, K. E. Torrance,
1982
 [Hoffman10] Crafting Physically Motivated Shading Models for Game Development. N. Hoffman,
2010
 [Lottes09] FXAA. T. Lottes, 2009
 [Mental07] mental ray Architectural and Design Visualization Shader Library. mental images GmbH,
2007
 http://www.mentalimages.com/fileadmin/user_upload/PDF/arch_and_design.pdf
 [Mikkelsen09] Microfacet Based Bidirectional Reflectance Distribution Function. M. Mikkelsen, 2009
 [Nießner12] Feature-Adaptive GPU Rendering of Catmull-Clark Subdivision Surfaces. M. Nießner,
C. Loop, M. Meyer, T. Derose, 2012
 [Olano10] LEAN Mapping. M. Olano, D. Baker, 2010
 [Schüler11] The Blinn-Phong Normalization Zoo. C. Schüler, 2011
 http://www.thetenthplanet.de/archives/255
 [Zhukov98] An ambient light illumination model. S. Zhukov, A. Inoes, G. Kronin, 1998
Monday, April 29, 13