The science of Shrek

A lot of science goes into making a computer animations like Shrek or effects-heavy feature films, such as the remake of The Poseidon Adventure or Pirates of the Caribbean.

This science is something I'm interested in these days: Why does skin look like skin, hair look like hair and fabric look like fabric? There is a traditional physics approach to this which involves measuring stuff with complicated looking equipment and drawing graphs. This is part of the job, but another important component is the work of people interested in photorealistic computer graphics.

If you think computer graphics is all computer games and obvious stuff like Shrek, then check out Autodesk's Fake or Foto test. Good photorealistic computer graphics are really good these days. Update: Actually, just watch this, full-screen.

There are three steps to making an image, or an animation, using photorealistic computer graphics:

Firstly, make a model of the scene in the computer, this should include the location and type of camera and lights as wells as the shape of objects in the scene. The shape is defined in terms of a "polygon mesh", this is basically a fishing net in the shape of the thing you're interested in. Usually "polygon" means triangle in this context. If you're doing an animation then you can automate some of the scene generation by running physical simulations of objects in the scene. That's to say, if you want a bouncing ball, you don't have to make a set of scenes by hand with each scene showing the ball in a slightly different location - you can do this automatically. There is academic research here because efficiently simulating the motion of more complicated things like liquids, or 3601 plastic chairs, is really hard.

Secondly, give the objects in the scene optical properties like colour, transparency, shininess etc. A key feature of the optical properties is the way a material reflects light. This is the difference between a piece of chalk and a mirror, both reflect pretty much all the light that falls on them but if you shine a spotlight on a piece of chalk then that light ends up all over the place, whilst with a mirror the light all leaves in one direction. There is an interesting compromise to be made here, chalk scatters light as it does because of its rough surface. Now this can either be handled by making a very detailed model of that surface roughness (lots of triangles) and then treat each triangle as a little mirror, or I can make a very simple model of the surface and just say "If light hits here it can be scattered anywhere". This is another area of academic research: how do I efficiently model a complicated material like fabric, because I really don't want to put in every single fibre in a piece of fabric?

Thirdly, simulate light in the scene: make light come out of the lights and follow it as it bounces through the scene (if it doesn't hit a camera at the end of it's journey, it doesn't count!). This final stage is known as 'rendering' or ray tracing. Ray tracing because in this model the photons travel in straight lines, "rays", between collisions with objects in the scene. This is a laborious process, at best a photon (a little bit of light) can contribute a fraction of one pixel in the final image and worse, when you fire the photon out of your virtual light you don't know whether it will even hit the virtual camera (which is the only way you're going to see it). Repeat the firing of photons out of the lights in all different directions, many times in order to build up an image, the more photons you fire the sharper, less noisy your final image will look. In order to build up a reasonable size, reasonable quality image you will need millions of virtual photons. For photorealistic rendering it can take hours to render a single image. There is academic research in trying to do this more efficiently.

Skin is another tricky material to model, the problem is that it's a little bit transparent: if you simply bounce light off the surface it ends up looking a bit odd. However, if you let light leak into the skin a bit, then things look much better:

The trick is to do this in a computationally efficient way, in fact this work won the Oscar for Technical Merit 2003 and ended up being used on Gollum in Lord of the Rings.

I went to a presentation, at SIGGRAPH (an enormous computer graphics conference) on the special effects for The Poseidon Adventure. The fluid dynamics simulation in this film are fantastic, when the ship sinks waves sloosh around impressively, but it looks a bit odd towards the end. It turns out this is because in the simulation they turned down gravity to make the waves bigger, and then hand paint on a splash right at end, as the Poseidon sinks below the waters. In the end this science serves what is ultimately an artistic, and a commercial, endeavour.

You can play with this sort of stuff for free, Blender is a very fine open source 3D design program used to build scenes (interface takes some getting used to) and luxrender is a physically based render, it's based on the code described in the book: Physically-based rendering.

p.s. If you what to know how donkeys can talk, I don't know.