Brief:
You will be expected to demonstrate the use of combining a number of digital formats including digital video, audio and animation and show the competent use of combining a variety of media including the possible use of CGI, motion graphics, title design, film, stop motion animation. You will be expected to demonstrate thorough research activity and show evidence of this by way of delivering a proposal.
The brief is fairly broad with not a lot of restrictions, which gives me a lot of room creatively. We have been looking at Compositing in lesson and I have experimented with 3D Motion tracking and compositing prior to coming to uni but it is not something that I touched upon since then. I used to use Boujou Match moving software to motion track as Cinema4D did not have an inbuilt Motion tracking system, which was very laborious, moving data back and forth between multiple programs. However with a recent update to Cinema4D, release 17 saw an inbuilt motion tracking system which has made it simpler to motion track footage.
Research
For research I used the two tutorials to learn a deeper understanding on how the new motion tracking system works in C4D R19. I looked at 2d and 3d solves and finally scene reconstruction.
My idea was to originally incorporate water simulation in the project as it is something that I have researched and studied further throughout this module. I wanted to have a motion tracked scene with fluid interactions happening in a real life environment. Here is a mock up of the idea.
Footage
I gathered all the footage myself, using locations such as the uni campus as it has beautiful water features and interesting architecture. Some footage was gathered from outside of uni in supermarkets All the footage was filmed on my Canon 70D with a 18-135mm lens.
These are stills from all the video footage I recorded
Initial Design
For one of my scenes I wanted to include a maelstrom storm into the river in my uni’s campus. I had made a maelstrom water effect before but I wanted it larger scale with more detail. 
But after creating, simulating and meshing the maelstrom effect I realised that I had to take time management into account. Creating this whole sequence which lasted roughly 8 seconds took me 3 days solid work, including time simulating and meshing the sequence. The deadline was still a few weeks away, however to make a whole 2 minute animation based around water simulation it would not be enough time. So I decided to not to use water simulation in my scenes so it would speed up my workflow.
Its a crude mock up render but it demonstrates my initial idea for what I wanted to do for the project. I have experimented with water simulations before and also motion tracking before but combining the two is something new that I wanted to pursue however the lengthy simulating and meshing times means that I wouldn’t be able to finish the project in time.
Stair sequence
After now knowing not to follow up with water simulation effects, I took a different turn and used the stair footage to motion track different coloured spheres tumbling down them.
Firstly, I tracked the footage using c4d’s 2D solving system which tracks the relationship and distance from each point by looking at the parallax in the footage. I then ran a 3D solve which assigns a position value to each point.
When filming, I took rough measurements of the stairs for when I would have to build it in 3D. I needed to build the stairs accurately so the object would have something to interact with.
As you can see by this image that the result is close to the original. I need to add sides to the stairs so it would mask the objects when against the left side of the wall
The left image is without masking, the right is with masking. I had to also build a cylinder shape which acted as the hand rail so the spheres were masked behind that also.
For lighting the scene accurately, I knew I needed to use HDRI maps as it is the most accurate way to light an environment.
I took photos of the sky at the time, which was overcast with no sun so therefore no hard shadows on the floor. This is a good and a bad thing, bad in the sense that I didn’t have the opportunity to use hard shadows on the 3D objects which would make the scene look more realistic and a good thing not having to make the shadows and get away with just using ambient occlusion. It sped up work flow but I missed a chance to make the sequences more realistic. I took the photos with the goal to make a custom HDRI but not realising that I would need more image to make a sphere and it would be quite labour intensive. Instead I used a HDRI pack from Greyscalegorrila titled Hilton Roof Top which was an outdoor shot of an overcast day similar to the one in my footage.
Rendered with the physical render, it adds accurate motion blur and reflections to the objects
Duck Sequence
For this scene I needed to do a few things, firstly model the perimeter of the water feature so the 3D objects accurately interact within the scene. Secondly, create a water flow for the objects to move down. There is a few ways I could do that, with Realflow, creating an actual flow of water that interacts with the objects creating buoyancy and move accurately and physically. Or just use a wind object to gently blow the objects along a flat plane, faking a water flow. This will be a lot faster to create but look less realistic than the other option. But because of deadlines I will be using the wind option.
To get an accurate measurement of the dimensions, I went into google earth, located the water feature and then pen tooled it. This was really my only way other than measuring each sides length with tape when I was at the location. This method is a lot faster and doesn’t require me to do any onsite planning but the scale wouldn’t be the same which isn’t that big of a deal anyway.
For example, the real water feature would be about 15mx3m, whereas when pen tooling and put into Cinema4D it was set to 150cmx30cm. For continuity shots and for being as accurate as possible you would want to chose the more accurate method but for my purpose it doesn’t make a huge difference.
Here shows the pen tooled spline, extruded and positioned to fit the composition. I used the footage as a texture, set to frontal projection and a compositing tag with “composite background” checked. This assigns the extruded object a texture that blends perfectly with the background footage.
I used a rubber duck for a comedic take for the scenes. I got the rubber duck model from here. I needed to make some adjustments on the model as it didn’t sit on a flat surface. I went into the sculpt system and smoothed the underside of the duck till it could sit flat on a plane.
Here are the ducks positioned with the water features boundaries positioned and the wind effector on the left side shown in yellow pushing the ducks down the flow.
In these images you can see reflection in the water mirroring from the ducks. This adds a big layer of realism to the animation with the use of ambient occlusion shadows and water reflection. I achieved this effect by duplicating the footage texture and adding reflectence with a bit of blurriness/roughness so it is more subtle.
One issue that I had with the flow of ducks is that they kept getting stuck in the corners as they travelled down. A way I tackled this is by creating triangle shaped blockers (highlighted in red) for the corners which were set to invisible and had a collision tag so the ducks would bounce of them but so it wouldn’t be visible by the camera.
In these 2 images, It shows a secondary camera angle for the sequence. In image 1 it shows the ducks fallen over resting on the plane but that is not how an object on water would normally interact. It would normally be partially submerged, however as i’m not using simulated water I cant create object submersion. A work around that I came up with is duplicating the floor/plane objects, deleting the collision tag and moving the floor/plane up along the Y axis a 10 or so cm so it creates the illusion of depth. The second image shows the ducks sitting below the water surface. This also adds more realism to the animation.
This is the sequence of footage that the duck animation will follow.
I did originally want a 5th scene which puts the ducks into the lake but there was an issue with tracking that shot. So I decided not to use this footage and instead end the duck sequence after 4 scenes.
Shopping Cart Scene
All the other scenes are very repetitive in how their made so I don’t need to document them. The last notable scene is the shopping cart scene. It uses a new system in Cinema4D R19 called Scene Reconstruction. 
This shot shows all the tracked points on the cart. The footage of the cart is a 360 view of the cart from all angles so the software picks up all the points. This is important for rebuilding the scene.
In this image it shows point cloud data from the stored tracked points. Interestingly it looks at the lighting and texture data from the footage and saves it and assigns it the the points.
In this image it shows the point cloud data turned into polygons. It is quite rough as photogrammetry is still in development as its a new technology. It also depends on the camera I used, the amount of tracking points, the weather that day.. etc.
Here is the same polygons but with a smoother deformer to get rid of some of the rough edges. In this image you can clearly see how the texture is applied to the polygons. One of the uses of scene reconstruction could be needing a model for dynamic collisions or with better equipment, being used as scanning real life objects into 3D models.
Final Video
Reflection
During my time time on this project I have deeply explored many techniques such as motion tracking, scene reconstruction and compositing. I have touched on basic compositing in previous years and my prior knowledge helped me during this project when it came to texturing, compositing and lighting the scenes appropriately.
In the final video, some scenes were stronger than others, for example all the duck scenes worked really well as they followed some sort of narrative and all had a purpose in the setting that they’re in. However, the tooth sequence in the shop and the floor-stair scene are my weakest in the project. I was having difficulty getting a clean track when solving the footage which let to the 3D objects sliding making it look out of place and thus making it unrealistic.
I talked about not wanting to go down the path of using water simulation for this project as it is very time consuming having to simulate and mesh everything, however, I feel I should have attempted to create one as it would give variety to the animation. So if I time managed better water simulation would be something I would include.
Overall I am pleased with how realistic some of the scenes look. The duck scene in particular I took the most time on and I feel it really shows. The reflection in the water and water depth add a subtle but much needed level of realism that adds to the photo realistic goal of the final animation.
DSpall Animation Blog 