Launching

Imagined Environments

Visual effects and digital compositing are well known and vastly used across the digital arts industry.
Georges Méliès's 1902 short film 'A Trip to the Moon' is considered by many to be the first film with visual and special effects. Méliès developed a special effects style that involved stop-motion animation; double exposures; split screen; cross-dissolves; and jump cuts to integrate images from multiple sources into a single seamless whole. The most impressive part of his process is producing the final result with all integrated footage, an image that doesn’t betray that its creation is by multiple source elements.
The role of this project is how to tell a story about something that doesn’t exist – the key thing is you have to make it as convincing as possible.

Research

Forced Perspective?

Miniature model set?

Fame Focus - Lord of The Rings

I started some of my research by looking into some of my favourite films that rely heavily on VFX, one of my first thoughts was Lord of The Rings. In this video by Youtuber Fame Focus, he breaks down exactly what effects were used and how they implemented these on set.

I knew there was a lot of effort in the filming of these but I didn't know that in most scenes they actually had to build two sets. Proportion and scale were key factors in the filming so to give a look of some characters being much larger, props and objects were significantly scaled down. This wasn't the only thing scaled down, the director was actually a keen miniatures enthusiast so a lot of the grand mountainous creations were in fact table top size. They had multiple miniatures of towns, to forests, to the scene with all of the skulls cascading down the walls and then to pull it all together, angled cameras and applied effects post processing gave the shots a sense of realism. I like seeing the behind the scenes of the forced camera perspective as this involved a more physical realtime effect. By adjusting the real surroundings you can create illusions.

Physically positioning the camera to help get desired shots e.g the 360 degree camera ring for the bullet shot.

Fame Focus - The Matrix

I found 'The Matrix' to be a really interesting choice for VFX, not only because it was a groundbreaking film of its time but also because of the camera ingenuity used on set. I was looking for information on the bullet effect and it turns out that a huge amount of labour went into getting that shot, no wonder they used it like 4 times. A 3D model of the scene was first established and textures were taken from images of the real scene. A 360 degree camera rig was set up with green screen and in order for the speed of filming to increase the physical cameras had to be spaced further apart, to get a slower shot the cameras were placed closer together. The shot was a bit choppy since it had to be compiled into one shot from all these cameras. Through playing with shutter speed and adding or manipulating individual frames post production, the final shot was made considerably smoother. I liked this example of having to think about the physical act of filming to create the effect rather than simply creating the effect post production.

John C. Gaeta

John Gaeta is the co-founder and Executive Creative Director of ILMxLAB, Lucasfilm’s Immersive Entertainment Division. He worked on the Academy Award-winning visual effects for the Matrix trilogy. Innovations in film such as "Bullet Time", "Virtual Cinematography" and "Volumetric Capture” played a huge part in shaping modern cinematography. Today, he is working with the Lucasfilm Star Wars Story Group and Disney. I watched a video interview of Gaeta at the 2018 international VIEW Conference premiere in Italy on Computer Graphics, Interactive Techniques, Digital Cinema, 2D/3D Animation, VR and AR, Gaming and VFX.

He spoke about the art of using visual effects for storytelling, how we as artists have vast amount of source material we absorb from our own real environments and how we can use visual effects to take these ideas and visualise them in ways we cannot physically create. Every writer can use VFX to enhance.

'A Trip to The Moon' by Georges Méliès

The first considered film to integrate VFX, 'A Trip to The Moon' by Georges Méliès tells a tale of a group of explorers setting off to the moon to see what life is on another planet. The film has no sound and uses primarily layering and compositing to apply visual effects. Through double exposure shooting and layering footage over footage, he created new video timelines to explore his space narrative. As well as physical real world props and artistry the film tells the story of the adventure so well that this film is still one of the most famous in film history.

'The War of World's'
directed by Byron Haskin

This example from our readings list was quite fun, It was filmed in a time that effects had to be more of a laborious physical task so in moments like the vignette binocular view I found to really add to the story telling. It is a much more basic form of effects in comparison to the likes of a Marvel movie now, but I would argue these effects are just as effective if not more in terms of cinematic story telling.

Amazing use of colour on the Netflix show. Completely engulfs the viewer and makes the episode itself interactive purely through light and colour.

Olafur Eliasson

Olafur Elliasson is a designer based in Berlin who creates art installations that warp the viewers concept of reality. Through bending light, playing with colour and forcing the eye to follow a constructed path, he creates an alternate reality within your real reality. He grew up in Denmark and visited his father in Iceland, he says he found himself outdoors a lot and being in such natural wonders opened his mind to using natural elements in his installations. These environmental factors drew him particularly to water, which in turn led to the creation of his indoor rainbow. "What is truth? Our reality is made as we move through space. I wouldn't say manipulated. I would say constructed." By using water mist in a blackout room then isolating the mist with a spot light the viewer was enticed to enter. Upon entering and standing in a specific spot so in turn your eye catches the light, you can see a rainbow in the water mist. This is helped by colour filters over the spotlight.

I know this example isn't entirely to do with VFX but I again liked the physicality of Elliasson's work, tricking the eye but with real tangible physical components like mirrors.

He did an episode on the Netflix series "Abstract: The Art of Design" explaining some of his work. The episode itself is almost like an installation for him, he speaks to the viewer in such a conversational manner it's like he's in the room with you. The opening to the episode he encourages you to turn off your lights and focus on the screen, he then places coloured paper over the camera to create a coloured screen. It illuminates your room and you feel as if you are a part of the episode. It is amazing how much effect colour can have on what the eye takes in.

So what are visual effects?

CGI - Computer generated imagery could be the most widely used form of VFX. CGI is made completely with visual effects software and doesn't have any real-world footage. Production companies like Pixar have paved the way for producing cinema that is completely computer generated, like the Toy Story franchise. CGI can create whole films but it can also be used to create backgrounds or characters in real world scenes. Marvel is a great example of how CGI elements are composited into the final video cut.

A scene from the BBC series 'Merlin' with the fully CGI generated dragon character.

Motion capture - Motion capture, or mocap as otherwise known, is a technique that attempts to take the authenticity of a live action performance and use this to in turn create a more realistic digital sequence. This could mean that actors would perform their scenes on a mocap stage while wearing mocap suits that are covered in tiny dots that act as anchor points. The advanced camera systems record the movements of the dots on the suit to turn it into data. VFX artists can use this data to generate believable digital characters based on the real actors movements. A good example of this is Avatar, when the actors switched to the world of Pandora and morphed into their Avatar the cast had to wear these suits in order to create the digital versions of the Avatar's.

Behind the scenes of the filming of Avatar. They had also added motion tracking dots to the actors face so they could map more life like expressions to the Avatars.

Compositing - Compositing is the process of combining multiple images or footage into one final sequence. One of the most common compositing techniques is filming with green screens. The actors film their parts in costume with any real physical props then have a green screen behind them. In editing, the green screen is keyed out, this allows for the background, effects, and additional characters to be added in with VFX software.

Behind the scenes of the filming of Avatar. They had also added motion tracking dots to the actors face so they could map more life like expressions to the Avatars.

VFX
vs
SFX

One of the main differences between visual effects and special effects is the computational aspect. Visual effects are made using a computer and special effects are made in real life through make up etc.

Through visual effects you can really create any background you want. A raining scene? Blazing fires? Whereas SFX would need to rely on actually creating these elements on set.

Visual effects also give an element of control back to the designer or director, effects can be manipulated to follow a specific path but special effects the director is relying on it working on the day to give the desired effect.

I could, of course, use VFX and SFX in the same shot however for this project I'll be sticking to screen based editing.

Sketchbook

My Idea

After looking at the media on the overview and doing some digging into VFX and its capabilities I'm looking to draw inspiration from my daily journey and make it into a fantastical underground scene. I'm being a bit tongue in cheek by calling this 'A Trip To Cowcaddens', i'm thinking of this as a remake of 'A Trip To The Moon'.
I want some very dull, mundane shots of Eve entering at Kelvinbridge and leaving the subway at Cowcaddens but to have the journey in between transport her to a weird underground scene. I want her to be alone on the train and not have a clue what's going on and then for the ending to bring her back to reality when the train in fact pulls up in Cowcaddens.

Media

@Tunnel Vizion

I looked at some VFX shots that were specifically involving trains, high movement and masking/tracking. I came across this wee film 'Man vs Train' by youtuber and vfx artist @tunnelvizion. Thought it looked very realistic and I liked the movement across the shots.

Derek Jarman 'The Garden'

A film shot on Super 8 then manipulated onto 35mm film. A brilliant sequence that I found to be mesmerising. It explores an intimate insight into Jarman's life and his studio at the time. It is said that an over arching theme of this piece is vulnerability and exploration into sexuality. Nothing to do with trains but I appreciated the way in which this piece was physically filmed as well as the sequential composition.

Georges Melies
'A Trip To The Moon'

Considered to be one of the first and most iconic VFX examples is French artist Melies' short film 'A Trip To The Moon'. The piece uses compositing techniques with masks and large painted set designs. I loved the whole grandness of the trip with the huge group shots in the beginning then the actors setting off in this tiny shiny shuttle. The shots of the moon are also very fun to watch, it is very well done but at the same time you know its fake, I quite like that about some vfx examples.

'Russian Doll' Natasha Lyonne

Netflix series Russian Doll has a season in which the main character goes into the New York subway and somehow gets transported back in time to the 1980's while on the train. I loved this series, from the actual story line to the moodiness of the shots to the kind of camera angles used specifically the POV shots.

James Mackay - The Tate Archives

I really loved all of Jarman's films on Super 8 so I found this documentary released by the Tate interviewing James Mackay, the man who helped develop the video library of Jarman's films. He dives into individual films and talks generally about how Jarman's painting practice was the driver in his creative his film practice and a large part of the content which he would capture.

This was a book published by Martin Frey which explored Jarman's collection of home movies more in depth and from a critical stance. He examines Jarman's film techniques and looks at his source of influences ranging from literature to fellow artists like Robert Rauschenberg and David Hockney.

The VFX behind 'The Green Knight'

I love the scene of the giants here and the contrast of scale, it really adds to the overall vfx effect. The use of natural elements created through vfx I think is quite interesting and goes back to my previous research of vfx vs sfx. In this example you can see why vfx is not only better for the overall effect but it is also safer since you can really burn an actors head.

@oliverkingvfx

Really liking the mix between real world and fantasy here. I like that theres clear transitions between normal life and then this underwater landscape. This of course looks very realistic which at this moment I know I'm not able to replicate something of this calibre but the underwater world has definitely got me thinking about what footage I could use to mask the windows. I know in these videos you can't hear the sound but I feel this only adds to the magical effect created by the nautical sea life.

Process

Compiling footage in Premiere.

I spilt the clips later into sub sequences and exported individually to After Effects after editing the footage in Premiere. This made it considerably easier for After Effects to handle and for me to later compile in the final film because all I had to do was replace the original footage with the new sub clip export post edit from After Effects.

Colour Grading

Using the Lumetri Colour suite I adjusted every clip as well as adding some basic correction. I could have added an adjustment layer but because I wanted the clips on underground slightly different to the station shots, I thought it better to individually adjust. I wanted a green/blue tint in the station and a more pink/blue when she was underground. I liked this colour contrast between normal and underwater.

Creating A Soundtrack In Logic Pro X.

I used a drum loop for the base layer then using my Akai MPK midi board I sampled sounds of synths and soundscape effects and applied these to my midi pads. I created a beat in line with the track so certain audio effects would line up with what was happening visually.

Sub Sequences

This is what a sub clip would like before export.

Tracking In Mocha

I first started with tracking in Mocha using the plug in Boris FX. It was reasonably successful for the first window at tracking with only little movement from the surface layer. The shape layer here was well fitted to the window panes. The second window was more tricky but I later realised I could just re track window 1 and apply that motion to window 2.

Assigning Shape To Track

I added a blue solid to my surface so I could see what I was actually going to be editing.

Adjusting Track

The shape was off centre from the track and wasn't following. I had to go into the settings and tell it to follow the track. It was almost like a parent child relationship.

Masking Footage

I had a mask layer and applied it to the track and edited the mask through feather and mask expansion.

Refining Masks

I wanted the mask to be a tight fit round the window to avoid large movements. To do this I really concentrated on certain areas and key framed what was necessary.

Underwater Footage

I wanted to go for a nautical theme after looking at some of my research and also because it makes perfect sense. Underground, underwater shaky movement=water moving.

Pixabay Footage

I started gathering 4k high res footage from Pixabay of underwater scenes. I would then add these into Premiere and trim them as well as colour grade and add effects to the clip such as noise, wave warp and noise.

Masking Details

After tracking and masking I then had to mask some more! In After Effects I would pre comp my edited footage with the rough track and then with that pre comp create a new layer with the original footage with no underwater footage mask. With this footage in the layer above I then edited in my masks like the pole and stickers on the subway windows.

Repeat

I would repeat these processes every time for a new mask subject. Such as window 1 vs window 2.

Masking movement

This shot was quite tricky I felt. The movement of her hair to mask was a nightmare. I key framed the mask path, feather and expansion to give it more of a smooth movement as her hair flicked round.

4 corner pin tracker

For stable shots like this tracking purely in After Effects worked nicely. The workflow was good once you got into a rhythm, but if you had a lot of slight jerky movements you would have to adjust around every 3 frames.

Key Framing

I had tracked each window separately so this meant the footage that was playing in each window had to line up. I had to do some work with the big fish scene to get the fish coming in window 2 at a believable time after window 1. I changed the actual position on the x axis that the footage would play and clipped the track a few seconds at the start so this fish would come in slightly quicker without delay.

Effects to Mask

I had already worked on the aquarium footage in Premiere, adjusting the opacity, adding tint and adding a wave warp in some of the clips. The wave warp was a sin wave with the height massively reduced and the width massively in increased. This screenshot was in After Effects however and here I added more adjustments so I could see it in line with the rest of the footage. These were glow, noise and vignette effects. All these effects let me create a kind of reflection in the window in some of the shots. I would like to add more so the footage would look like glass.

Manual adjustment for exposure. Increase the exposure for the underground shots to give even lighting. Watch your exposure if you're on a high frame speed because it'll pick up more flickering lights.

Aperture adjustment for depth of focus. Increase the aperture to 16 for the shots where you want to capture far away depth. Decrease the aperture to 2 and you can create a blurred background effect with close up focus. With the manual on screen focus you can adjust this mid shot to give you a nice focus transition.

Press the small black button on the right to get up the focus 'map'. This is where you can adjust where in the shot the camera is focussing on.

Press this silver button on the side to give you all the Fuji film presets. This is great for a digital camera because the 35mm lens suits a vintage kind of look so these pre sets look cracking on the final output even pre edit. I used Eterna Cinema as this filter really brings out cool tones and it is the closet pre set to a cinematic film camera.

Useful tutorials

How To Rotoscope

Masking and Tracking After Effects

These were the successful tracks in Mocha. I found keeping the shape layer fairly close to the edge of the window and then making the surface almost like a parallel box within, gave me the best track.

I added mask layers for the footage but some of these had to be feathered and key framed since sometimes the would jut out of place slightly to the left or right.

This was surprisingly one of the hardest shots to integrate. I edited this in Premiere and After Effects to give it a look of a fading portal for a transition shot at the end. When attempting to put this in the mask though, the transparency background filled to white. I tried basically everything I could think of like adding shape layers underneath, key framing opacity etc. In the end I had to split this into two clips and have two versions of original footage. When this transition ended I added some grain to the window then it cut back to the original footage. Bit of a choppy transition but better than a solid white.

Final Film Track
Made in Logic Pro X

Final Export

Reflections

My final reflections on this project are that I actually really enjoyed it from start to end. I enjoyed learning new skills and would really like to push myself more trying to use Mocha. I love video editing, it's something my cousins and brother do so we all enjoy helping each other out, I couldn't have made a film as good a quality as this without my cousins camera and I'm really grateful to him for letting me use it. I'm super pleased with the narrative of the film and the shots, my idea, I feel, is just the right level of whimsical for this project.

Going forward I would like to of course add more complicated tracking shots, some shots I loved but were just to choppy to make the final edit. Like I said above I would like to use Mocha a bit more so I am more confident. I loved the nautical theme but I feel like I could use this footage to make a wee series where Eve could be in space or at a rave etc. I liked my tracked shots but of course I always think I could get them cleaner and more realistic. I did want this to be a bit of a funny video where you know it's not really real but to add more textures to the aquarium footage might help to make it look more blended with the real world footage.

I also like most would've liked more time on this project. I feel 2 weeks was too condensed to actually formulate an idea, conduct good solid research, take in all of the new information from the workshops and then actually film and produce. I also loved it so 3 weeks doing this project would've been fun :)

Launching

Imagined Environments

Research

Forced Perspective? Miniature model set?

Fame Focus - Lord of The Rings

Physically positioning the camera to help get desired shots e.g the 360 degree camera ring for the bullet shot.

Fame Focus - The Matrix

John C. Gaeta

'A Trip to The Moon' by Georges Méliès

'The War of World's' directed by Byron Haskin

Amazing use of colour on the Netflix show. Completely engulfs the viewer and makes the episode itself interactive purely through light and colour.

Olafur Eliasson

So what are visual effects?

A scene from the BBC series 'Merlin' with the fully CGI generated dragon character.

Behind the scenes of the filming of Avatar. They had also added motion tracking dots to the actors face so they could map more life like expressions to the Avatars.

Behind the scenes of the filming of Avatar. They had also added motion tracking dots to the actors face so they could map more life like expressions to the Avatars.

VFX vs SFX

Sketchbook

My Idea

Media

@Tunnel Vizion

Derek Jarman 'The Garden'

Georges Melies 'A Trip To The Moon'

'Russian Doll' Natasha Lyonne

James Mackay - The Tate Archives

This was a book published by Martin Frey which explored Jarman's collection of home movies more in depth and from a critical stance. He examines Jarman's film techniques and looks at his source of influences ranging from literature to fellow artists like Robert Rauschenberg and David Hockney.

The VFX behind 'The Green Knight'

@oliverkingvfx

Process

Manual adjustment for exposure. Increase the exposure for the underground shots to give even lighting. Watch your exposure if you're on a high frame speed because it'll pick up more flickering lights.

Press the small black button on the right to get up the focus 'map'. This is where you can adjust where in the shot the camera is focussing on.

Useful tutorials

How To Rotoscope

Masking and Tracking After Effects

These were the successful tracks in Mocha. I found keeping the shape layer fairly close to the edge of the window and then making the surface almost like a parallel box within, gave me the best track.

I added mask layers for the footage but some of these had to be feathered and key framed since sometimes the would jut out of place slightly to the left or right.

Final Film Track Made in Logic Pro X

Final Export

Reflections

Forced Perspective?

Miniature model set?

'The War of World's'
directed by Byron Haskin

VFX
vs
SFX

Georges Melies
'A Trip To The Moon'

Final Film Track
Made in Logic Pro X