An overwhelming majority of my readers (a whopping 2 out of 3!) have keenly noticed that the latest string of posts didn't really fit the primary "CGI" nature of the blog. True, I mostly touched upon marketing (something I predominately deal with at work nowadays) and some down-to-life topics which held particular importance to me personally. Some might even go so far as to blame me of procrastinating rather than working on the short film.
That's... Actually pretty close to ground truth, as you will momentarily find out, and it's something I need to finally admit. However embarrassing it might be.
This is a long post that starts slow, takes a dark turn, raises serious questions, but ultimately changes gears and sets everything right. With some cool stuff and a look at the industry from the eyes of an amateur along the way, as usual.
So get yourself a glass of some sparkling beverage and a couple snacks, and let's roll.
How It All Began
In 2016 I started this blog as a means of documenting the everlasting journey into the world of computer graphics. Luckily, I didn't go unprepared: with a fair amount of experience in video production, editing, 2D VFX and compositing as well as basic understanding of the most important aspects of 3D, as well as a released mobile game project in the portfolio, I was armed with just enough tools and knowledge to challenge myself further.
That "further" somehow turned into an animated short film project I've been working on ever since.
Well... Until last month to be specific.
What Went Right
Everything up to about... a year ago was a dream come true. I had fun working out the script and storyboarding the first draft, recording the first mocap session and sorting out the takes, trying out some exciting special effects and simulation frameworks, practicing in ZBrush.
Sculpted a couple characters, rigged them for the first "clean" take in one of the scenes to thoroughly test the overall idea and get the "feel" for the tempo of the narration – something I find especially important to get right when it comes to storytelling, before going any further.
What Went Left
A number of issues popped up along the way (actual issues, not challenges), mostly related to my catastrophic lack of skill in specialized areas, but also because of a more... Artistic nature of the project compared to anything I had ever worked on, including past live-action short films and games. I didn't bother to bring those up at all hoping I'd be able to "git gud" and "power through"...
Therefore you probably won't be surprised to find out that there was another issue I never actually mentioned before...
The Soundtrack
It's difficult to overestimate the importance of the musical score for something like a short film.
Because it's... Well, short.
Therefore the soundtrack needs to lead the narration, create proper emotional response and feel fitting overall – all in "one short swoop". There's just not enough room and material to "play with" to put the viewer in the right state of mind. Musical score was an aspect of the project I almost felt dismissive about at the very beginning. I remember thinking: "Pfft... How hard can it be? I've got good ears, I hear and feel harmony and the "mood" of a fitting chord, it'll be a breeze!"
Half a year into production I sat down with a highly qualified professional composer and we worked on the soundtrack using a roughly animated storyboard draft as a visual base for several weeks. We came up with a bunch or chord progressions, patterns and leads, whomped up a fitting melody and the chorus, harmonized the end result and mastered the final track with a different guy from the team, one who mostly specialized in that sort of work. In the end I was about 80% satisfied with the result and archived the track to come back to it later.
Except... As time went on, the false feeling of security and accomplishment faded, and as I was combining draft renders on a timeline about half a year ago it hit me like a firetruck:
The soundtrack was complete and utter garbage!
It was so mediocre it almost made my ears bleed. No idea why I didn't realize this before and why I let it pass. Probably due to the rose-colored glasses (or would it be "honey-coated headphones" in this case?..) and complete lack of any experience composing "serious" music of any kind even with a professional by my side, and not just not browsing a catalog of royalty-free tracks to choose from. So naturally, like a noob I am, I patted myself on the back and moved on to other tasks like special effects studies (something I will get to in a minute).
The bottom line is: the score sucked and soon after I realized this very fact, everything started going downhill, as I dove into a part of production I to this day consider the "scariest" since it was the one I was least familiar with...
"Pixar-like" CG Cffects
I sincerely apologize for sounding like a broken record by relentlessly alluding to the preeminent "Pixar" when talking about the quality of the final render and animation I was after this whole time. Strictly speaking, there's nothing particularly inexplicable or outright "magical" about Pixar or DreamWorks pipelines. Except there's one major aspect of the pipeline each company gradually solved over the years which I still haven't — they actually have one.
I have a general understanding of the production stages, but those are painted in largely broad strokes, an approach that worked for me just fine until this project and the level of production I attempted to reach. Medium-to-large companies and CG-oriented studios have established processes and specialized teams who work on different parts of their projects. This is certainly not news. But the crux of the matter is: they have the tools and the skills required to build and use them to their maximum potential whereas I don't.
Take hair simulation for example. I spent over 6 hectic months trying out different techniques, approaches, even Software to come up with a robust and more or less performant solution (I wouldn't want to spend a week simulating a 5-second long clip to then have to redo the simulation due to even the smallest changes in the animation). None of them did the trick, not even my beloved Softimage ICE. And I'm not paying for and installing Maya for this! That's the freedom of choice I still have thanks to being an amateur and the only way I can keep giving the finger to Autodesk for going the subscripton-only route and setting an example for other companies.
Now. Back to the topic:
Let's suppose I found a way to hack and cheat my way through the simulation steps and got the final render. Do you have any idea how long it takes to path-trace hair to eliminate noise and aliasing? Well, you probably do. It's a nightmare. And don't even get me started on re-lighting and compositing hair passes with other ones. And how could we forget the classic: "Oops! A simulation artifact I missed, gotta redo it all from scratch!"
Therefore the dream of the female character having luscious flowing hair hit a concrete wall which dealt a serious blow to my motivation. And dignity.
A Voice of Reason
At this point of this pathetic excuse of a post a flying boot from a member of the audience hits me on the head followed by an angry yet legitimate objection: "You do realize you don't have to reach quite the same level as AAA-companies for your project? The girl doesn't necessarily need to have "luscious flowing hair" if the story is good enough!"
Perfectly logical and well put to words. The boot was a nice touch as well.
Now allow me to retort:
What's The Point Then!?
No, seriously.
The original idea was to put maximum effort into a short video, to surpass the level I was previously able to reach with real-time rendering approaches – in Unity, for example – but not to go overboard with project scale, like chasing a full-length feature, which would be outrageously stupid and childishly naïve. If I originally decided on a much more stylized and simplified approach (think Overwatch shorts by Blizzard) – with sculpted spring-driven hair, for example – I wouldn't even bother to spend over a year studying simulation techniques and trying to push myself so far as to try and replicate the level of production of those large studios.
I do realize why Blizzard themselves settled on a highly stylized vision for majority of their games and marketing materials of all sorts – it's easier to concept, iterate and finally produce.
It's also much simpler to turn those characters into plastic figurines and other cool recognizable merch. My original idea was to not go the easy way, but the final run of events ultimately made me realize my lack of specialized training and/or 3DCG-related experience.
Is It Worth The Trouble At All?
Mind you, everything I've worked on was destined to culminate in the form of an eight-minute animation. 8 minutes-worth of video! That's it.
"Thanks for watching and see you in another 5 years", right?
Along the way I would also have to be absolutely and utterly committed to each decision following the production prep stage, spend countless hours on every little detail trying to reach the highest production level possible by (mostly) a single "artist". Produce several rigs for various shots, including "finger" and custom "facial" rigs for close-ups as well as simplified ones for background characters with dynamic retargeting corrections to save time on animation polish (think full-body IK that makes sure motion and poses stay biomechanically plausible for bipeds)...
70% of these don't feel like exciting experiments and CG exploration, but boring routine work which, granted, I had a lot of to get through while working on the mobile game, but I never felt as "helpless" as with this endeavor. Repeatedly hitting concrete walls, being surrounded by them, and having a hundred rakes eagerly waiting to kick you in the balls almost every step of the way is anything but "fun".
In this particular case majority of the work I'd have to do would be boring and tedious as hell. Also, as it is with most non-commercial side-projects, it's primarily about the journey and not the destination. And I can't even put into words how much I hate the place I ended up after a couple years working on this tiny Feature...
Now What?
Ah... From here on things start to fall into place. Finally.
With the shackles of a "full-blown CG-like raytraced"-production level requirements finally shattered and quality standards forcefully reduced, I decided to go with a simpler approach. I was finally free to dive into a real-time rendering engine like Unreal, Cryengine or Unity and go though a check-list of stuff I would need to "simplify" or "stylize" in order to fit the project within limitations of a rasterized rendering approach (maybe with a couple of real-time ray-traced passes for reflections and such, as explained in my previous post on real-time rendering).
Unity blog post on how this short film came to be. A must read!
As I was moving actors and lights around in Unity, I had a chance to check out:
- the brand new High Definition Rendering Pipeline;
- the new amazing material shaders (even refractive ones which actually distort objects behind them and not just the sky-box);
- animation and sequencing tools like the Timeline;
- unified procedural camera system – the Cinemachine;
- some truly breathtaking performance-related developments like the Entity Component System and Data-Oriented Technology Stack (DOTS) which allow to build extremely detailed large scenes with nested prefabs and actual sub-scenes with automatic LOD (distance-based level of detail mesh substitution) and memory management.
... – just to name a few!
I also checked out a couple of specialized assets on the Asset Store to make sure the ones best suited for rendering out the best quality pictures were readily available.
Those include supersamplers, improved shadows, deferred render pass exporters and such.
I'm happy to report that every single asset one would need to render our the prettiest picture possible with the modern rasterization tech as well as ray-traced passes here and there were available and most affordable.
What followed suit was another realization that shot through my brain like a bullet train while I was setting up cameras around the scene:
Wait a minute... Unity is a game engine! I can make interactive stuff with it!
A truly shocking revelation, I know... By all means, do take a moment to collect your thoughts after such a profound discovery.
Ready? Here it goes...
What If...
Obviously, after switching to a "real-time" rendering pipeline a fair bit of "true CG-look" in the final rendered frames would be lost. Also, as far as my storytelling skills go, the short I planned wouldn't have had some uber-original and groundbreaking story that would've carried it all the way regardless of the visuals.
But! What if I tried to compensate the inevitable "downgrade" of the visuals with some kind of interactivity or even change the type of the project from a "canned" animated short into...
Dare I say it...
A Game!
Man, did I miss this feeling!
Good news at last – recently I've been having the time of my life mucking about in the latest build of the Unity engine and... Oh, wow, has it changed! There's even more cool tools for me to talk about, and for you to read and see whether they would be useful for your projects. So it's a win-win situation however you look at it.
I believe another reason I kept secretly considering switching from a film to a game were all those "deep games" I listed in another post. Most would never work in any other form like a movie, or a book, and the very reason they were such intense experiences was – they engaged more senses then just eyes and ears: the control over characters' actions and decisions as well as the feeling of progressing the story with your own actions, and even choosing your own pacing while doing all of the above, where possible.
Also, having a much better understanding of my own skill level as well as limitations I will use them to steer the project into a route much more suited to my style of work, where it's sometimes a requirement to tweak or outright change shots and timings even during Production stages. And knowing that having a real-time-rendered pipeline I would be free to truly experiment and see a final rendered frame...
The "dam" was finally breached and the project kicked into high gear.
"Project v2.0"
Not being constrained by the short film format any more I decided to spend some time looking at the story from new angles. Since a fairly interesting cast of characters would absolutely benefit from having story arcs longer than a couple minutes I thought about giving them more "screen time" or even a couple voice-lines, some unique character traits and so on. One thing led to another and...
After a month of putting those ideas into words, I ended up with a decent draft of...
The Design Document
Writing Software/Web project DD's is nothing new to me, but none of those which I had undertaken over the years were as passionate and elaborate as this one – 120 pages and counting (the primary reason it's been so quiet in the blog lately – it's all about the DD nowadays):
Still, I'd say the description is only about 1/4 done and it should take me at least 6 months to produce a semi-complete summary. A well structured project doc is something I consider an absolute requirement when dealing with large projects.
It's like building a house: if you have a blueprint or two and a solid foundation, all you need to do is follow the former and rely on the latter. There's nothing worse than having to go back to the planning stage mid-project. Not only is it counter-productive, it's also demoralizing as hell, at least for me.
So for the next several months I'll be mostly focusing on inventing quests, defining story progression and milestones, exploring gameplay possibilities, delineating compelling character arcs and dialogs, listing and describing interactive items, locations and so on and so forth. All of this – in order to cover as much ground as possible before moving on to the prototyping stage. I'll try to find the time to share the progress and report on the most exciting new features of the Unity engine I would want to use during development and hopefully one day ship the final product, to keep you entertained and in the loop at all times.
To be continued!
tl;dr:
Challenging yourself is good. Overestimating your abilities and stressing out – not so much.