THE GREAT WAR - RECREATING THE WESTERN FRONT FROM THE AIR - PART 2 by sandy sutherland

Following on from the previous blog post, this one covers how I used Houdini to generate all the fields and crop distributions that cover the majority of the terrain.

Houdini ?

If you are not familiar with what Houdini is, it is 3D content creation package, known for it's powerful procedural tool-set. It's probably better described as a tool that lets you build other 3D tools. By wiring together nodes that perform very specific tasks, you can build up a chain of processes that crunch data and spit out pretty pictures, simulations and models that would often be impossible to make manually. If I was to sit and try and paint all the natural looking divisions and distributions of fields that cover my terrain it would never get done. Using Houdini I can churn out revisions in a couple of hours by focusing on defining the "rules" that lead to the fields being where they are.

PIXELS TO GEO

So I had the image I had assembled of all the roads covering my terrain in Photoshop. It was a simple black and white mask. Houdini has a handy node called "Trace" that reads in an image and cuts up a piece of geo based on the image. So I dropped down a simple grid, cut it up with a trace node and then I was left with a single polygon for each "field" area between my roads. All in all there are about 19,000 of them. From here I did some simple edge re-sampling and clean up.

USING ROADS TO DEFINE FIELD ALIGNMENTS

It's important to note that in the previous step, I'm basically getting 1 field = 1 polygon. This make it easier to set up an operation that loops over each input polygon 1 at a time. The polygon by itself is a bit messy, it has too many edges and is non-triangulated but it doesn't matter right now, the 1:1 relationship is the important bit.

Looking at real maps, I decided that it would be logical for fields to be aligned to the main direction of the fields shape, or perpendicular to that direction. It's not a random distribution, but it does appear to be different field to field. So, I had to work out a way to calculate the main "direction" of each field.

Roads, and the 19,000+ "fields" between them

Roads, and the 19,000+ "fields" between them

The basic method was this

  • Get the normal of each point along the polygon border. Re-sampling the edges from before gives me more points to work with
  • Make the normal point towards the neighbouring point
  • Convert the point normal direction value to a rotation value
  • Down sample the potential rotation values into a smaller range of options (15 degree increments for example)
  • Tally up the rotation values rounded up to sit within the smaller range of increments
  • The rotation value with the highest number of points rotated in that direction is the dominant direction for the field
  • Finally, store that rotation as a colour value in the blue channel of the per polygon colour attribute

Now I'm not totally sure how sound that logic is but it seemed to work pretty well. I'm sure in the real world the rules that govern how any one field is planted are more complex. Sun direction, slope of the terrain, wind etc I'm sure are all factored in. They are things I could work out but for now, this will do. I made a loop that does that calculation for every single polygon, and stores the resulting direction value as a custom attribute on the polygon.

The field "direction" mask. The texture itself probably isnt that useful, but the attribute it represents is used for rotating UV coordinates per polygon

The field "direction" mask. The texture itself probably isnt that useful, but the attribute it represents is used for rotating UV coordinates per polygon

Roads and Field rotations together

Roads and Field rotations together

Now, originally I planned to render out a texture map with this blue value stored, and use it to rotate the UV's inside my material in Unreal. This was a bad idea from the get go, because texture compression was always going to corrupt the rotation values per pixel. 

In the end, doing the same kind of operation, rotating the UV's in Houdini was the way to go, and it simplifies the Unreal material anyway. Win win.

 

FIELD & FIELD CLUSTER GENERATION

So now I have each polygon UV mapped and its rotation calculated. Time to generate some crop patterns! This was a pretty simple process, but it did go through a few revisions to get the right kind of look. The beauty of doing it in Houdini is that it's super simple to experiment without breaking anything.

Always work with reference! Here is a small patch taken from Google maps, sized to fit a stretch of the Somme river on my map. It shows a tiny sample of the field complexity I need to create.

Always work with reference! Here is a small patch taken from Google maps, sized to fit a stretch of the Somme river on my map. It shows a tiny sample of the field complexity I need to create.

The idea was to make a square image, rendered as an RGB mask, with a unique field arangement on it. Then I would render out 500 or so variations, and assemble them again into a cluster of combined field arangements giving another level of complex variation. Then in turn I rendered out 1000 of those combinations.

Generating the patterns simply involved mixing and merging simple grids with randomly generated numbers of rows and columns. Then I might rotate the whole grid 90 degrees. This maybe/maybe not switching was controlled by switch nodes that picked a stream at random. Then, a range of red colour values were assigned per primitive, so each polygon in the grid got a random colour. Then, I resampled the polygon edges to get some more detail and applied a very subtle amount of noise, just to break up te straight lines.  Then, I looped through each polygon again and applied a tiny amount of jitter to its position and rotation. I also added a variable amount of smoothing to the edges of each polygon, so the fields had slightly rounded corners. Last thing to do then was to stick a flat grid colour under the whole thing to fill any gaps, and when you render it out you get these kind of random combinations. This was all super quick and dirty, I didnt care about geo overlap, or nicely fused edges, non of that mattered. I just wanted random shape arrangements.

A small snippet of a typical, all be it messy Houdini graph. The blue nodes are switches that randomly pick which chain of the graph above to read in as data flows down the chain. 

A small snippet of a typical, all be it messy Houdini graph. The blue nodes are switches that randomly pick which chain of the graph above to read in as data flows down the chain. 

So here is the kind of thing I would get back from the first pass of just creating field patterns

Field patterns

Field patterns

Then I read them in as textures and further combine them to create field clusters

The first pass of field patterns applied per polygon to another grid

The first pass of field patterns applied per polygon to another grid

Now for the fun bit - I apply the cluster textures WITH the rotation values from the first step to each polygon field. It gives me something like this:

A bit of a jumbled mess, but it's promising

A bit of a jumbled mess, but it's promising

RENDERING AND PHOTOSHOP ASSEMBLY

I render out the entire terrain covered with these field patterns as 4 x 16k tiles. I then have a single 32k Photoshop file where I'm assembling all the layers. It's a massive document, but I have some Photoshop actions that cut the layers into 8k tiles that fit together as described in my last blog entry. There are lots of layers including roads, trees, waterways, height and slope masks, city and town placements, various noise textures etc. All of these will get mixed and enhanced with hand painted details at some point.

MASK AND GRADIENT MAPPING

Now that I have this field texture made up of shades of red, it's time to do some gradient mapping. I took a screenshot from good maps and sampled it to make a new gradient in Photoshop. Remapping the reds to match this new gradient, and then pasting the google screengrab on top looks a bit like this.

The google screengrab in the middle almost melts away as it mixes with the surrounding Houdini generated fields. Pretty cool! Still needs some colour tweaks and noises mixed in but it's a good result I think. 

The google screengrab in the middle almost melts away as it mixes with the surrounding Houdini generated fields. Pretty cool! Still needs some colour tweaks and noises mixed in but it's a good result I think. 

That's it for now, next post will cover tree distribution, detailing the colour maps, normal map generation, and other good stuff. Stay tuned!

The Great War - Recreating the Western Front from the air - Part 1 by sandy sutherland

This is the first part of a "making of" series of posts I will do, covering the process of how I put together this giant terrain. The goal is to share knowledge, but also to show how things are often not a straight path to a finished result.

The Goal

Flak Jack is a game that takes place entirely in the air. Even so, to provide the proper context and backdrop for the action in the sky I wanted to depict the landscape, battlefields, and towns of the Somme and Western Front, to a reasonable degree of accuracy. There are no guides in the game or magic arrows that point you to your next objective, so using geographical landmarks like the Somme river, or trench lines will be all the player has to get their directional bearings.

I also wanted vast view distances as you fly around in the clouds, giving the player a sense of the scale of the battlefields, but also the sense of freedom and loneliness fighter pilots felt high above the trenches. VR is naturally quite a claustrophobic experience and the goal is to counteract that as much as possible.

Given my target frame rate is at least 90fps, making such a vast landscape is a big technical challenge, taxing the game's render resources even before I add any planes, characters, FX etc.

So some quick stats on what I'm currently working towards

  • A 200km x 200km area of Northern France and the Somme river, derived from Satellite DEM terrain data
  • A view distance of 90-130 km, with the player flying at an average altitude of 1.5-3km
  • 8K terrain textures for base colour, roughness and maybe normals
  • Real world road data
  • Procedurally generated crop and field patterns and tree distributions
  • Dressing the terrain with possibly millions of trees, basic buildings for towns, mesh based trench lines, towers of smoke FX breaking up the horizon and ground battle FX
  • A playable flying area of approx 50km at any one time, varied from mission to mission
  • Truesky driven dynamic lighting, with clouds casting shadows over huge areas of terrain
  • A mechanic in the game that limits the player getting any closer to the ground than 1-15km above the terrain, with a story based transition effect that occludes the landscape at that close range.

The last bullet point there is a design choice I made in order to save time, meaning I don't need to create terrain that holds up at a close distance. The game is all about the flying, there is no air-to-ground combat, no take offs, no crashing into the ground. The landscape is just a backdrop, there to provide context and atmosphere.

 

Reference material

I'm not trying to be a slave to reality, but I do want to create an environment that feels correct. Anyone who has been in a plane has a sense of what its like to fly over farmland, what it feels like to pass through the clouds etc. With that in mind I decided to use a mix of real world data, and procedural generation. The real world will give me all the natural scales and distributions, the details will come procedurally. I also have the added complication of creating a real place, but as it existed 100 years ago. On top of that, huge areas of my landscape will be heavily damaged, dotted with artillery craters and trench lines. Luckily, there is a stack of material available online. Aerial photographic reconasissance was actually a huge role for aircraft of the time, and many of the photos taken then still exist. They are amazing and terrifying in equal measure, as they give a true sense of the scale of destruction that occured along The Western Front. 

Here is a small sample of the reference I've gathered so far. Some is from the area I'm trying to create, some is from other wars like the Vietnam War, some is just good reference for scale, lighting and atmosphere. Other games of course are also an interesting point of reference (The Battlefield 1 concept art book is brilliant inspiration!).

 

 

Try and try again

When I started this, I didn't know exactly what approach I would take. It has been a huge amount of trial and error. I've started over from scratch a few times now, sometimes because the results didnt look good, other times because the approach didnt allow for much revision, and also because the methods might not be very efficient and optimised once in engine.

I grapple with technical tasks in a strange way - I don't have a very solid maths or computer science background, so I don't often see the clearest path from the get go. I have just enough grasp on the technical side of things to be able to try different ideas, and more often than not I do manage to get the result I was after, but it's far from an efficient path. Most of what I write on this blog is going to be the messy, unfiltered break down of my process.

Before going to much further, I have to thank my friend Dan on the great guide he put together for getting height maps into UE4 at the correct scale - check it out!

Below is a quick summary of things I've tried and then started over again on in regards to making a 200kmx200m terrain:

First attempt

  • Get DEM data, assemble it into tiles as per UE4 terrain docs
  • Cover it with the maximum allowable sized single texture (8k), generated from the height information in the DEM data.
  • Add roads extracted from assembled screengrabs taken from snazzymaps, assembled into an 8k texture mask
  • No good solution for generating the field patterns at this stage...
  • Tested procedural foliage placement tool for trees. Good start, can use texture masks, not totally sure how well it will scale at this point
  • Basic truesky added for atmospheric perspective test. View distance feels good. Set base cloud height to 1.5km

Outcomes

  • Assembling and processing the DEM data involved jumping between a few different apps. Not very procedural
  • Way too many terrain tiles! Good for a streaming tile system, but I need all tiles loaded all the time because of my huge view distance
  • Nowhere near enough resolution trying to cover the whole terrain with a single 8k map
  • Roads took ages to assemble, and then the base image resolution still wasnt high enough, resulting in blocky, very wide roads. Throws off scale.
  • Shadows from clouds in Truesky are done as a light function. At the time of this test light functions don't yet work with the new forward renderer in UE4.
First attempt - 1024 individual landscape tiles! 1 8k colour map stretched over the entire 200kmx200km terrain. Terrain itself has its height values doubled across the whole terrain to exaggerate the shapes. In the real world it's a very flat area. From this distance, it *kinda* works but any closer the texture resolution falls apart. 

First attempt - 1024 individual landscape tiles! 1 8k colour map stretched over the entire 200kmx200km terrain. Terrain itself has its height values doubled across the whole terrain to exaggerate the shapes. In the real world it's a very flat area. From this distance, it *kinda* works but any closer the texture resolution falls apart. 

Second attempt

So after getting to this point I decided a few things coud certainly be done better. It was obvious texture resolution was going to be a problem. I didn't know how I was going to generate the farmland crop patterns yet. The number of terrain tiles seemed silly. So I came up with a few revisions.

  • Use Houdini - especially the new terrain tools in Houdini 16
  • I sourced the DEM data from a higher resolution source. 30m res instead of 90m I think it was...
  • Cut back the terrain tile count. I'm not streaming them in, so I don't need anywhere near as many. I think each tile is a draw call, so the less the better
  • Make a field patch generator in Houdini
  • Refine the roads texture mask, so the roads were thinner
  • Cover the entire terrain with a series of smaller, tiled textures. The middle of the map could be higher resolution than the outer half
  • Treat all these base textures as masks for distributing tiling detail textures
  • Use Houdini to work out the general "direction" of each crop field, so that the field patch textures could be aligned in a logical way based on the direction of the roads that border them
  • Maybe generate a texture where this direction angle is stored as a colour, and then applied as a rotation value going into the UV's of my material textures

I prepared the newer higher res DEM data into a single image, and loaded it into Houdini. I enhanced a few areas a bit, just making some of the features more exaggerated. I also tripled the height range towards the very edge of the terrain. I figured it would help break up the horizon, which is an area of the map the player will never get to anyway. I was also able to output some textures from houdini including height, slope, erosion masks etc.

 

Simple processing of the height map in Houdini

Simple processing of the height map in Houdini

I made some big changes to how I mapped materials and textures to the whole terrain. Instead of using 1 material, I took advantage of the fact that the terrain was already split into tiles, and then made material instances that were mapped to groups of those tiles. In terms of textures, I split into 4 middle tiles and then 4 outer tiles. The middle tile textures are 8k, the outer are 4k. They overlap of course, but when applied to the groups of terrain tiles the overlap is not a problem. I originally tried "stitching" the tiled textures all in the one material. This presented me with a big problem where textures were smearing outside their UV borders. Eventually I realised this was because of mip mapping and general texture compression side effects. I also had a bunch of fiddly UV offsetting to manage to get the texture tile to sit where I wanted it. I have to thank Cory from Morepork Games for helping me out here, explaining the problem. Masking the texture borders could have been done in the material, but it would have added unwanted shader complexity, and in the end his material instancing / mapping to tile groups suggestion was the way to go. Thanks Cory!

So the texture tile breakdown looks something like this:

This is the full 200kmx200km area. The inner 4 coloured squares are each covered by an 8k texture. The larger more transparent 4 tiles are also covered by 8k textures, but cover 200% more terrain so are effectively 4k. The blue lines in the top left quarter show the landscape tile divisions in UE4 (8x8 = 64 landscape tiles). The rings are distance markers, 100km, 50km, 10km and 1km area. Each colour tinted tile has a material instance applied, so 8 instances in total.

This is the full 200kmx200km area. The inner 4 coloured squares are each covered by an 8k texture. The larger more transparent 4 tiles are also covered by 8k textures, but cover 200% more terrain so are effectively 4k. The blue lines in the top left quarter show the landscape tile divisions in UE4 (8x8 = 64 landscape tiles). The rings are distance markers, 100km, 50km, 10km and 1km area. Each colour tinted tile has a material instance applied, so 8 instances in total.

That is probably enough for this post, part 2 will be coming soon covering how I made these crazy field patterns in Houdini!

Look Ma, I'm a Farmer!

Somme Terrain progress by sandy sutherland

Things have been moving along nicely in regards to terrain work. I pretty much went back to square one, after trying several different process. The main hurdle has been trying to pack 40,000 sq km's worth of texture data into a manageable set of textures, that give a respectable level of resolution. Things like getting the width of the roads to feel approximately correct has a big influence on how you perceive the scale of the world. 

As I've mentioned a huge tutorial will be done outlining the whole process, but until then here is a small update video. Hopefully it gives a little sense of the atmosphere and scale I hope to create.

 

The Vintage Aviator - Gallery by sandy sutherland

As mentioned in the last blog post, earlier this year I got the chance to visit Hood Aerodrome in Masterton, home of  The Vintage Aviator. This is an amazing collection of WW1 aircraft, many of which are still fully functional flying machines. It was a great chance to find out about the history of the planes, the early design ideas of the time, and also just how fast things developed over the period of the war. This was still a time where the rules of flight were still being figured out.

I took a stack of photos during the visit, with the goal of recording a lot of the small details of the planes, what materials they are made of and how the surfaces react to light. I also took a bunch of video which I may post at some stage.

Enjoy!

Follow this link to go to the full page gallery - The Vintage Aviator GALLERY

A long overdue update by sandy sutherland

It's a little embarassing to notice my last blog post was August last year. I've actually been thinking about and working on Flak Jack a whole lot since that time, but my day job at Weta has taken over. For almost a year I've been working on a major sequence for the film, War For The Planet of the Apes. We are days away from finishing which hopefully means more free time to dive back into Unreal Engine.

Scope vs time

At this stage, progress on the game has been very slow in terms of actual on-the-box development time. I'm being very ambitious, and available time to work on the game has certainly been less than I would have liked. Having said that, I'm not overly worried. I don't have a deadline I'm being held to, and I have plenty of experience working on things that just take time, so motivation and persistence are on my side. I've also had time to document a lot of ideas and spend time thinking about systems. I have a solid day job that pays the bills, even if it requires crunch periods that take time away from other things. Basically, this thing is for fun, it's not something I have to do to stay off the streets. Time is just starting to free up again so I'll be able to block in a few core game systems like flying and basic shooting, as well as a solid pass on the environment art.

Design progress

I've done a huge amount of writing and re-designing the structure of the game, really thinking about ways I can make the game a smooth, seamless, immersive experience in VR. On one hand, I'm making "just" another third person shooter game. However, there are a whole lot of really strange idea's I'm planning to try that I hope will set the game apart. It's cinematic but with no cutscenes. It's a small character study, on a massive stage, its silly and fun but with some darker undertones. One positive side effect of work getting in the way is that I've had time to think over ideas and concepts. Many have evolved into much better ideas. Some faded and died. Overall though, the ideas have had time to just sit with me, and I can say that I'm still super excited about what I'm aiming for. Words are cheap of course, so until I start to implement some of these designs, who knows how well they will really click together. It's going to be really hard to strike just the right tone. One thing is for sure, nobody has made what I'm going to try, especially in VR. That is both terrifying and exciting. More details will come down the line once some implementations are further developed.

Vive!

Late last year I applied for a developer grant from Epic Games. I shot them a quick summary of my game, and then kind of forgot about it. A few weeks later, I get an email back saying they were going to send me out a Vive headset. Nice! Thank you Epic! I could say goodbye to the Oculus DK2, and Razer Hydra. 

The Vintage Aviator / RAF Museum Visit / Imperial War Museum

Earlier this year we had a chance to go and visit Hood Aerodrome in Masterton, a couple of hours north of Wellington. This is the home of The Vintage Aviator, one of the best collections of functional, flying WW1 airplanes on Earth. We were there for one of their flying weekends, where you are given incredible access to the planes, there are flying demonstrations and a very informative walking tour of each plane as they sit out on the field. It was an extreamly valuable opportunity to gather photo, video and audio reference. Some of the planes in the collection are the only ones in existence in the whole world, and they are only a few hours drive from my house. Amazing.

In February, we went on a 3 week trip to the UK. I took a day to myself to go and visit the RAF Museum in North London. They have a relatively new exhibition focusing on the aircraft of WW1 and the history of aviation as part of the war effort. It's an amazing display, and I had the whole thing almost to myself for a few hours. I also paid a visit to the Imperial War Museum which itself has an amazing exhibition of WW1 stories and relics. WW1 sure was an insane time period.

I will post a gallery of the reference photos and video I took on these trips very soon!

Terrain / Environment development

One major goal for the game is to give you that vast sense of scale, of what it feels like to be 3km's up in the air, in an open cockpit wood and fabric plane, cold air and engine oil blasting you in the face, high above the vast wasteland of The Western Front. To achieve that I need a space with huge view distances, right now I'm targetting an area of 200kmx200km in size, with an accessable play space maybe 100 - 150km in size. I'm using real world Satellite terrain data, and then detailing that procedurally. I will eventually do a far more detailed blog post about it, but right now I'm going through a second revision of the terrain process as I try and use Houdini 16's new heightfield tools, combined with Quixel Megascan textures and Substance Designer textures. I am still using Truesky for my clouds and lighting solution, which is due to receive a pretty major update in the not to distant future. Some very cool stuff.

Some early progress on terrain. 200km square. Real world scale and real roads.

Some early progress on terrain. 200km square. Real world scale and real roads.

A very quick test of mixing textures in Megascan Studio. No mans land craters + puddles

A very quick test of mixing textures in Megascan Studio. No mans land craters + puddles

 

 

Bonnie snaps - Some visual references for Flak Jack by sandy sutherland

Thought I would post some of the images I've found as inspiration for Flak Jack. Some of these are targets for what I want the game to look like, some are character relationships,  some point to the sense of humour I'm going for. Others are just weird and wonderful things I've discovered looking into WW1 history, early aviation and Scotland.

 *** NONE OF THIS IS MY WORK - It's all pulled from the net and made by people far more talented than I.

"It's nae a skirt!" - the first 5 months developing Flak Jack by sandy sutherland

What? Why?

Back in early April, I got slapped in the face with a ridiculous idea. It was so vivid from the get go, that I knew I had to make it; 

"A comedic VR game where you fly a plane with your head movements, and shoot with motion controllers in air to air dogfights, set in WW1. The pilot is a mad Scotsman. He has a co-pilot dog. It's called Flak Jack."

That all came to me in that very first instant. It was so daft, I had to dive right in. Nobody else was going to dream that one up.

Early reference images - The 3 main elements of Flak Jack

 

I've been tinkering with game development for a couple of years now since the release of Unreal Engine 4. Spurred on by a couple of friends at Weta who are also interested in game dev, I've been stepping it up, working away on various things several nights a week. In that time I've realised that the bit I really enjoy the most is game design itself, dreaming up worlds and gameplay systems. So, after a fair amount of time playing with certain aspects of UE4, learning the engine, I decided I was just going to go all in, and make a complete game. I was also going to try and do it just using UE4's blueprint system for visual scripting. That means I would be typing next to no code.

It's a stupidly ambitious thing to try and do. Even a "simple" game is more complex than most people would imagine. Still, if there is one thing I've learned from 13+ years of film VFX, it's that I've got a pretty solid sense of how to break complex things down into manageable pieces and construct a system. I know how to create spectacle. I've got a fairly good idea of how drama and pacing work. In that way, there is a lot of crossover between VFX and game development. More than that though, after years of making other peoples stories and worlds, I felt it was time to put something out there that was my idea. 

Where the hell did that come from?

At first glance it seems like a pretty random game concept, but ideas do not come from a vacuum. My Dad's side of the family is Scottish and I grew up exposed to Scottish culture (I grew up in Australia). My Mum's father built model air planes and I used to build model kits as a kid. I spent many a school holiday at their house playing flight sims...and Doom. We played lots of Doom. It was my grandfather who first introduced me to 3D graphics, he used to draw plans for his model builds in AutoCAD. I've played plenty of shooter games over the years, and despite the vast numbers of shooters out there, I feel I've found a somewhat fresh take on things - especially for a VR game.  There are dozens of other references and influences that have since fed into the game idea; Tintin and Snowy. Hans and Chewie. Wallace and Gromit. John Woo movies. Tarantino. Slapstick comedy. Indiana Jones. Bagpipes. Sea Shanties. Homing pigeons. The advancement of technology. It's a good ol' mix.

Get on with it.

So I started. Armed with UE4, an Oculus DK2 VR headset, and a Razer Hydra - hand held motion controllers.  Beginning with the flying game template provided by Epic, I took their crude flying saucer demo and wired it up so that the pitch/roll/yaw of the DK2 controlled the pitch/roll/yaw of the spaceship. Then I swapped out the spaceship for a dodgy model of a plane I found online. Already I could fly around a level steering the plane with my head, the plane sitting in front of the view, moving at a constant speed.

VR Camera + Plane + Distance Markers

The absolute basics of flight were now working. There is a huge amount still to do on flight mechanics but I'll save that for another day.

Clouds

One of the main goals I have for Flak Jack is to be able to use 3D volumetric clouds as a play space, not just a backdrop. Clouds and skies in many games are either an image mapped on to a giant sphere that surrounds you, or 2D cards/sprites with a cloudy texture applied. Neither was going to be enough in my case. UE4 does not have a native method for creating volumetric FX like clouds. They are typically heavy to render and require lots of data. So, I had an idea and fired up Houdini. The basic concept was to make cloud volumes in Houdini, render them from a bunch of angles with lighting baked in, and then use UE4's imposter sprites to draw the clouds in the game. Imposters are a trick used to show the impression of a 3D object from a series of images taken at slightly different angles. Here is an early test result.

Houdini volume renders with lighting baked, as imposter sprites in UE4

Houdini volume renders with lighting baked, as imposter sprites in UE4

This was an interesting test, but I knew it was going to be limited and time consuming to generate enough variety. 

Then I discovered Truesky. It's a plugin for UE4 that allows for the creation of very natural looking clouds, with automatic sun light control and atmospherics. It's also fully volumetric and controllable with blueprints. It runs great in VR, and they have an Indie licencing option. Boom! Clouds. Sorted.

Early Truesky test with a terrain ripped from Google maps. Promising!

Blueprints / UI

I realised fairly early on that I still had a huge amount to learn about UE4's Blueprints system. Mainly, how various blueprints should talk to one another. After I hooked up the headset/flight control I saw straight away some kind of sensitivity control should be exposed to the player so that the plane movement wasn't too twitchy.  I got started on a UI, and made a slider to control sensitivity. In VR games you really need to have any kind of UI as a physical thing in the 3D world, you cant just slap it on the screen with no distance from your eyes, otherwise it makes you want to puke and you can't focus on it. The Oculus Store has a nice UI interaction system where the selection cursor is locked to the middle of your view, so you just look at the thing you want to click on. I decided on the same thing for Flak Jack. I made my UI buttons, stuck them on a 3D widget, added the widget to sit a bit in front of my game camera, and then did a line trace from the camera to the widget to work out what button was sitting in line with my gaze. I added a crosshair graphic, and made the buttons change colour when hovered over. I decided to use the sensitivity slider purely as visual feedback on the UI - the user changes it with the left stick on the controller.  It's not a slider that you click on and grab. I did this because it makes it faster and easier to change when mid flight. Originally, the slider was visible whenever you call the game UI, but now it only shows up when you are actually in flight and call the UI.

Oculus style menu selection for VR UI, with slider feedback

Motion controllers / Jack IK

Flak Jack is a third person game. Jack flies standing up in the cockpit so he can shoot out over the top of the planes wing with hand held weapons. I wanted his arm movement to be in sync with how the player was moving the motion controllers. I set up some really basic arm IK, and did some calibrating and offset of the Hydra controllers, so that the player motion was offset from their position, to where Jack was standing in front of them. Eventually I'll need to do something where I get the players maximum reach / arm length and adjust the IK and Jack animation to match. The plan is to incorporate gesture controls for reloading and things like that.

July rebuild

After a couple of months of slapping things together, my blueprints and project were already getting messy even at this early stage. I stopped adding new shit and did some house keeping. I had originally just added lots of things as components on the flying pawn. The plane mesh, Jack, all the motion controller stuff, particle FX, the UI etc etc. I began to bundle things into their own blueprints and then added those blueprints as child components to the flying pawn. I then added all the required functions to each blueprint, moving them off the flying pawn. It's now much cleaner and logical and is more in line with how I should have done things to begin with. I began to get more comfortable with casting, and getting all the blueprints talking to each other. All rookie errors I'm sure, but I'm glad I caught them early.

Mission control

One goal is to make the steps from first opening the game, to flying an actual mission as seamless as possible. It needs to ease you in to things so you can get comfortable being in VR - once you are actually flying and shooting its going to get pretty hectic. The game loads up with you positioned just above the clouds. The idea is you pick a mission type and then various aspects of the mission are randomly generated, one of those being the clouds and time of day. So, before the mission actually starts, the game goes through a little 15 second sequence I'm calling the "pre trip". It gives the player a moment to get settled in. During the pre trip a timelapse happens and the clouds/sky changes, you get transported to the mission area, and Jack swoops in from behind you, does some funny things or starts telling a story, and you are gradually given control of the plane flight as it settles in front of you. It's visually cool, gives a chance for some character dev with Jack, allows time for the mission elements to load and basically takes you from point A - the start menu, to point B - the mission start in one smooth move. Sweet. I'm still sussing out the clouds timelapse and need to smooth out the travel speed, but the basic idea of it I think will work.

I created a blueprint called "Mission Control" that handles all of this pre trip stuff, sets all the parameters for the selected mission, and just generally initiates and keeps track of the current mission setup.

Art - I mean "Art"

Over the years my art skills have got pretty rusty, but I started to flesh out a few stand in art assets to act as place holders. The first of those was the logo, which I wanted to make look like trails in the sky. I also mocked up a super simple model of the plane you will fly with in the game - The mighty Bristol F2b.

OOOOOOOOOO purty

Probably the best 3D model ever made

Where it's at 

So that covers a lot about what's been involved in getting things to this point. Right now it's all still something I'm just bashing out a few evenings a week, I'm happy with the progress however. It's got a long long way to go, but here is a video of it all together.