Why 3D?

We are seeing a huge resurgence in interest in the use of 3D.  The most obvious case is the resurgence in interest in 3D for cinema.  Not since the days of The Creature from the Black Lagoon has 3D been so popular.  James Cameron's Avatar was a quantum leap in 3D story telling.  However, we need to make a distinction that will seem, to some, like splitting hairs, but there's a difference between 3D and stereo projection.  Stereo projection is where you have to wear glasses of some kind and the whole process mimics how we see things in the real world.  The 3D that I refer to here is rendering designs, ideas, or plans in a 3D environment.  How the 3D environment is then displayed is left up to the presenter.  Most people are familair with the use of 3D in the form of movies.  Most modern action films have a great deal of 3D content in them, and things keep getting more sophisticated all the time.  Sometimes, these days, that 3D content is also displayed in stereo at the theatre and this is referred to as 3D as well.  In what follows my reference to 3D is about building 3D models or representations - I'll leave the presentation aspect out of it for now.

Traditional 3D (Models, motion, Animation and Games)

Traditional 3D is your garden variety 3D models created in CAD or modeling software like 3D Studio Max. This kind of 3D is usually used to mimic everyday life and has found a home in movies (typically visual effects, but also entire features) Games, and VR Simulators for the military or manufacturing markets.  Even the most pedestrian movie these days has a few frames that were enhanced digitally in some way.  3D Games have become a huge industry since their not so humble beginnings in the form of the iconic DOOM.  As the fidelity of graphics hardware and software explodes exponentially, the sheer variety of what can be accomplished in 3D is approaching the unreal (literally - actually its usually about the unreal...). Games share a great deal with the VR simulators used by the military and by industry, the only real differences at this stage are technical issues with regards to the ability to guarantee that a simulation provides positive training value.  The construction of a game engine and a VR simulation engine are identical.

Having been alive at the right time, I got to experience the phenomenon that was DOOM and its follow-ons the QUAKE games, and then the rapid expansion of first person shooter gaming with the creation of the Unreal Engine, and then the Cry-Tek engine, the open source OGRE engine and on and on.  As I once explained to my students, any game is, and must necessarily be, a tunnel - this is why the dungeon based DOOM and QUAKE games worked so readily.  The game paradigm was exactly matched to game content.  The limitations of this paradigm are readily felt in big open battle games like Battlefield and the Call of Duty series where there's really only one place to go and you have to get there.  The basic recipe of building a 3D game is building 3D assets, building an engine that enables intereaction with the assets and coming up with a good story to tie it all together (by far the hardest part).

In the manufacturing world, VR simulations are useful for training and exploring a new product's capabilities or appeal.  But, when used as a marketing tool, traditional 3D allows for the creation of a 3D model of a new product, coloring it correctly and then rendering it out in video or in imagery to allow people to see it.  Most people are very visual so this kind of simulation gives the manufacturer a sense of how people will react ro the color and styling of a product without actually being able to touch it.  Most car ads these days can be created long before an assembly line is set up with the 3D models that the car designers still have on their CAD machines.  The process of rendering is so sophisticated at this stage that videos of non-existent cars can show them zooming around a track with correct physics and what appears to be a live driver (profession driver on closed course - do not attempt.  Ya, no kidding, the physics of what was done in the ad might not even be possible in the 'real' world).  This kind of simulation or testing can be carried out with any physical product that a manufacturer might consider building.  Indeed, it might be considered ill-advised at this stage to cut metal on a product that has not been tested this way.

The most quoted caveat or impediment to this process is concern about cost.  This is a specious argument for any product requiring an assembly line, as the construction costs for the line tend to be huge, while even the most expensive 'Spielberg-like' production is very affordable in comparison.  Keeping in mind that such a production might test very badly and result in not spending the money constructing the line.  In recent years a lot of tools have been incorporated into engineering CAD software that would allow an engineer to create their own rendered video of a product being designed.  But, to be honest, most engineers only need to look at the drawings to get a sense of the product.  The real use for this is outreach and product demoing to investors and the public at large, which is usually a marketing process anyway. However, marketing people tend not to be all that technical, and 3D is really very technical both in its creation and in its use.

The complexity in creating 3D models is probably the reason that people still show up at consultation meetings for a new park, or a new public building with water colors, paintings, and drawings.  Audiences these days really understand 3D and go to movies where 3D projection is used, so the reason cannot be concern for the audience that might see a 3D simulation of a park or new building.  Indeed, 3D works best in these situations.

I have been working in 3D for some 15 years.  3D is at its most effective when used to explain complex designs or complex intent on the part of a city planner or engineer.  In short, 3D is best when used as a communications medium.  3D is one of the most effective communications tools available.

Of course, along with the ability to explain complex things, there has to be a need.  In our work over the last decade with 3D, first with SGI, and then with Niagara College, we finally found the synergy between market demand and the ability to explain complex design and intent.  Any engineer engaged in a project that deals with the public has the need, or requirement, to explain their ideas to an audience that is not conversant in the nuances of complex design, and certainly cannot read or interpret engineering drawings.

Virtual 'Reality'

I once got a cease and desist letter about a 3D model that I created.  I was at a public institution at the time so we quickly complied with the letter so as not to create waves.  The reason the letter was written was because the writer was a developer and the model I had built showed his building in what he thought was a bad light - personally I thought it was awesome.  The simulation was of a VR type that allowed a user to take control of the mouse and fly around the building and its environs (a public park).  The simulation allowed you to fly into the park and look up at the building, and fly up to a balcony on the building and look down at the park.  The model was textured and lit so that the scene was somewhat realistic - at least enough to suspend a viewers disbelief to the level that they understood exactly where they were and what was being proposed (a big building next to a public park).

The cease and desist letter focused on the word 'Reality' in Virtual Reality.  No one could argue that the simulation wasn't virtual, but the Reality aspect drew fire because its an easy target.  Representing the exact nuances and detail of nature in a VR simulation is not even marginally possible.  It might be possible to get certain views reasonably accurate, but a full 360 degree from all points correctness is not even approachable (most 3D movie scenes are created knowing the exact path and field of view of a camera through a scene).  So, we were forced to admit that Virtual Reality is not Reality.

Augmented Reality

The basic idea of Augmented Reality is very cool - you look at the world through a camera and you then superimpose a high resolution data annotated VR world over your current view of the real world that then let's you tap into data that is relevant to the current view.  Honestly I'm not sure there's ever going to be a real need for this, but the data paradigm that it implies is very useful.

The only part of Augmented Reality that I think has legs is the situationally aware database paradigm that is needed to drive it.  The idea that you are at the center of your data universe as you walk along the street is very appealing to me.  The idea that the data that is relevant to my current situation flows past as I move around is fascinating.  If I had a live window into my data universe it would show different things as I walk along, these different things being related to my interests and preferences and how these relate to where I am.  As a user interface problem it is second to none because it has to constantly adjust and modify what is right in front of me based on where I am.

GIS and 3D (Automatic Modelling, or SAT - Scene Assembly Tool)

All GIS data is at least 3D data. At one point I was working on a system that would reach into a city's archive of GIS information on where all its assets are and automatically create a 3D model of the city.  Of course, most cities don't store their GIS data the same way, but this is a minor detail.  There are a huge number of reasons to do something like this that extend from emergency planning to simply keeping track of light bulb inventories for streetlights.  My former friends at ITSpatial created a 3D database of New York's Central Park that contains the species and position of every one of the 20,000 different varieties of trees planted there.  Attached to each tree were the nutrient requirements and feeding instructions, and inherent to the model were the directions to the tree.  The management of the park spoke english, the maintenance crew does not.  3D can be very helpful in a number of ways.  It can be argued that a simple 2D map could accomplish the same thing - but the 3D model had lifelike representations of the tree that were automatically assembled for the crew as necessary.  The 3D shapes in the park only have to be modeled and handled by human hands once.  The rest is all automation based on GPS and calendars.

RFID and 3D (Visual Asset Tracking)

Much in keeping with the automatic modelling described above, RFID and asset tracking can be accomplished visually using a 3D model of a factory or inventory site and 3D models of assets.

Complex Data

Today the existence of social network has seriously changed the playing field in defining what 'large data' might mean.  People are only beginning to recognize the sheer volumes of data that result from one person making one transaction on Amazon.com, or posting a response to a blog, or finding a friend on Facebook.  For any given transaction there is a huge volume of tracking data that is currently realized and an even greater volume of data that is not.  The idea of correlation is simple and implies a measure that quantifies how two or more variables are related to each other.  So, for example, the age of customer might be correlated with purchasing habits.  Seems simple enough.  However, this correlation of two variables results in a two dimensional field of numbers.  So, what used to be two simple feeds now results in a 2D field of numbers that might carry some crucial marketing information that can be mined and exploited.

The new and exploding science of GIS, once something only map geeks paid any attention to is now in everyone's life.  iPads, iPods, iPhones, Cadillacs, and just about every other car or communcation device can now tell you where you are.  And that data is all 3D (latitude, longitude and altitude).  If you combine that information with purchasing habits and drag along a whole bunch of demographic and personal data, then you have a multi-dimensional space of numbers and variables that is somehow correlated with what you might do next.  Every time you visit Amazon.com you can see it - they're trying to figure out what you might want to buy next.  That ability is the result of a complex search in a multi-dimensional data-space that, at the moment, gets a lot of things wrong, but sometimes gets it right.  Visualizing these relationships is the realm of 3D.

A few years ago we ran an experiement using RFID in conjunction with Binational Tourisms Doors Open weekend.  We used RFID tags to track participants and their travelling habits.  There was a contest that required each participant to visit each of six shops on Queen Street in Niagara Falls.  The resulting data comprised only 100 or so participants, but the sheer complaxity in trying to show their patterns and travelling behaviours was staggering.  I wrote a special query to kml routine to show the paths on a google earth window - no two people followed the same path, so the result was a criss-cross mesh of daunting complexity.

In the new age of complex data relationships and the fact that every site wants to know who is visiting, why they are visiting and what they thought about the site, the amount of data is huge.  It s a good thing multi-terrabyte drives are cheap, and its a good thing that 3D Visulization is alive and well, even if very few people have yet realized its actual potential.