3ds Max and Unity -- The good and the bad (and some potential solutions)

[In this reprinted #altdevblogaday in-depth piece, Super Mega Awesome Games founder Mike Wuetherick talks about quirks he's encountered while developing in Unity and potential workarounds for those problems.] To begin, I definitely want to mention that I'm far from a "Unity expert" at this point – I'm still learning and doing a lot of trial and error at this point, but am at the point where I'm at least comfortable enough that I feel like throwing this out there for the community. But first, a bit of background: In November, we shipped our first game, built using a custom engine and all of the pains and joys that brings. Unfortunately the sales and feedback have been less than what we were hoping for, and the rest of the team has downsized themselves to part-time or less as the demands of paying rent and eating regularly started to creep into play. This is the joy of the indie developer. As a team, we've been doing a bit of experimentation with mobile titles (Android in particular), as the barrier to entry is about as low as it gets and we all have hardware. But this post isn't about Android development – I'll save that for later. For myself, I spent most of December doing some serious analysis of the project and decided that instead of embarking on yet another engine development cycle (I've spent the past 10 years building and licensing custom engines in various shapes and sizes), that I would instead spend a month or two prototyping and experimenting with Unity, which in my mind is the hands-down winner of the engine / middleware battle (at least for the time being). The sheer size and depth of the community that has sprouted up around Unity makes it an absolute no-brainer for someone looking to dive off into development, and I have been pleasantly surprised that the toolset itself holds up to detailed inspection in almost every way. Now, three months in, experiments with Unity have been going well, I've started making some good progress on my current prototypes – to the point where I have begun the slow process of building up the framework for the "real" game that has been in the works. Along the way, I've noticed a few things that I thought would be worth writing about – and a few potential caveats and "gotchas" that might be of interest if you are working with Unity yourselves, specifically about the Art Pipeline and in particular, how it works with 3ds Max and other external tools. First off, I have multiple workstations setup: my primary desktop at the office, and a few mobile / portable options. The primary machine that I carry with me is an Acer W500 – a 10″ windows tablet doohickey that has been absolutely great as a portable device. Having a small tablet with a real keyboard and a real operating system is very slick. Say what you will about Android and iTablet things, but when it comes to doing some REAL WORK, they are expensive toys. You can PLAY games on them, but MAKE games? Not any time soon at least. Short version of my setup is that I have Unity setup on the desktop and the tablet, and use Subversion to sync projects. The one downside to the tablet is the (lack of) hard drive space – the whole system drive is all of 32 GB, so I haven't installed Photoshop or 3ds Max on it because there literally is no space for them. After the Windows setup (Windows 8 Dev Preview, which is actually pretty slick once you ditch Metro), the primary drive has all of 15 GB total for all my apps / content etc. Installing any of the "big" apps that make up my typical production flow on the sucker would eat up the majority of that drive. So, for the time being, I've kept most of the big apps on the desktop. Which leads me to the first major "gotcha" that I've run into working with Unity and 3ds Max. Unity's native 3ds Max support IS NOT what it claims to be When you first get started with Unity, one of the first "aha" moments that strikes you is the claim that Unity can load native 3ds Max files. This seems like a dream come true for artists, and for the most part it does seem like it works great – except that it's a trick ;} What Unity does when you import a 3ds Max file is a behind-the-scenes FBX export from Max at load. So every time you refresh your asset tree in Unity, it does a dynamic call to Max in order to get it to export all of your Max files into FBX, and THEN loads the FBX files into Unity. Now this is all fine and great – and for a lot of people (particularly single-developer teams), probably works like a dream. In fact, it's a pretty slick solution for what has always been an annoying Art Pipeline nightmare. UNLESS – you don't have Max installed on your machine. If you don't have Max installed, Unity fails completely and, from the programmer's standpoint, you suddenly have no content in your game. Let's go through the workflow in a bit more detail:

Workstation 1 (Artist Workstation)

Software Installed: - Autodesk 3ds Max - Unity 3D

Workstation 2 (Programmer Workstation)

Software Installed: - Unity 3D (using MonoDevelop)

Artist creates a new 3d model in Max, saves the native .max file in the Assets/Resources directory of the Unity project. Unity loads it up seamlessly on the artist's workstation, artist proceeds to spend hours of time building a fancy scene, populating it, lighting it, etc. At the end of the day right before leaving on vacation for a week, the artist does a commit of the awesome new scene so that the programmers can work on the gameplay while they are gone. Programmer comes in, updates the SVN repository on their machine and sees all this amazing new work that the artist has done coming down the update log. Having heard through the grapevine that this level is a masterpiece, the programmer then proceeds to fire up Unity so they can check out the progress on the game. The scene loads, except there's nothing visible. After poking around a bit, the programmer finally happens to see this tiny message in the status bar: This is the ONLY indication of what might have gone wrong with the scene load. It does give you a bit more info in the console window as well, but if you don't have that visible, this is all you get. Not exactly a verbose message. Now, Unity will still load the scene, and (theoretically) not lose any work – except for the fact that all the programmer sees is basically a bunch of empty Game Objects with no meshes associated with them. Not exactly a productive way to get your gameplay implemented. So if you have a team of say 3 artists and 3 programmers – it doesn't matter – you need to install 3ds Max on EVERY SINGLE MACHINE irregardless because Unity fails unless you have Max installed. What do you mean? Just get your artists to export to FBX? Now there is an easy fix for this of course – simply export your content to FBX in the first place. Ok fine, sure this works. But it also kind of DEFEATS THE PURPOSE of having native loading of Max files does it not? Which is all fine and dandy as well, except that Unity will CONTINUE to import and load the Max files anyways, whether you want it to or not. There is no way to get Unity to "ignore" any files in the Assets/Resources folder – that's the whole point of it – so you're kind of in a chicken and egg position. You want to setup your content so that it's easily Streamable (Note: this isn't 100% true – see the section regarding 'Asset Bundles', below for more clarification), so you put it in the Assets/Resources folder, but you only want Unity to load the actual stuff that you are using. It would be nice if Unity somehow stored the cached mesh files that it generates somewhere useful so that you could check them into version control etc. This would not only speed up the first load of a project infinitely, but would also fix team based development with the system.

With GameCore, we simply saved the cache files for each object in the same location as the source objects – both textures and objects – so you knew immediately what files went with which and, worst case, if you needed to clean / delete OR SAVE them – you could.

When you publish your final game, the engine simply packages up the cache files where available and ignores the source files for that content. I'm not suggesting that this is an ideal solution, but it's definitely a potential solution.

Now Unity has billions of random incomprehensible cache files stored in that libraries folder – including I'm assuming the cache'd versions of the FBX files – but you can't simply check that folder into Subversion apparently (and I'm scared to try). There doesn't seem to be any workaround for this that I can see. You either have Max installed on every single machine you are working on your game with, or you don't use the native Max loading. It's one or the other. I've been told that Unity has some kind of 'Cache' server that helps provide a central repository for your content that might solve this problem, but frankly I don't trust custom version control systems. The last thing you need 90 percent of the way through your project is a corrupt repository (SourceSafe, I'm looking at you!) and a custom solution that is only available for one piece of software is just too new / untried for my liking. No one gets fired for buying IBM (or using Subversion in this case) after all ;P Which brings me to my next point: Unity does not give you any indication of the source file format for a particular object Enter annoyance #2. This one might just be as a result of how I'm used to working, but for the majority of projects I've worked on in the past have enforced a "work in place" policy for art content. What this means is that all of the source textures / source models and other game assets were stored in the same folder as the exported / game assets. This simplifies revisions and ensures that you can find everything later on when you inevitably need to do a patch / update or other work on a project in the future. For people that are used to engines like Unreal that have custom pack files / asset tools, this might not be the case, but all of the engines I've worked with in the past decade have allowed this kind of system. What this means is that for any game asset, you have all of your Max files in the same folder, organized nicely. You export each piece of content to FBX so that your programmers can actually see the content on their end. Unfortunately because Unity will automatically load the .max files, this results in a bit of an unwieldy mess. Those 100 MB Max scenes that you are building to organize your content? Unity loads it automatically, whether you want it to or not. As an example, to the right is a folder for one of my vehicle packs. There are three wrecked cars, in .max and .FBX format. Which is which? Do you know? I don't know (unless I manually look outside of Unity). Does Unity tell us? No – whether it's a Max file or an ACTUAL FBX file, both have 2 components added,an Animation component and an FBX Importer component. In fact this is what led to me finally figuring out why my content wasn't loading on the programmer machine – the fact that both Max files and FBX files have an FBX Importer component applied. So, even if you are exporting to FBX and want to ONLY use FBX files so that your team members can load the scene on their machines – you have to be VERY careful to only load the appropriate content into the scene, because Unity isn't going to tell you. Additionally, whether you are using the native Max files or not – they will always be present in the project folder because Unity forces you to load all of the content in the resources folder, whether you want it to or not. I know, I know, I get WHY they load all of the resources folder content, but seriously, it wouldn't exactly take much to have a flag to NOT include certain objects – they have all of these .Meta files for everything. Why not a simple bit flag for specific pieces of content to IGNORE, even if they are present in the Resources folder? The animation component Another pet peeve that I've been running into – and why I've manually been building all of the prefabs for the content manually – is that Unity automatically adds (for me at least) an 'animation' component to every FBX file that I load into the project. Supposedly there is a way to get it to "not" do this. apparently Unity shouldn't do this if I have the animation element deselected in the FBX exporter, but it's not behaving for me. I'm going to spend some time working on that today to see if I can figure out why. Manually building the prefab to remove the animation component adds unnecessary work. The revised Art / Asset workflow The only solution to the above issues is quite simply to NOT include your Max files in the Unity Project hierarchy at all. As I mentioned above, this goes against a lot of what I feel to be 'best practices' when it comes to asset organization for a project. When it comes to asset organization for a project, over the years, I've spent a LOT of time trying to train (berate / beat) artists into making sure that they are working on their game content IN PLACE – in the project – and NOT on some random location – like their desktop folder or wherever. Having an identical folder structure from one development workstation to another goes a HUGE way towards streamlining the development process. If you forget to commit a piece of art into the game, you KNOW where it is located – in the project folder – and don't have to hunt randomly around the file system for it. I can't count the number of times that I've had to go onto an artist's machine and do content hunting because something wasn't checked into a project.

Interns in particular love doing this - I swear that if there was one thing that so-called "Art schools" should teach – it's content organization.

Work around / Potential Solution – the 'Content' folder As a work-around and potential solution for the above situation is as follows – what I've started doing is a slightly revised folder structure for the game that goes like so:

My new top-level "Content" folder is basically a mirror of the Resources folder that stores the native Max files for the scenes / content that I am producing for the game. All content gets exported from Max to FBX (I have a few max scripts that streamline this process) into the appropriate folder in Objects. The Content folder is intended to store the source .max files for the objects in question and can also be useful for larger source textures (those 300 Meg PSD files that you don't want the engine to load). It has also been suggested that I move the Content folder up one additional folder – i.e. completely out of Unity's reach – just in case Unity attempts to parse / load the directory even though it's outside of the Asset folder. This is definitely an option, and might be a cleaner solution. In this case, the resulting top level directory structure would be something like so:

Where possible, the .max files that I'm working on are parent-level inventory objects that store all related content of a specific type, so all of the trees for the environment are in a single trees.max file, etc. that then get batch-exported into the proper Assets/Resources folder (or custom Asset Bundle folder) for use in the engine. Having a master Max project file helps keep the scale unified and so on. The batch exporter that I use also optionally does a transform so that every object's pivot is moved to the world 0,0,0 on export, so I can actually build and design full levels in Max and export them with their pivot reset properly in the game engine. This gives me the option to either export objects where they were in Max or properly centered, it's up to me. The key is that I can batch export large numbers of objects from Max into the engine with a single click making revisions very quick. Organizing your Content I'm a bit of a content organization freak, so I spent a bit of time (and refactoring art / texture paths as appropriate) and feel that I've also come up a good way to organize the individual content folders for a scene. For example, I'm working on content for an Arab town setting for the game. My default approach would be to simply store all of the models and textures in the same folder and just go from there – saves having to remap textures and dealing with Max's absolute path tendencies, but also makes for some messy directories. The NEW approach that I've started using and what I have setup for the main Arab town folder is like so: Assets/Resources/Maps/Arab-Town This top level folder stores the primary prefabs that I manually build for each object. Assets/Resources/Maps/Arab-Town/Meshes Assets/Resources/Maps/Arab-Town/Textures Assets/Resources/Maps/Arab-Town/Materials Unity automatically creates a 'Materials' folder for an object, whether you want it to or not. So I've decided to adopt this structure and mimic it for my meshes & textures as well. All textures for a scene go into a sub-folder 'textures' and all of the meshes for a scene go into the appropriate 'meshes' folder. This way, whenever I'm adding content to a scene, I know that the proper object that I want to include is the one in the top-level 'prefab' folder – in this case 'Assets/Resources/Maps/Arab-Town' and can be 100 percent sure that the content not only works, but is setup and exported properly so that when I commit the content, it will work across the development network on the programmer's workstations etc. I've only started doing this in the past few days, so I'm not 100 percent sure whether it's going to work completely for all content – I will say that the manual organization does take a little longer than just dumping everything into random folders, but I am fairly confident that the amount of time that it will save in the long run will be well worth it. Streaming content, asset bundles and web deployment (oh my!) Now the folder structure that I have written up above is far from complete. Particularly when looking at Web deployment of your projects (which I am), it seems that Unity treats everything in the "Resources" folder as a single Asset Bundle, that is everything in the "Assets/Resources" folder gets packed up into a single file that the end-user must download before being able to access the content within. For a small game with minimal assets, this might be a useful approach. It's by far the simplest to implement, but as your project gets more complicated and you have more and more content to include, the next step in the content organization process is to begin integrating and working with custom Asset Bundles. I haven't used them myself as of yet, but my goal for organizing the content as I have above, is that I can take a single sub-tree of the content folder – say my "Arab Town" folder – and move it into a custom asset bundle that gets streamed and loaded when the player gets to that particular level. As I continue along the path of this project, I'll get my head wrapped around the asset bundle workflow and report back with what I find. The project I'm working on incorporates both Character customization and procedural environment creation, so being able to stream in and out content on the fly is extremely important. I suspect that I'll become very familiar with how Asset Bundles behave very shortly ;}

Now that I have my art workflow cleaned up and figured out, I've started to dive into procedural generation of the game worlds for the project. I'm planning on using a similar technique to what we used for the last game. I'll write up something about how that works and how you can use something similar next time. I'll also hopefully have a few more screenshots of the project work-in-progress to show off as well. Maybe… Until then… Ideas? Comments? Do you have any Art Workflow tips for working with Unity that you want to share? [This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]