Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Is anyone doing anything with 3d modeling for VR?  (Read 2930 times)

LordBucket

  • Bay Watcher
    • View Profile
Is anyone doing anything with 3d modeling for VR?
« on: September 05, 2015, 05:57:38 am »

So VR is going to be a thing here pretty soon. Unreal Engine and Unity seem to be the preferred environments. I've recently downloaded both and done some dabbling, but both seem abysmally horrificly nightmarishly designed for graphic artists rather than programmers. Creating models is a nightmare. Animating them is a nightmare. Both engines seem to heavily favor scripted, static animations rather than animations controlled by code. The mesh/texture/bone design seems like whomever made it expected 3d modeling to be done exclusively by teams of people with poor communication rather than one person.One guy does meshes. One guy does textures. And never shall the two speak to each other. And incompatibility issues are terrible. The amount of effort required to download a model and turn it into anything usable is ridiculous. Taking a Gary's mod or Skyrim model and getting it into Unreal or Unity seems to require 2-3 intermediary steps through various 3rd party converters and even then I always seem to end up with something mutilated beyond  usability.

The whole of 3d modeling, from what I've seen, seems like it was designed deliberately to be difficult. I was able to download the android SDK and in an hour or two get a "hello world" app up and running on my tablet in an hour or two despite having never worked with java before. With this, even after a week or two of casual dabbling, I still don't have anything to show for it.


But, VR seems like it's going to be big. And having the ability to create my own experiences seems especially awesome. I would dearly love, for example, to make Summer Lesson with Twilight Sparkle as the tutor. The code logic for such a thing would be completely trivial. But the 3d modeling seems to present a huge barrier.

Is anyone here doing anything with VR? Or planning to?

itisnotlogical

  • Bay Watcher
  • might be dat boi
    • View Profile
Re: Is anyone doing anything with 3d modeling for VR?
« Reply #1 on: September 05, 2015, 02:32:07 pm »

Modelling, as in using a 3D app to create things, isn't really the difficult part. In fact, using your actual hands to create a mesh would be MORE difficult, since human hands are the shittiest and most imprecise tools ever invented and you don't even have the feedback of an actual object underneath your fingers to help you out.

3D modelling isn't designed to be difficult, it's just a skill like anything else. Almost all 3D apps are the same, it's just a matter of the features they offer and the price tag. The "easy" solutions are either completely unsuitable for realtime games (Photo-to-model? Enjoy doing a lot of manual retopo anyway) or severely lacking in features to the point you might as well not bother (a lot of Fuse clothing/hair is pretty generic, and the automatic rigs have trouble in the shoulder area).

Taking a Gary's mod or Skyrim model and getting it into Unreal or Unity seems to require 2-3 intermediary steps through various 3rd party converters and even then I always seem to end up with something mutilated beyond  usability.

I've never done Gary's Mod or Skyrim modding, but I imagine that your trouble is because they come in a format used specifically by their respective game engines. Unity supports plenty of standard model types, even .blend (and virtually NOTHING supports .blend files).

If it makes you feel better, I've been trying to model for about five years and the best I can manage is some simple furniture.
« Last Edit: September 05, 2015, 02:44:03 pm by itisnotlogical »
Logged
This game is Curtain Fire Shooting Game.
Girls do their best now and are preparing. Please watch warmly until it is ready.

mesilliac

  • Escaped Lunatic
    • View Profile
Re: Is anyone doing anything with 3d modeling for VR?
« Reply #2 on: September 05, 2015, 03:53:13 pm »

I've been doing a lot of work with VR recently, and while what you say about Unity and UE4 is true (favouring scripted animations rather than code driven, designed for teams of specialists who don't communicate much, etc), a single person can learn to use them well. But if you're using one of these frameworks you have to learn to fit your problem into their method of solution, which can be a pain.

If you prefer to have something entirely controlled by code, it is also quite possible to do away with them entirely and write your own renderer using DirectX or OpenGL, but this takes a lot more specialized knowledge than learning to model. I actually did this myself using OpenGL, with Assimp for model loading, FreeImagePlus for HDR image loading, OpenCV for video loading, and the Oculus Rift SDK for display. It was fun, but i had to write an entire rendering pipeline from scratch.

Maybe a middle ground would be using something like Ogre3D? But i've never used it myself so couldn't say...

From my own experience with 3D modelling, the biggest thing needed to learn it is persistence. I recently finally got the hang of dealing with modelling in Blender on my 6th attempt. Every other attempt ended in "WTF this program is so complicated what is all this stuff, OMG what i want to do is so simple" and quitting. But this time i finally learned enough about what i didn't need to only focus on what i did need.

So i'd say, stick with it in Unity or UE4. Learn to tweak the models and animations yourself with Blender, then import them from there. It takes some getting used to, but if you can get comfortable with the tools. the workflow should all be quite smooth, and you'll find that the way to achieve what you want to do with these tools becomes more and more apparent.
Logged

Araph

  • Bay Watcher
    • View Profile
Re: Is anyone doing anything with 3d modeling for VR?
« Reply #3 on: September 07, 2015, 11:41:34 pm »

Fuck yes, a topic that I am useful in. Maybe.

Animations in Unity (because I'm not experienced with Unreal)
Unity uses Mecanim a really, really convenient animation handling system. Yes, it's designed for animators rather than programmers because the vast majority of games don't need script-driven animations. It DOES support inverse kinematics (allowing hands to be accurately placed on ledges, feet to land on slopes realistically, etc), as well as allow the developers to construct elaborately blended, variable-driven systems through blend trees.

For example, a character could have an idle animation and a walk-forward animation that are blended together based on a script that sets the current forward velocity, and then that blended animation could be blended between two MORE animations to turn left or right depending on the character's change in rotation. Not particularly script-heavy, because that would suck to have to recreate that every time you started a new project (or even once to be recycled for multiple projects).

I'll get to script-driven animations in a bit.

Creating models and animations
Game engines aren't built for animating, and they really aren't built for modelling. That should all be done in a variety of other programs (Maya, Blender, Mudbox, zBrush, xNormal, etc). A normal workflow might work something like this:

  • Simple mesh created in Autodesk Maya
  • Mesh gets UV mapped for textures
  • Mesh gets subdivided and sculpted in Mudbox
  • Sculpted mesh gets painted
  • Painted mesh gets exported in two forms: a high-res version for later use and a low-res version for retopologizing
  • Low-res mesh is brought back into Maya and has a new, simplified mesh projected over it (this is called retopologizing)
  • Retop-ed mesh gets UV mapped again
  • Retop-ed mesh gets exported into xNormal along with the high-res mesh from earlier, and texture, normal, and ambient occlusion maps are baked from the two
  • Retop-ed mesh gets put into Unity along with atlased texture and normal maps

...In other words, a whole lot of steps because the world is complicated and that's just how it is because that's the only way it works.

If you need the model animated, you have to add in rigging and animation, which are both a whole 'nother can of worms. It's all just skills (lots of skills, unfortunately). There's no way around it. If you want to download models instead of making them from scratch, look for .fbx files for Unity and you can literally click-and-drag them into a scene. USING the models is dead simple.

Using models from Source Engine games or Skyrim
As it happens, I've done both of those. Skyrim models were easier: there's a program that reads the .bsa files from Elder Scrolls games and lets you take out files. You can open up a .bsa, copy models and relevant texture files to your computer, run them through another program, and get a usable file from them (can't remember if it's .obj or .fbx or something different entirely, though). Plug that into Maya, Blender, or 3ds Max and export it from there with the correct settings (don't try to put them directly into the engine because it almost certainly won't work).

Source models... I know GCFScape is the way to open up .vpk files. I somehow got a Spy model from TF2 into Unity, complete with textures and animations, but that nearly two years ago. I remember using Blender, but I don't know if I had to find a plugin or if they came in an importable format right off the bat.

I remember it being a pain in the ass, though. Once they were in Unity, it was all sunshine and rainbows. Getting them there... not so much.

Script-driven animations for VR
When I heard about ControlVR (that VR glove/arm thingy), I wanted to make a very specific game. You would control a wizard and duel other players by casting spells with gestures. I wrote a system to control the hand animations, relying on the fact that Unity handles each bone as a seperate transform - 'transform' meaning a combination of position, rotation, and scale (in a hierarchy, but the script didn't care about that). The script had a series of variables that tracked each bone (hand, index_1, index_2, index_3, middle_1, middle_2, etc) and adjusted the rotations based on arbitrary inputs (I left it fairly open because I didn't know exactly how the inputs would work).

In practice, it made a pretty cool system. I could cast fireballs and create shields based on the script reading the gestures (specifically, comparing the rotations of each bone to predefined spells).

I then proceeded to completely forget about it once the initial wow-factor of VR faded from my mind.



To sum up this entire rambling essay of dumb bullshit:

Everything requires complicated skills, but they're not that bad. Unity has a very effective and logical animation system once you get more into it. You can do anything you want if you're willing to script it, but it takes effort. If you want the easy way, use the built-in systems... which require learning skills to effectively use.
Logged