Adding details on a large world that works in VR

Recently I added a new spy mission to Dogfight Elite. The idea was simple: allow players to unmount their plane and find a spy that is hiding inside an abandoned subway tunnel. The problem is that the tunnel is quite large with tons of assets and custom lighting. At the same time, outside the tunnel, you have a large terrain and multiplayer combat going on. This must run at 60 fps on Oculus VR.

How did I solve this?

There were several steps:

1. The usual draw calls, shader, and mesh optimizations. Most items inside the tunnel share textures and materials. They are highly optimized for polygon counts. The items also utilize the same shaders with the exception of the lights. Everything that should be static, is marked as static. Batching was easy to achieve.

But this was not enough. I had to write a custom dynamic loading/unloading system for the assets in memory. The RAM was not enough to load all the assets and Unity culling is not smart enough to avoid rendering distant objects.

2. Based on distance and orientation, load and unload items. Also, enable/disable assets that will require no rendering. I had to create an in-memory grid coupled with some triggers. This way I could achieve things like: when you enter this area, you can stop rendering the water, terrain, and trees. You will not see them when deep inside the subway, but you need to see them when you look out from the large subway entrance.

You can see here a video while developing the dynamic system. When the pilot arrives and enters, it begins to dynamically load the assets while walking inside the tunnel. The assets also unload from memory when you leave those areas. In the next iteration, I also stopped rendering water, trees, and so on, as you go deeper into the tunnel. But I could not unload them from memory for the reasons that I will explain below.

So far it all seemed good. Framerate was good. RAM was good. I could run this even a Kindle Fire 7 at a decent speed. But then I arrived at the hardest problem to solve: hiccups.

In Unity, when you load an asset into memory for the first time there is always a hiccup. It does not matter if you load them asynchronously. Unity has a design flaw that when you add a new GameObject, it instantiates/runs initialization code in the main thread. There is no way around this. The Monobehaviour class is just badly designed for this scenario. You can hide it by preloading or “warming up” different assets and then unloading them. But the problem with this is that you will need a large amount of RAM and make people wait a long time on their loading scenes. In my case, I also have a terrain with trees. When I unloaded the terrain from memory and I loaded it back upon exiting the tunnel, the hiccup was huge. So how can we solve this?

Well, it is certainly a known problem. After years of complaints, Unity decided to address it by creating a new system called DOTS/ECS, or Data-Oriented Technology Stack and Entity Component System. This was what seemed the right approach. Unity developed several demos of the technology, including this one loading/unloading thousands of assets in real-time with no hiccups.

But then, it lost traction and Unity stopped talking about it or releasing updates on its progress. Until recently. You can follow the thread in the forum here:

https://forum.unity.com/threads/dots-development-status-and-next-milestones-march-2022.1253355/

With this announcement, Unity acknowledged this is a standing problem in their design and they will include DOTS as part of Unity’s core. This should be the way to go for large world asset loading now. Development is still in the early stages but you can already use it in your projects. You can follow the roadmap and download the latest packages in:

https://unity.com/roadmap/unity-platform/dots

Advertisement

Unity URP vs Built-in.

I spent a few weeks “upgrading” my Unity 3D game from built-in to URP, using Unity 2021.2 and 2022.1 and the experience in URP has been pretty bad. I wasn’t expecting to find so many issues when URP is already in version 14, but it’s a disaster and I can’t recommend anyone to migrate their projects to URP as of April 2022. Here is a summary of the issues that I found and that I directly reported to Unity.

I decided to port Dogfight Elite www.dogfightelite.com to URP, and as my test device, I used a Kindle Fire 7. This device is known to be slow, but it is my base test device for most of my games: if they run there, I know they run practically anywhere. And my game runs at a decent 30 fps in the Kindle Fire 7 when compiled in built-in. I was expecting URP would be able to run it at the same speed or faster.

For the built-in system, I used mobile/diffuse for almost every shader. For the water, I use Lux Water shader from the asset store.

In URP I used mostly URP Simple Lit, with only a custom shader for the water (I tried with Lux Essentials and Stylized Water 2 with no noticeable FPS difference).

I also added the option to lower the quality to improve performance, when I do this, the water is completely removed so as to compare apples to apples and all shaders are either mobile/diffuse or URP Simple Lit.

The game also has a few thousand trees created with Unity Tree Creator. A 4km x 4km terrain. No grass.

Here you can see a video of the game in 2015 shot from a mobile phone. As you can see, the FPS in built-in was pretty decent and high quality even on phones back then.

Results comparing URP vs Built-in in a Kindle Fire 7 (2019 edition):

1) Full resolution and highest game settings (real-time lighting, shadows, and water).
– In built-in: 30 fps. The maximum. It will fall under 30 fps only when many players are on screen.
– URP: It won’t load. It runs out of memory. The profiler shows that 2/3 of the RAM is going into the URP Shaders. Over 350MB of RAM is used up by the shaders.

2) Lowest quality settings, full resolution. (I disable real-time lighting, no shadows and I remove the water completely).
Builtin: 30 fps. The maximum. It never goes under 30 fps.
URP: It loads. It runs at 9 fps. The RAM requirement is almost double the built-in requirement, this time the shaders are taking about 200MB of RAM.

3) Lowest quality settings, half resolution.
Builtin: Same as step 2. It never goes under 30 fps.
URP: It runs at 14 fps. Same RAM as step 2.

Some might argue that a Kindle Fire 7 (2019 edition) is a horrible device. It is by today’s standards. But it runs my game perfectly fine at full speed and with real-time lighting when using Built-in. And this game used to run on an iPhone 4 when Unity4 was out. URP performance and RAM consumption is horrible compared to Built-in as of today.

Problems I encountered while porting and that I did not expect:
1) URP shaders take a HUGE amount of RAM. You have to manually do some serious shader stripping and even after all the “optimized” settings, the RAM usage is off the charts.

2) URP Terrain shader loads about 4 other shaders in memory. Including URP Lit, not even Simple Lit. It totals about 150MB in RAM only for the terrain shaders. My 4km x 4km terrain with 4 textures takes about 30MB of RAM. The terrain shader alone takes 5 times more!
I tried using Microsplat and Lux Essentials terrain shader. Their URP version seems to run at the same fps and consume the same RAM because internally they are loading the URP Terrain shader too.

3) URP shaders perform similarly in speed to the Standard shaders in built-in. But they are ultra-slow compared to mobile/diffuse. In built-in, if we wanted performance, we never used the standard shaders, we switched to mobile versions. We don’t even have that option in URP so we are stuck with slow PBR shaders.

4) Tree creator is not supported in URP. One of the most wonderful things about Unity which I never complained about, is that it was able to render thousands of those trees even in an iPhone 4. URP forces us to use Speedtree, which is notoriously slow on mobile. (I’ve always made fun of the “speed”tree name).

5) I encountered tons of issues rendering sky and weather in a performant way. The shader is either slow or takes too much ram. The available packages in URP are not really optimized for mobile. I had to spend lots of time writing my own shaders and optimizing everything I purchased in the store. I was able to x3 the performance of Unistorm and other packages from their default settings, but they were all still rendering slower than their built-in counterparts.

6) Terrain rendering in URP is even slower than Built-in. Which was already pretty bad. Also, URP Terrain in mobile looks horrible due to long-standing bugs.

7) Lens flares and sun flares. They sometimes work, they sometimes don’t.

8) Shadows. I had to spend a crazy amount of time to get real-time shadows for mobile in URP to look decent compared to built-in and at worse performance.

9) Water. It always amazes me that we have so many water packages in the store. The community ocean package (which is free) was rendered at the highest quality in an iPhone 5 and looked fantastic. Unfortunately, nobody maintained that package and you have to spend money in URP to achieve 1/4 the same quality at half the performance and triple the memory requirements in 2022 URP vs 2015 Built-in. I just don’t get it.

10) Vulkan in URP. In almost every test I’ve done on my Android devices, Vulkan in URP performs worse than Opengles 3.0, to the point I gave up the fight and I just don’t use Vulkan and force it to build for Opengles 3.0. This was especially bad in my Oculus Quest 2 tests which are supposed to have tons of benefits in Vulkan. I experienced no benefits whatsoever.

11) I had to spend a ridiculous amount of money to migrate some assets to their URP version. Just to find out that the non-URP version was performing much better. But you won’t get a refund… so it’s a huge money pit to migrate just to “check”.

Conclusion: As of today, migrating your project from built-in to URP seems to offer no benefit whatsoever. I cannot think of anything you can do in URP and you cannot achieve in built-in, faster, and with fewer memory requirements when developing for mobile at this moment. Meanwhile, you can see all the issues and troubles I found while doing the migration.

I also did all the tests in Unity 2021.2.19f, Unity 2022.1.15b, and Unity 2022.2.9a and I found not much difference or improvements between the versions. I’ve also submitted several bug reports with the things I was encountering, I hope Unity pays attention to them.

URP has still a long way to go in performance and RAM optimization as of April 2022 just to catch up with what we already had in built-in.