Adding details on a large world that works in VR

Recently I added a new spy mission to Dogfight Elite. The idea was simple: allow players to unmount their plane and find a spy that is hiding inside an abandoned subway tunnel. The problem is that the tunnel is quite large with tons of assets and custom lighting. At the same time, outside the tunnel, you have a large terrain and multiplayer combat going on. This must run at 60 fps on Oculus VR.

How did I solve this?

There were several steps:

1. The usual draw calls, shader, and mesh optimizations. Most items inside the tunnel share textures and materials. They are highly optimized for polygon counts. The items also utilize the same shaders with the exception of the lights. Everything that should be static, is marked as static. Batching was easy to achieve.

But this was not enough. I had to write a custom dynamic loading/unloading system for the assets in memory. The RAM was not enough to load all the assets and Unity culling is not smart enough to avoid rendering distant objects.

2. Based on distance and orientation, load and unload items. Also, enable/disable assets that will require no rendering. I had to create an in-memory grid coupled with some triggers. This way I could achieve things like: when you enter this area, you can stop rendering the water, terrain, and trees. You will not see them when deep inside the subway, but you need to see them when you look out from the large subway entrance.

You can see here a video while developing the dynamic system. When the pilot arrives and enters, it begins to dynamically load the assets while walking inside the tunnel. The assets also unload from memory when you leave those areas. In the next iteration, I also stopped rendering water, trees, and so on, as you go deeper into the tunnel. But I could not unload them from memory for the reasons that I will explain below.

So far it all seemed good. Framerate was good. RAM was good. I could run this even a Kindle Fire 7 at a decent speed. But then I arrived at the hardest problem to solve: hiccups.

In Unity, when you load an asset into memory for the first time there is always a hiccup. It does not matter if you load them asynchronously. Unity has a design flaw that when you add a new GameObject, it instantiates/runs initialization code in the main thread. There is no way around this. The Monobehaviour class is just badly designed for this scenario. You can hide it by preloading or “warming up” different assets and then unloading them. But the problem with this is that you will need a large amount of RAM and make people wait a long time on their loading scenes. In my case, I also have a terrain with trees. When I unloaded the terrain from memory and I loaded it back upon exiting the tunnel, the hiccup was huge. So how can we solve this?

Well, it is certainly a known problem. After years of complaints, Unity decided to address it by creating a new system called DOTS/ECS, or Data-Oriented Technology Stack and Entity Component System. This was what seemed the right approach. Unity developed several demos of the technology, including this one loading/unloading thousands of assets in real-time with no hiccups.

But then, it lost traction and Unity stopped talking about it or releasing updates on its progress. Until recently. You can follow the thread in the forum here:

https://forum.unity.com/threads/dots-development-status-and-next-milestones-march-2022.1253355/

With this announcement, Unity acknowledged this is a standing problem in their design and they will include DOTS as part of Unity’s core. This should be the way to go for large world asset loading now. Development is still in the early stages but you can already use it in your projects. You can follow the roadmap and download the latest packages in:

https://unity.com/roadmap/unity-platform/dots

Secret Photo Folder App fix coming

First things first, if you are one of the thousands of current users coming here to check for the status, I’m pleased to announce that it is fixed and under review by Apple. It should be published in the next 48 hours.

The app should be available as an update in the store. Direct link here: https://apps.apple.com/us/app/secret-photo-folder/id367327563

• 

Now, let me explain the issue.

Apple recently released iOS 15.5 and I had to update many of my apps. If you are familiar with Apple App development, you know that Apple enforces developers to constantly update the apps in the store otherwise they get removed. This is the reason why even though I’ve developed more than 80 apps, you will find only a few in the store. I don’t have time to keep updating all of them and Apple removes them as time passes by.

This system has some advantages, mainly that Apple makes sure your apps are always updated to the current iOS. But it also comes with two serious caveats:

1. Most people do not have the time and resources to keep updating apps that do not really need any updating if not enforced by Apple.
2. Apple constantly deprecates and breaks features with new iOS updates. Even if your app hasn’t changed a bit, it will begin malfunctioning unless rewritten with the newer iOS. This brings me back to point 1 and you enter the endless loop.

Enter my app: Secret Photo Folder.

I developed this app years ago. The app allows you to import photos and videos, organize them into folders, add descriptions and keep them hidden with a password and outside your camera roll. It has millions of downloads and it worked perfectly. No issues whatsoever with the app. Well, no issues until Apple released the iOS 15 update. With the iOS update, I begin receiving emails that the “forgot password” option is not working any longer. Apple decided that when you press “face id” to identify yourself, you have to add a text saying something as silly as “We require Face Id to identify you”. As if pressing “face id” was not clear enough of what was about to happen.

So I load my Xcode, modify the app to add that line, compile and… errors. Tons of them. Apple has deprecated many of its APIs that handle photos and videos. And my app is mainly a photos and videos app! I find myself in a bad place: my app is working fine, but if I don’t update it to the latest iOS, it will be removed soon. On the other hand, to update it, I need to spend weeks developing and rewriting thousands of lines of code to use their new APIs. Doing the latter means I need to retest everything almost as if it was the first time I wrote this app.

After some thought, knowing that this app is used by many people I decided to bite the bullet. Weeks later I have the update ready and submit it to Apple. And guess what… that same day the app is updated, iOS 15.5 update hits the public and they included several more bugs. In particular, the app is not even able to import photos!

I’ve spent the last 3 days upgrading again the code to the latest iOS. The update is ready and submitted but the users (thousands of them) are not happy (rightly so) with the current version. I’ve gotten over 1000 emails and reviews in the app are horrible. I hope Apple releases the update asap. The problem is that Apple keeps doing this release after release. If you check the app reviews throughout the years, you will see reviews complaining about “photos can’t be imported anymore” or “colors changed”. In 2017, 2019, and now 2022. Ironically, nothing changed in the app. It was Apple making changes that broke the app with its iOS updates. Most developers give up and their apps get delisted. Myself included, I’ve let over 50 apps get delisted at this point because I don’t have enough hands to keep up. I will continue updating Secret Photo Folder because it is used by many, but oh boy! it gets hard!

If you are a developer and are having a similar experience, let me try to help you out a bit.

This code now fails to work in iOS 15.5:

[PHAsset fetchAssetsWithLocalIdentifiers:@[assetId] options:nil];

You will probably get some errors when importing Photos or Videos. Instead you need to use the url with:

[PHAsset fetchAssetsWithALAssetURLs:@[_assetURL] options:nil];

This is crazy because the last line of code is now deprecated. In theory, the correct line is the first one. I will expect Apple to fix it in the next iOS update, but for now we have no choice.

Another issue, is that the photos and videos showed up rotated. The reason is because this line does not create images as they used to:

image = [UIImage imageWithCIImage:image];

Now you need to specify the scale and orientation. Otherwise you will find most of your photos loading up in landscape. The correct method now:

image = [UIImage imageWithCIImage:image scale:1 orientation:orientation ];

Lastly, videos used to be able to load and import them with a simple call to:

PHImageManager requestImageDataForAsset:self.asset options:options resultHandler:^(NSData *imageData, NSString *uti, UIImageOrientation orientation, NSDictionary *info)

This will only work for images with the iOS update. To import videos you need to handle them independently with:

PHImageManager requestAVAssetForVideo:self.asset options:nil resultHandler:^(AVAsset *asset, AVAudioMix *audioMix, NSDictionary *info) {
 
AVURLAsset* urlAsset = (AVURLAsset*)asset;
                                NSNumber *size;
                                [urlAsset.URL getResourceValue:&size forKey:NSURLFileSizeKey error:nil];
                                NSData *imageData = [NSData dataWithContentsOfURL:urlAsset.URL];

                   

And then, save that imageData to the destination folder. Happy development!

Dogfight Elite released with full Oculus support!

With this update, Dogfight Elite now runs on Oculus Quest 2 with full support. In addition, it also works with any other standard VR glasses if you download the game in Windows Store or Steam. I made a short video recorded with my Oculus Quest.

You can download it from the Oculus store to check it out, or as usual from any other app store by searching for Dogfight Elite

The game is currently available in Steam, Google Play, Amazon, Apple, Microsoft Store, Mac and PC.

Unity URP vs Built-in.

I spent a few weeks “upgrading” my Unity 3D game from built-in to URP, using Unity 2021.2 and 2022.1 and the experience in URP has been pretty bad. I wasn’t expecting to find so many issues when URP is already in version 14, but it’s a disaster and I can’t recommend anyone to migrate their projects to URP as of April 2022. Here is a summary of the issues that I found and that I directly reported to Unity.

I decided to port Dogfight Elite www.dogfightelite.com to URP, and as my test device, I used a Kindle Fire 7. This device is known to be slow, but it is my base test device for most of my games: if they run there, I know they run practically anywhere. And my game runs at a decent 30 fps in the Kindle Fire 7 when compiled in built-in. I was expecting URP would be able to run it at the same speed or faster.

For the built-in system, I used mobile/diffuse for almost every shader. For the water, I use Lux Water shader from the asset store.

In URP I used mostly URP Simple Lit, with only a custom shader for the water (I tried with Lux Essentials and Stylized Water 2 with no noticeable FPS difference).

I also added the option to lower the quality to improve performance, when I do this, the water is completely removed so as to compare apples to apples and all shaders are either mobile/diffuse or URP Simple Lit.

The game also has a few thousand trees created with Unity Tree Creator. A 4km x 4km terrain. No grass.

Here you can see a video of the game in 2015 shot from a mobile phone. As you can see, the FPS in built-in was pretty decent and high quality even on phones back then.

Results comparing URP vs Built-in in a Kindle Fire 7 (2019 edition):

1) Full resolution and highest game settings (real-time lighting, shadows, and water).
– In built-in: 30 fps. The maximum. It will fall under 30 fps only when many players are on screen.
– URP: It won’t load. It runs out of memory. The profiler shows that 2/3 of the RAM is going into the URP Shaders. Over 350MB of RAM is used up by the shaders.

2) Lowest quality settings, full resolution. (I disable real-time lighting, no shadows and I remove the water completely).
Builtin: 30 fps. The maximum. It never goes under 30 fps.
URP: It loads. It runs at 9 fps. The RAM requirement is almost double the built-in requirement, this time the shaders are taking about 200MB of RAM.

3) Lowest quality settings, half resolution.
Builtin: Same as step 2. It never goes under 30 fps.
URP: It runs at 14 fps. Same RAM as step 2.

Some might argue that a Kindle Fire 7 (2019 edition) is a horrible device. It is by today’s standards. But it runs my game perfectly fine at full speed and with real-time lighting when using Built-in. And this game used to run on an iPhone 4 when Unity4 was out. URP performance and RAM consumption is horrible compared to Built-in as of today.

Problems I encountered while porting and that I did not expect:
1) URP shaders take a HUGE amount of RAM. You have to manually do some serious shader stripping and even after all the “optimized” settings, the RAM usage is off the charts.

2) URP Terrain shader loads about 4 other shaders in memory. Including URP Lit, not even Simple Lit. It totals about 150MB in RAM only for the terrain shaders. My 4km x 4km terrain with 4 textures takes about 30MB of RAM. The terrain shader alone takes 5 times more!
I tried using Microsplat and Lux Essentials terrain shader. Their URP version seems to run at the same fps and consume the same RAM because internally they are loading the URP Terrain shader too.

3) URP shaders perform similarly in speed to the Standard shaders in built-in. But they are ultra-slow compared to mobile/diffuse. In built-in, if we wanted performance, we never used the standard shaders, we switched to mobile versions. We don’t even have that option in URP so we are stuck with slow PBR shaders.

4) Tree creator is not supported in URP. One of the most wonderful things about Unity which I never complained about, is that it was able to render thousands of those trees even in an iPhone 4. URP forces us to use Speedtree, which is notoriously slow on mobile. (I’ve always made fun of the “speed”tree name).

5) I encountered tons of issues rendering sky and weather in a performant way. The shader is either slow or takes too much ram. The available packages in URP are not really optimized for mobile. I had to spend lots of time writing my own shaders and optimizing everything I purchased in the store. I was able to x3 the performance of Unistorm and other packages from their default settings, but they were all still rendering slower than their built-in counterparts.

6) Terrain rendering in URP is even slower than Built-in. Which was already pretty bad. Also, URP Terrain in mobile looks horrible due to long-standing bugs.

7) Lens flares and sun flares. They sometimes work, they sometimes don’t.

8) Shadows. I had to spend a crazy amount of time to get real-time shadows for mobile in URP to look decent compared to built-in and at worse performance.

9) Water. It always amazes me that we have so many water packages in the store. The community ocean package (which is free) was rendered at the highest quality in an iPhone 5 and looked fantastic. Unfortunately, nobody maintained that package and you have to spend money in URP to achieve 1/4 the same quality at half the performance and triple the memory requirements in 2022 URP vs 2015 Built-in. I just don’t get it.

10) Vulkan in URP. In almost every test I’ve done on my Android devices, Vulkan in URP performs worse than Opengles 3.0, to the point I gave up the fight and I just don’t use Vulkan and force it to build for Opengles 3.0. This was especially bad in my Oculus Quest 2 tests which are supposed to have tons of benefits in Vulkan. I experienced no benefits whatsoever.

11) I had to spend a ridiculous amount of money to migrate some assets to their URP version. Just to find out that the non-URP version was performing much better. But you won’t get a refund… so it’s a huge money pit to migrate just to “check”.

Conclusion: As of today, migrating your project from built-in to URP seems to offer no benefit whatsoever. I cannot think of anything you can do in URP and you cannot achieve in built-in, faster, and with fewer memory requirements when developing for mobile at this moment. Meanwhile, you can see all the issues and troubles I found while doing the migration.

I also did all the tests in Unity 2021.2.19f, Unity 2022.1.15b, and Unity 2022.2.9a and I found not much difference or improvements between the versions. I’ve also submitted several bug reports with the things I was encountering, I hope Unity pays attention to them.

URP has still a long way to go in performance and RAM optimization as of April 2022 just to catch up with what we already had in built-in.

Analisis de AENA

Value Investing: Mito y Realidad

 

Explicacion del PER y su utilidad

 

Impacto de las FAAMG en el S&P500

 

Actualizacion sobre las ideas del 17 de Marzo de 2020

Os dejo un video aqui hablando sobre las ideas que propose el 17 de Marzo de 2020

Entrevista en Finect sobre como invertir en tiempos de crisis