Unity Issues

Ripper said:

My point is that different skill sets generally have different values in the market, and therefore different costs - not that there are no talented developers working at lower budget productions. If, as an example, you want an experienced C++ graphics programmer to work on a modern 3d Vulkan renderer for your game, that is almost certainly going to be more expensive than someone that can build levels in Unity and do some C# scripting. That latter guy might be a brilliant young talent of game design working his first job, but, by and large, that hardcore technical skill is going to come at a high hourly cost. And, in relation to what we've been talking about, the more hardcore, low-level work you need to do, that's generally going to push the budget up - to a point that most indies won't want or be able to go there.

Click to expand...

It's a little bit like the gun vs bow argument (TLDR: even if bows were more efficient, guns required less training and progressively replaced them in warfare), and not the first time I see such a shift to get resources more easily.

Redglyph said:

It's a little bit like the gun vs bow argument (TLDR: even if bows were more efficient, guns required less training and progressively replaced them in warfare), and not the first time I see such a shift to get resources more easily.

Click to expand...

Yes, and I think this was the idea behind the Xenko engine (now Stride). Sadly, Silicon Studio open-sourced it, but also abandoned it, and it hasn't really picked up much of a community. Which is a pity, because I thought the central idea was a good one - let's have an engine that really is C# all the way down. Yes, it wont be quite as performant as C++, and obviously it has to be designed cleverly to prevent GC issues in the fundamentals of the engine, but the trade off is making an engine much more approachable to a lot more devs, and perhaps more efficient for studios.

I think you must have missed my last point Ripper - about how we're talking about different things.

I'm not saying GC in Unity is inherently bad. Not at all, as I wouldn't know. I'm saying it's been quoted as the reason for performance issues (for obvious reasons - as it halts everything else until it's done). In the same way - I could say the sun is the reason for sunburns - but that doesn't mean it's smart to stand naked outside without sun blocker.

If the official warnings about how to handle Unity GC in specific ways to avoid precisely the kind of performance issues in question isn't evidence enough - I really have no clue what would suffice.

Please remember that Unity provides a bunch of convenient methods for doing things - including how to easily instantiate and destroy objects. The fact that you script using C# - doesn't take away that it's the "Unity way" and how they intend for it to be possible.

That it's apparently also unwise in many cases and will cause problems with GC is hardly all on the developers. At least, I wouldn't say so. It's pretty obvious that many developers will choose the most convenient way to do things - and they're using Unity precisely to cut down on development time in many cases.

Here's a random reddit post, explaining how the Escape from Tarkov developers upgraded to a version of Unity with incremental GC - to fix their previous GC stutters.

https://www.reddit.com/r/Escapefrom...e_were_talking_performance_potential_stutter/

But, really, these posts are all over google - both from developers and tech savvy people.

No, it doesn't mean Unity = bad - it just means it does things in ways that a lot of developers struggle with.

As for your point about specific skillsets often being more expensive, that's not in dispute at all. I'm saying that having a skillset doesn't necessarily translate into being more skilled than people without that specific skillset.

Having a fancy and super expensive Vulkan guy on your team - doesn't mean that guy would necessarily be great at memory allocation or handling GC in Unity, for instance.

Again, your claim that "how skilled" developers are - is largely reflected by the budget, is simply not true. It's pretty self-evident, in my opinion.

Now, if what you meant to say is that senior developers get paid more than rookie developers - I would certainly agree. That's not really what I'm talking about, though.

Maybe it's because I've worked in IT all my life - and I've seen exactly what "senior developer" means in terms of actual skill.

But, as I said, I lack the stamina required to go endlessly in circles about this.

Too much time being spent on pointless stuff - and not at all what I enjoy exchanging about

Anyway, have fun!

https://steamcommunity.com/app/544550/discussions/0/2952595757893968803/

Another fix to GC stutters - by developers guided by Unity guys.

Quote:

Unity directly approached us and we have implemented their recommendations - the primary one of which was running the garbage collected incrementally as the patch includes.

We are continuing normal optimization.

This is new functionality Unity included to specifically help with garbage collection. It has made a great difference for many users.

----

Another developer quote:

[Dev] Nanorock (Programmer) 4 years ago
Hi Kanakotka.

The subject have been discussed countless time, and as such I will be brief here.

1) The game is coded in C#, so there is no "before proper GC"

2) Big project end up using multiple framework, which are black boxes that might generate GC without us easily knowing (Unity being one)

3) You mention foreach for example. It's such a simple language feature yet actually improperly handled by mono and produces GC. It shows that "the bed on which we lie" is not very comfy right. And the list on those incorrect handling goes forever. A simple event subscription ? Mono considers it needs to regenerate the full list of subscriber, producing GC. A simple access to direct property of Unity, like transform ? A GC is produced for marshaling the ref from their native code.

4) You mention leak. People have played for more than 4h straight without OOM. I'm not saying it is not possible to trigger one, I'm sure some unfortunate scenario might, but we can park this one at least.

5) Have you ever tried to profile a game of this scale with Unity tools ? Probably not by the sound of it. The profiling just become unresponsive after a certain amount of nested calls. And due to the code architecture we have, there is quickly too many nesting for it. Not to mention the constant crashing it produces. Or the fact that Memory snap never worked for us.

So we tried cleanup routine. There is no "new" in update, there is no foreach used anymore. We're using pooling as much as we can. We had optimization sprints. I'm not saying we're perfect. There is no doubt room for improvement. But due to the nature and the scale of the project, those point you mentioned are very far from being as simple as you make it sounds like.

I could provide dozens and dozens of these quotes - and I've seen many over the years.

So, a lot of both experienced and less experienced developers struggle with what's very obviously related to GC in Unity. For good reason? I honestly think many of them sound like they're pretty competent developers and coders. Maybe they're not? Who knows.

Just interesting that monodevelop and Unity are part of so, so many performance issues - and the developers all point to GC and how Unity handles it?

I think I've promised to end it a few times - but I guess I couldn't help googling a bit.

But now I'm properly exhausted.

DArtagnan said:

If the official warnings about how to handle Unity GC in specific ways to avoid precisely the kind of performance issues in question isn't evidence enough - I really have no clue what would suffice.

Click to expand...

These are not "official warnings" - this is just Unity trying to explain to devs how to use the C# scripting in a way that won't cause performance issues, which is obviously a possibility with C#. Clearly, the fact that poor C# code can cause GC performance hitches ≠ therefore any performance hitch is likely caused by garbage collection. If you read their documentation on performance troubleshooting, they point out that such issues are often caused by other many things. All I was asking for was actual evidence for the claims that specific hitches in specific games were shown to be due to garbage collection problems. I mean, it may be true in some cases, and that would be interesting to know.

DArtagnan said:

As for your point about specific skillsets often being more expensive, that's not in dispute at all. I'm saying that having a skillset doesn't necessarily translate into being more skilled than people without that specific skillset.

Having a fancy and super expensive Vulkan guy on your team - doesn't mean that guy would necessarily be great at memory allocation or handling GC in Unity, for instance.

Again, your claim that "how skilled" developers are - is largely reflected by the budget, is simply not true. It's pretty self-evident, in my opinion.

Click to expand...

Well, what I'm talking about is relevant, specific skills needed to solve specific problems, and what they tend to cost. A young level designer may not be capable of assessing code and devising the optimal way to handle memory, and he may also be the greatest composer since Mozart - but the question of who's ultimately "more skilled" is kind of meaningless. All I'm saying is that I think you will tend to find fewer man-hours of high-level technical skill available in a smaller, lower budget project than a larger, well-funded one. And if you know you need more of those man-hours to solve problems, you better budget for it.

EDIT: Lol, a couple more posts sneaked in while I was replying. I'll have a look tomorrow.

I'd venture they didn't have to fight against that in '94

If foreach creates and dumps objects, no matter how good the GC is, it will have too much to cope with. But I'm surprised Mono wouldn't implement that efficiently, maybe it's more a problem with what they put in the loop. foreach is the recommended way to iterate through collections after all, it would be strange nobody had seen that before.

EDIT: I'm using "collections" as in "Collection<T>", not garbage collection

Another strange remark about the subscription list, that would mean something as basic as event is badly implemented. But by searching a little, Unity apparently implements their own system in C++, I don't know if that's true.

So for both, either way, it looks like GC wouldn't be the culprit.

As I understand the issue of using C# for game logic, the crux of it is that you don't trust a garbage collector not to cause performance issues, no matter which algorithm it might be using, and the correct approach is essentially to give the GC almost nothing to do if it triggers during realtime play. Job one is to minimise the allocations and de-allocations in the first place. Then, two approaches they discuss in Unity docs specifically are to either go with a large heap, and infrequent collections, ideally manually triggered during menus, cut scenes, etc, or to go other way, and have a very small heap and very frequent collections (which they recommend for FPS and such), so the collections are sub millisecond, and imperceptible. But there's no two ways about it - if we want the benefits of C#, a lot of thought still has to go into memory management, even if it's not being handled explicitly, like C++.

I think part of the problem is that a lot of people learn about C# in the context of desktop programs or other non-realtime applications, where the GC is not such an issue. And then they start putting together code for games without realising that it requires a different approach.

I was curious about that quote from a developer that Dart posted, so I looked up the original context. He's actually having conversation with a Unity developer that I think clearly knows his stuff, and I think it's interesting to read their exchange.

That's disturbing:

Foreach is actually "improperly" handled by the .NET framework Unity uses. To be specific, it is improperly handled from the viewpoint of being used in a game engine; it is excellently handled from the viewpoint of a generic programming language. While patch 5.5(IIRC) made a workaround to prevent boxing usage in foreach clause, the end effect is a less efficient foreach. Keep in mind game programming is extremely abnormal when compared to normal, sequential programming.

Click to expand...

So Mono would rely on .NET for its Windows version, while it would have to implement the code for Linux and other platforms? If I had to write a framework, I'd use the same code on all platforms for consistency. And it would be badly handled by .NET? I'm even surprised a language construct would have anything to do with any .NET framework.

And it would be boxing data? We're not talking about Java, they didn't make the same mistake

EDIT: or is he talking about a ForEach method instead?

Ripper said:

As I understand the issue of using C# for game logic, the crux of it is that you don't trust a garbage collector not to cause performance issues, no matter which algorithm it might be using, and the correct approach is essentially to give the GC almost nothing to do if it triggers during realtime play.

Click to expand...

Apparently. That would be like trying to control the gear selection in an automatic car (I've tried, it's not fun).

So the idea is to use C# because more people know the language and its barrier entry is lower. But at the same time, you have to use it with the perfect knowledge of what's under the hood, and understand what it's doing with memory, so the very concepts they were trying to spare the developers. And if by accident anyone is using it normally, it's prone to generate stuttering and other performance issues.

It sounds like a bad deal

Maybe it's not so bad, in practice. For that sort of issues, it's perfectly possible to have a pool of unaware developers, just give a few basic rules, and have a specialist profile the code, to chase down and optimize the critical GC problems. It's typical development practice.

Well, to use the automatic car anology, I think using C# for game programming and being surprised by GC problems, is a bit like driving an automatic, being startled that the gears change by themselves, and driving into a hedge.

But I don't think it's a crazy thing to do. Cryengine does scripting in C#, so does Godot - and the same considerations apply. At the end of the day, I think coding for smooth realtime performance in games is difficult. Using C++ presents plenty of its own challenges, and for many developers, they trade the difficulties of manual memory management for the relative ease of automatic, but realising that they'll have to adopt certain different practices to avoid the new problems of the GC kicking in.

I don't think it's case of using C# in a way that is not "normal", but just using it in a way that is appropriate for realtime performance. I think that when inexperienced devs (or devs not used to those languages for games) use it without understanding those considerations, they wonder why the GC isn't miraculous. But that's where you do need people that understand the advantages and issues of the tools they're using.

BTW, when that guy was referring to dotNET, I think he meant Mono. AFAIK, unity has used Mono since way back.

C++ has its challenges, I love the language (or loved it, it's been a long time) but I've been spoiled by C#, Kotlin and Python

I suppose that using a lot of new and delete has an impact on performance too, after a while.

Yeah, by "not normal" I mean you have to avoid some patterns. It's common to use immutable objects to avoid reference conflicts, I suppose that's one of the practices that could quickly lead to performance issues in games.

MOVED POSTS

[In relation to bugs in Pathfinder]

Is there any evidence of memory leak problems, beyond anecdotes and hearsay?

Here's the mod for Kingmaker.

https://www.nexusmods.com/pathfinderkingmaker/mods/39

How it works

• The largest amount of memory uses the statistic.json file. The mod watches him and does not allow to grow above a limit.
• The mod adds a new "Clean" button to the loot window when you leaving any zone. It removes all loot and NPS corpses from current scene, that will further help to reduce size of your saves.
• Additional tools in the mod options allow to remove useless NPC such as: citizens, animals, frozen NPC. This tool reads the save file, so to display the current information you need to make a save. Crucially, do not remove unique NPC, they may be necessary for quests. Also, do not remove the hidden creatures, they can be involved in future quests.

Click to expand...

The mod says "Mod reduces size of save files and slightly reduces memory usage in game." Sounds like save file bloat, if anything. Not the same thing as a memory leak.

Mind you, we can't assume that modders are correct. BS around memory management has always been rampant. I once I had it out with a guy touting a "memory freeing" app - he threatened to sue me and the Nexus.

You know this is why I don't reply or engage you on most threads.

To be fair with Ripper, it probably also has to do with the way C# deals with objects and how it chooses to "forget them" (or not to, in this case), which just keeps clogging up memory and making every process slower. Because Unity is based on C# it just inherits this issue, so games based on it will have a harder time than if they were coded with, say, C++.

A clever thing to do would be going the Rimworld way, which uses Unity, but it's not coded in C#, it only uses it for the graphics, and that's why that game can run smoothly even through the longest games and sessions.

Nereida said:

To be fair with Ripper, it probably also has to do with the way C# deals with objects and how it chooses to "forget them" (or not to, in this case), which just keeps clogging up memory and making every process slower. Because Unity is based on C# it just inherits this issue, so games based on it will have a harder time than if they were coded with, say, C++.

A clever thing to do would be going the Rimworld way, which uses Unity, but it's not coded in C#, it only uses it for the graphics, and that's why that game can run smoothly even through the longest games and sessions.

Click to expand...

Wasteland 2, Tyranny, and a few other Unity RPGs all had the same memory leak and save bloat problems. So you might be onto something about the C# code problem.

Wasteland 2 was patched and another problem with unity is with GPU fans & overheating. It happens less nowadays but I still see reports on that problem.

TBH, if it's a problem with save bloat, that's really a problem with the way the game's coded, that would apply no matter what language or engine is used - they're just trying to hold on to too much data, and need to come up with some better routines for pruning what's not essential.

Ripper said:

Is there any evidence of memory leak problems, beyond anecdotes and hearsay?

Click to expand...

Yes there is.
If you play the game for a long time, eventually the game stops reacting to ESC key press as it has no memory left for keyboard interrupts.
Another issue also after long time playing, eventually you enter some area and see nothing but a black screen as the game can't load the area anywhere any more (no, this isn't connected to blackscreen cutcsene bug).
Finally, also after long playtime, dialogues start to microfreeze.

I do not care about potential savebloat issues as everyone has SSD in their PC today.