Investigate insufficient memory cleanups on level unload

[xezon]

As seen in GameLogic::reset, the m_objVector does not shrink on level unload.

	m_objVector.clear();
	m_objVector.resize(OBJ_HASH_SIZE, NULL); // 8192

I expect it is not intended to not shrink memory back to initial levels on reset. It means that the maximum memory allocated from a long match session will carry over to all subsequent match sessions.

In case of m_objVector we are looking at 400+ kb memory savings per 1 hour match or so.

This is probably a general issue across many classes and should be optimized.

[Mauller]

Calling shrink to fit after resizing the vector should cause it to free unused memory.

	m_objVector.clear();
	m_objVector.resize(OBJ_HASH_SIZE, NULL);
	m_objVector.shrink_to_fit();

[xezon]

Yes. And if we want this to be VS6 compatible then we can use swap.

[Mauller]

Yes. And if we want this to be VS6 compatible then we can use swap.

So we might want to ifdef this

m_objVector.clear();
m_objVector.resize(OBJ_HASH_SIZE, NULL);
#if defined(_MSC_VER) && _MSC_VER < 1300
m_objVector.swap(m_objVector);
#else
m_objVector.shrink_to_fit();
#endif

For STL, apparently swap is not guaranteed to free the memory if you are swapping back onto the same vector.

[xezon]

A template function freeContainer<T> is good for this.

[Mauller]

Looking over at generals, it uses a Map for the object lookup table. Swap appears to be the way to shrink a map when you clear it.

	// set the hash to be rather large. We need to optimize this value later.
	m_objHash.clear();
#if USING_STLPORT
	m_objHash.resize(OBJ_HASH_SIZE);
#else
	m_objHash.reserve(OBJ_HASH_SIZE);
#endif

Although interesting that they seem to have an STL port setup sorted for this one.

I also ran a little test running resize_to_fit and it does appear to free the memory, if not immediately.

[Mauller]

Ah, for STL unordered maps, m_objHash.rehash(OBJ_HASH_SIZE) should force a resize and cause memory to be freed.

[Mauller]

For generals this seems to be the best way to handle clearing the map

	m_objHash.clear();
#if USING_STLPORT
	m_objHash.resize(OBJ_HASH_SIZE);
#else
	m_objHash.rehash(OBJ_HASH_SIZE);
#endif
	ObjectPtrHash tempHash(OBJ_HASH_SIZE);
	tempHash.swap(m_objHash);

[xezon]

That maybe can be simplified to

ObjectPtrHash(OBJ_HASH_SIZE).swap(m_objHash);

It may compile with both STLPort and regular STL.

[Mauller]

Having slept on it a bit more, i think they were on the right track with using a hashset in generals.

The problem being that objectID keeps growing as the game progresses, so the objVector will just grow over time in longer games as well and balloon in size as more entities are spawned in.

I am going to give ZH a go with a hashmap put back in place.

But i think in the longer run, we want to implement a paginated sparse set for things like this.

Example of why paginated sparse sets suit the problem at hand.
https://skypjack.github.io/2020-08-02-ecs-baf-part-9/

[Mauller]

I think i have also found some other places that are growing over time and not being reset properly.

in StateMachine the stateMap never has elements within it cleared when those states go stale and is never cleared when the state machine is reset.

I think there are also similar issues with particles and the particlesystem.

For both if you do a memory profile you can see constantly increasing numbers of allocations over time for these.

As long as i am understanding the memory profile right, my takeaway is that something is not being freed properly with them.

[Mauller]

GameClient::reset also has a similar issue that i just came across while running debug.

They resize the vector before removing object pointers from it and destroying the related object.

Just making a PR for this now.