Add Mapblock meta

[kromka-chleba]

Problem

Modders may want to store and track some data related to mapblocks in order to for example modify specific mapblocks especially with connection to the newly emerging block callbacks #15999
Currently the only way to do that is to keep that data in mod storage with hashed mapblock pos as the key.
There's a chance this way of storing data is not efficient.

Solutions

Introduce mapblock meta which would allow to store key-value pairs for mapblocks, similarly to node meta, etc.
Different functions would then help to search through the database or find mapblocks with specific information.

Alternatives

Leave it up to modders to reinvent the wheel which I'm already doing anyway.

Additional context

No response

[appgurueu]

Currently the only way to do that is to keep that data in mod storage with hashed mapblock pos as the key.
There's a chance this way of storing data is not efficient.

This is not so bad and in fact what I would currently suggest. If this way of storing data is not efficient, we ought to fix that.
However this is probably a bit awkward to keep in sync (e.g. after a /deleteblocks the data in mod storage will still be there).

---

To add another use case: Dense auxiliary data. Maybe a game wants to store HP for every node. Temperature. Age. Whatever. Node metadata is rather high overhead due to the node -> hash map structure. If we instead turn this around to be a hash map of dense typed arrays of auxiliary node data (think numpy style), we could be much more efficient.

For the long term, something like this is how I envision custom meshgen flags beyond param2 (which would then be processed by SSCSM on the client) should be done eventually.

[sorcerykid]

I encountered this same obstacle while developing my Ambience II mod back in 2017. I needed to cache ambient and dynamic sounds on a per-mapblock basis. Ultimately the solution I devised was to spawn a transparent entity at the center of each loaded mapblock. Since entitites are much less prone to destructive map operations than nodes, they proved to be the ideal candidate for storing mapblock metadata

Here's a synopsis of the approach I used in that mod:
https://forum.luanti.org/viewtopic.php?p=441968#p441968

Of course, even entities have their share of shortcomings, not the least of which is that they can be purged by the engine without warning. This is why I have long wanted to submit a proposal for builtin support of mapblock metadata. As you may or may not know, Desour actually began work on such a feature several years ago, but alas PR #8870 was eventually abandoned.

I've been brainstorming the past few years on an advanced Mapblock Manager API for Luanti. Mapblock metadata would certainly be a suitable fit for that proposal. So I might go ahead and post more details about that idea here on GitHub.

[Zughy]

@appgurueu is that a "Concept approved"? (Unsubscribing)

[appgurueu]

Yes, I approve the concept; though the details still need to be discussed.

[Desour]

Desour actually began work on such a feature several years ago, but alas PR #8870 was eventually abandoned.

It was closed because of missing core dev support (of #8605).

[kromka-chleba]

@Desour any chances you'd pick this up?
Asking for clarity.

[Desour]

@Desour any chances you'd pick this up? Asking for clarity.

No, at least I don't plan to. Don't wait on me.

TBH, I don't even support the feature in principle now. I don't see the need nor the benefits.

• IIRC, back then the whole world storage had to be loaded as a whole into memory (because it was just a file with json), but now we use an sqlite db. So, nowadays, using worldstorage instead has no real issues.

• And block meta is also a bit meh because the modder has to deal with block bounds themselves, while world storage easily allows to be block agnostic (e.g. store stuff in 64^3 chunks).

• (My old issue and PR probably had some misconceptions, btw. Take my bullshit with grains of salt.)

• If you adopt my PR, please also rethink if it is actually useful to send block meta to clients (nodetimers aren't sent either).

• To add another use case: Dense auxiliary data.

  Idk how block meta would be appropriate for dense (or semi dense) auxiliary node data, it looks like a different feature to me (that I find very interesting, but it's a separate topic).

• a bit awkward to keep in sync (e.g. after a /deleteblocks the data in mod storage will still be there).

  This needs callbacks for when blocks are deleted.

• spawn a transparent entity at the center of each loaded mapblock.

  So hacky! x) But world storage got better, so you can probably replace it with that now.