I want to save a large amount of persistent data to a minecraft world's save. This data is not tied to any one specific entity, and needs to be accessible from within the game. It needs to be able to handle what could potentially become a large amount of data.
Essentially, I'd be saving data entries for individual entities that contain portions of their saved data. This data would include things like their ID, any custom name they might have, and other traits defined through their NBT. This data would be used as a sort of "index" or "registry" that a player could access, making it easy to find basic info about their animals. This data needs to be editable on a per-entity basis based off their ID, to update things like custom name, etc. when accessed. It needs to be able to hold either an indefinite amount of data, or at least a large amount (60-100+ entries per file). It needs to be writable during gameplay, triggered by certain actions within the game, and should also probably be saved at world save.
It also needs to be per-player-- so if you have multiple people playing in a world or on a server, the game needs to identify whose pet index to access based off who's calling for the information.
Ideally I'd like to use .dat files, but I can't find much information on them at all for 1.11-1.12 and what I have found is very outdated. I'm also having trouble tracking down the actual classes that write them in the code, so I was hoping someone here might be able to lend some insight.
Does anyone have a good idea of the best way to accomplish this?
I haven't wanted to use a capability because I need to be able to save an absolutely massive amount of data that will only be accessed under certain circumstances. I'm literally trying to add a database of registered entities for every type of mob I add to the game-- which eventually will become an extensive list. Every registered entity needs to have a tag list filed under its UUID with several associated strings of data. That can add up very fast, and I just haven't seen anyone use a capability for storing so much data. Would there be any concerns with using a capability for that sort of thing?
I'm mostly not sure of how much capabilities are kept actively loaded-- if the data is kept stored in active memory all the time, then that's going to be out of the question. That's why I was hoping to go with a world data approach-- Minecraft can store tremendous amounts of data in a world save, enough to encompass an entire map's worth, but most of it isn't kept actively loaded. But if capabilities aren't always kept actively loaded, it might be possible. I'm definitely concerned about trying to copy databases on player death, though, again given the potential for a huge amount of data being stored...
The data I'm trying to save is as follows:
Entity UUID- this will be used to "look up" the saved data and get the associated information.
Entity's genetics- The genetic code for a WCWolf entity is 24 characters long
Entity's pedigree- The pedigree for a WCWolf entity will end up being about 7 characters long
Depending on storage efficiency, POSSIBLY data about the entity's health/speed/damage
We'd then be looking at saving dozens, potentially even hundreds of these entities (if you had a truly dedicated player maybe even thousands) into data, per player.
I suppose the alternative to this could be to simply look up the entity's UUID and see if an entity by that UUID exists currently in the world-- but would that be limited by which entities are loaded/unloaded? If it isn't, I could potentially just save a list of UUIDs and then dynamically grab info straight off the entities associated with those UUIDs...
In terms of modern computing, I don't think a list of entities is really that much data. Even 100k entities would probably take less than 10MB.
I would just use the world save data extension mechanism. There is are built in classes for saving data: WorldSavedData. How you do it depends if you want to store it per-dimension (different data for e.g. nether and overworld, these are two World instances in the code) or per save-file (same data for all dimensions). For the former use world.perWorldStorage, otherwise use world.mapStorage.
Warning: Be sure to always call markDirty() on your data when you change a value, otherwise Minecraft will not save it.
Alright, thank you so much jabelar! Is there a limit to how much data can be stored in WorldSaveData, or is it a big enough limit to not warrant concern?
Alright, thank you so much jabelar! Is there a limit to how much data can be stored in WorldSaveData, or is it a big enough limit to not warrant concern?
I'm not aware of a limit. If you think about it a chunk has 65k blocks in it, so most worlds have to save millions of block informations, plus it already stores information on all the entities in the world, and so on.
Note that while I don't think you need to worry too much about perf and memory, you should worry a bit. For example, you should not reload and process the whole data every tick for example. And you should try to be somewhat compact in how you store stuff -- like UUID might be sufficient, but if you need to store additional info about the entity just do it in a compact way.
Lastly, in all performance optimization it is best to start by simply coding the way you want it to work logically and then profile the execution to understand if/when there is actual performance issues. I see a lot of people work hard and introduce complex, bug-inducing schemes that are unnecessary -- only work on optimization when you see an actual perf problem.
I'm not aware of a limit. If you think about it a chunk has 65k blocks in it, so most worlds have to save millions of block informations, plus it already stores information on all the entities in the world, and so on.
Note that while I don't think you need to worry too much about perf and memory, you should worry a bit. For example, you should reload and process the whole data every tick for example. And you should try to be somewhat compact in how you store stuff -- like UUID might be sufficient, but if you need to store additional info about the entity just do it in a compact way.
Lastly, in all performance optimization it is best to start by simply coding the way you want it to work logically and then profile the execution to understand if/when there is actual performance issues. I see a lot of people work hard and introduce complex, bug-inducing schemes that are unnecessary -- only work on optimization when you see an actual perf problem.
Alright, I'll keep that in mind. Hopefully the way I'm doing it will already be pretty streamlined, but I won't go out of my way to make it moreso unless it actually causes problems.
Thanks for all your help!
Hello,
I want to save a large amount of persistent data to a minecraft world's save. This data is not tied to any one specific entity, and needs to be accessible from within the game. It needs to be able to handle what could potentially become a large amount of data.
Essentially, I'd be saving data entries for individual entities that contain portions of their saved data. This data would include things like their ID, any custom name they might have, and other traits defined through their NBT. This data would be used as a sort of "index" or "registry" that a player could access, making it easy to find basic info about their animals. This data needs to be editable on a per-entity basis based off their ID, to update things like custom name, etc. when accessed. It needs to be able to hold either an indefinite amount of data, or at least a large amount (60-100+ entries per file). It needs to be writable during gameplay, triggered by certain actions within the game, and should also probably be saved at world save.
It also needs to be per-player-- so if you have multiple people playing in a world or on a server, the game needs to identify whose pet index to access based off who's calling for the information.
Ideally I'd like to use .dat files, but I can't find much information on them at all for 1.11-1.12 and what I have found is very outdated. I'm also having trouble tracking down the actual classes that write them in the code, so I was hoping someone here might be able to lend some insight.
Does anyone have a good idea of the best way to accomplish this?
Thanks!
I haven't wanted to use a capability because I need to be able to save an absolutely massive amount of data that will only be accessed under certain circumstances. I'm literally trying to add a database of registered entities for every type of mob I add to the game-- which eventually will become an extensive list. Every registered entity needs to have a tag list filed under its UUID with several associated strings of data. That can add up very fast, and I just haven't seen anyone use a capability for storing so much data. Would there be any concerns with using a capability for that sort of thing?
I'm mostly not sure of how much capabilities are kept actively loaded-- if the data is kept stored in active memory all the time, then that's going to be out of the question. That's why I was hoping to go with a world data approach-- Minecraft can store tremendous amounts of data in a world save, enough to encompass an entire map's worth, but most of it isn't kept actively loaded. But if capabilities aren't always kept actively loaded, it might be possible. I'm definitely concerned about trying to copy databases on player death, though, again given the potential for a huge amount of data being stored...
The data I'm trying to save is as follows:
Entity UUID- this will be used to "look up" the saved data and get the associated information.
Entity's mother's UUID
Entity's father's UUID
Entity's generation- 0 = wild spawn/egg spawn, 1+ = bred
Entity's genetics- The genetic code for a WCWolf entity is 24 characters long
Entity's pedigree- The pedigree for a WCWolf entity will end up being about 7 characters long
Depending on storage efficiency, POSSIBLY data about the entity's health/speed/damage
We'd then be looking at saving dozens, potentially even hundreds of these entities (if you had a truly dedicated player maybe even thousands) into data, per player.
I suppose the alternative to this could be to simply look up the entity's UUID and see if an entity by that UUID exists currently in the world-- but would that be limited by which entities are loaded/unloaded? If it isn't, I could potentially just save a list of UUIDs and then dynamically grab info straight off the entities associated with those UUIDs...
In terms of modern computing, I don't think a list of entities is really that much data. Even 100k entities would probably take less than 10MB.
I would just use the world save data extension mechanism. There is are built in classes for saving data: WorldSavedData. How you do it depends if you want to store it per-dimension (different data for e.g. nether and overworld, these are two World instances in the code) or per save-file (same data for all dimensions). For the former use world.perWorldStorage, otherwise use world.mapStorage.
You can see diesieben07's simple example here: WorldSaveData example codeWarning: Be sure to always call markDirty() on your data when you change a value, otherwise Minecraft will not save it.
Alright, thank you so much jabelar! Is there a limit to how much data can be stored in WorldSaveData, or is it a big enough limit to not warrant concern?
I'm not aware of a limit. If you think about it a chunk has 65k blocks in it, so most worlds have to save millions of block informations, plus it already stores information on all the entities in the world, and so on.
Note that while I don't think you need to worry too much about perf and memory, you should worry a bit. For example, you should not reload and process the whole data every tick for example. And you should try to be somewhat compact in how you store stuff -- like UUID might be sufficient, but if you need to store additional info about the entity just do it in a compact way.
Lastly, in all performance optimization it is best to start by simply coding the way you want it to work logically and then profile the execution to understand if/when there is actual performance issues. I see a lot of people work hard and introduce complex, bug-inducing schemes that are unnecessary -- only work on optimization when you see an actual perf problem.
Alright, I'll keep that in mind. Hopefully the way I'm doing it will already be pretty streamlined, but I won't go out of my way to make it moreso unless it actually causes problems.
Thanks for all your help!