NBToolkit is a set of offline processing utilities for your MC worlds, unified into a single package.
The oregen utility is an evolution of the original OreGen application released for Beta 1.2 and allows you to create random structured deposits of blocks (typically ores but any block works), which are similar to the native ore deposits created by the MC map generator. There's a suite of options to control the size and number of deposits generated per chunk, as well as depth.
The replace utility replaces one block type with another in every chunk of your map, and can additionally be used to set block data values.
The purge utility deletes chunks from region files (and the region files themselves, if completely emptied).
The relight utility will clear all the lighting information in your world and recalculate it. This should fix all the obnoxious lighting errors from the MC map generator, as well as errors introduced by other third party tools.
The dump utility will dump some or all of your world data into a JSON text file for visual inspection or use by another third party tool.
All utilities support a set of common options to limit the number of chunks or blocks that actually get updated. Such limits include limiting updated chunks to a bounding box defined in chunk coordinates, or updating chunks conditionally on whether they contain or don't contain some block type. Many more options will be supported in the future to allow for interesting map processing.
Backup the world directory you plan on executing this tool on. Although this tool is built on the relatively mature Substrate library, there may be bugs that could result in damaged worlds.
For Windows users:
- If you run Windows XP, make sure you have at least .NET Framework 2.0 installed on your system. This can be obtained through Windows Update if you do not already have it.
- Unzip the nbtoolkit.exe file into any location on your computer (for example, C:\NBToolkit). You do not need the source directory.
- Open a command prompt (Win+R, run 'cmd'), and navigate to the folder you placed the exe into (cd C:\NBToolkit).
- Run nbtoolkit.exe with whichever command and parameters you want to use
For Linux users:
- Make sure you have Mono installed on your system. For Fedora/Red Hat users, 'yum install mono-core' will bring down everything you need. Ubuntu/Debian should be similar but for apt.
- Unzip the NBToolkit source directory to some location on your computer.
- Either run nbtoolkit.exe directly in Mono, or use the included nbtoolkit shell script which wraps the call
- Run nbtoolkit with whichever command and parameters you want to use.
For Mac users:
I have no clue but some variation of the Linux instructions should work. Someone please test and correct me.
Usage
Running 'nbtoolkit' or 'nbtoolkit help' will display basic help information and list all supported commands.
Running 'nbtoolkit help <command>' will list additional information and all supported options for that specific command.
Companion Bukkit Plugin
If you run a Bukkit server, I have recently published a plugin that will let you automatically apply a variant of "oregen" (and to a lesser extent, block-replace) to each chunk as it's generated.
when I put in following line
C:\Users\jakeo\Downloads\nbtk2>NBToolKit.exe oregen -b 15 -w "C:\Users\jake\AppData\Roaming\.minecraft\saves\blank" -r 100 --cxa=79 --cxb=92 --cz
a="-76" --cxb="-68" --min=60 -vv --oo=60
Does not add any iron block to those chuck that would be covered , I got the coordinates for Risugami's Sign Tags, what am I doing wrong here?
You repeated --cxb twice. Also --oo does not take a parameter (were you trying to use --oi instead, to replace a specific block [and only a specific block]?)
You repeated --cxb twice. Also --oo does not take a parameter (were you trying to use --oi instead, to replace a specific block [and only a specific block]?)
I know -oo does not take anything but it crash program if I do not put one in at least with (windows 7) and in help it list as --oo needing value.
Are you trying to specify block coordinates, by any chance? The c* parameters take chunk coordinates ('c' for chunk), and the coordinates you specified are out of range for the discovered regions.
I actually have block coordinate options implemented that will be in the next release, but in the meantime you could just divide by 16 and floor the result.
The problem once again appears to be a stupid slip on my part. TKOptions.cs, line 80, it's trying to test if 'Level.dat' can be found, but in real world directories level.dat is all lower-case. Windows is indifferent but of course that's a major problem for Linux.
Just change that and recompile, and hopefully everything else should just work.
So, I did some testing tonight. I think the lapis default size should be adjusted to 2 (instead of 7).
Testing Procedure
I used three basic worlds:
"Converted" = Generated originally in a 1.1_02 client then converted in a 1.3_01 client. (This was the last version before lapis was added. I used MCNostalgia to revert client.)
"level.dat" = I copied only the level.dat from the original 1.1_02 world before it was converted and then opened a 1.3_01 client and had it convert it. (This was mostly a curiosity test.)
"RandomSeed" = A standard 1.3_01 client generated world. (This is the ideal comparison world, basically what I'd love to see the OreGen function imitate as closely as possible.)
I generated each world and walked to 0,0 without doing anything else then saved the world folder to a dedicated testing folder.
Then, I copied the "Converted" world to a bunch of new folders labelled with the distinguishing parameters I'd use the OreGen function on.
Next, I used the OreGen function to generate lapis in each of the copies of "Converted" with different sizes (Ranging from default of 7 down to 1. I skipped 6 because the default of 7 just seemed obviously too big right off the bat.)
After running the OreGen function, I used c10t to generate statistics with a radius of 5 which processes exactly 81 (9x9) chunks centered about the 0,0 coordinates.
The -cx parameter is pointless in these tests, I just threw it in originally to make sure nothing freaked out. I also adjusted the min and max based on the Wiki documentation. Is it perhaps possible to adjust your routine to do mimic the distribution pattern identified in the Wiki perhaps? Where it indicates lapis is more common near layer 19 then tails off in each direction in frequency? Your min/max is 0/31 for lapis. You might want to adjust that to 4/32 to perhaps match the Wiki's indications of the most common areas it's found. I did 0/35 because I hadn't fully thought it out until now really.
I also realized my initial tests of allowing it to replace dirt and gravel blocks wouldn't exactly mimic ores appearing only in stone as they do with the default algorithm. Not sure why I got that in my head originally, but I don't think it ultimately matters for my basic distribution testing I was doing here.
The RandomSeed and level.dat heightmap renders of lapis seem to "look" different in that the lapis veins seem to be commonly more diagonal and the OreGen function seems to generate them more north/south or east/west in orientation overall. But overall frequency seems to "look" about the same.
--size=2 seems to nail it for the overall count of lapis blocks. I suppose I could run through a lot more tests to ensure reliability, but I'm impressed I held focus long enough to get this data out even, LOL. Maybe someone with more initiative can download the worlds and do some further testing... Or I might be motivated again to do so soon. If you plan on doing multiple updates for this ability, I might even be willing to setup some scripts to automate the testing.
I have a little more information about the algorithms available to me now as opposed to when I initially wrote it, so I intend on doing some tweaking (or rather, giving different choices).
mmmmm tweaking. Do you have any timeline for said tweaking? I was just debating pushing the buttons on my SMP world for this over the weekend, but if you might have them out soon, it might be worth it to wait?
Also what exactly? Can you quickly list the ideals/concepts? So hard to choose lol... want... lapis... BADLY! I just want to mimic it as close to default as reasonably possible also.
I also just realized running this with --cx=21 might end up repopulating newly generated chunks that have already been mined clean of lapis... I'm not sure there is any way to determine logic on that though.
It's true, chunks mined clean of Lapis would have it regenerated. Personally I wouldn't call that a showstopper but some people are more purist than I am :smile.gif:
I still claim that the algorithm is "good enough" for giving you the resource that's reasonably close to the native algorithm, but erring slightly on the bigger side. Since you've done some experimenting and seem to have found a better approximation with the size parameter, I think you should just go with that (if you're happy enough with the results, anyway!).
I don't expect an update to the tool for at least a few more days (other commitments )
There is also the consideration that you can run the tool now with the existing algorithm, and later on if the native algorithm gets added, you could do a global block-replace to erase all existing Lapis and reseed it. The downside of course is it will affect naturally-generated chunks and you'll again be resetting everything you've mined. But I know some people have taken this route on their maps.
As for the types of generation I want to support, that would include native generation (which produces oblong/blobby deposits), and something along the lines of the current algorithm which generates more of a true vein, but obviously with some improvements such as not being biased to northeast.
I so wish chunks had a "Generated" or "Version" tag lol.
I would also advise against (or at least make an option) on "improving" the native algorithm. It has bugs in it for sure, but to me I find the charm that is Minecraft in that. I appreciate the nuances no matter how seemingly silly they are.
Yeah I just read that crazy quadrant thread you've been active in lately myself. Mind blowing. Personally I'd like to see this utility keep true to that algorithm.
Reseeding seems even more disastrous of a concept with so much mining that has already taken place. I'm already willing to just say whatever because at least it brings lapis closer. That said, I might be able to define some basic map regions that I know are new/old chunks and limit my seeding efforts as close as possible to "mostly only old chunks".
When I said improvements I meant with my original oregen algorithm, not the native.
Due to the offset in the native algorithm though I need to build up more infrastructure in the tool to deal with chunk boundary crossing, so it'll probably take a little longer because of that.
Oh I know you are only going to fiddle with your OreGen algorithm. But my point was to leave an option at least for mimicing the native in all it's "buggy" glory. It defines Minecraft.
Makes sense on more time needed. I'll probably push forward with what you have now. It seems sufficient enough. But I'd still be willing to help test if you need someone to do so. I think a utility like this will come in handy in the future for sure. Having it ready for when I need it again can only benefit myself at least. :smile.gif:
Also, any more thought on including an entity export function perhaps? Output to JSON would be pure hawtness.
NBToolkit is a set of offline processing utilities for your MC worlds, unified into a single package.
The oregen utility is an evolution of the original OreGen application released for Beta 1.2 and allows you to create random structured deposits of blocks (typically ores but any block works), which are similar to the native ore deposits created by the MC map generator. There's a suite of options to control the size and number of deposits generated per chunk, as well as depth.
The replace utility replaces one block type with another in every chunk of your map, and can additionally be used to set block data values.
The purge utility deletes chunks from region files (and the region files themselves, if completely emptied).
The relight utility will clear all the lighting information in your world and recalculate it. This should fix all the obnoxious lighting errors from the MC map generator, as well as errors introduced by other third party tools.
The dump utility will dump some or all of your world data into a JSON text file for visual inspection or use by another third party tool.
All utilities support a set of common options to limit the number of chunks or blocks that actually get updated. Such limits include limiting updated chunks to a bounding box defined in chunk coordinates, or updating chunks conditionally on whether they contain or don't contain some block type. Many more options will be supported in the future to allow for interesting map processing.
Getting Started
Download NBToolkit here
GitHub Project: here
Backup the world directory you plan on executing this tool on. Although this tool is built on the relatively mature Substrate library, there may be bugs that could result in damaged worlds.
For Windows users:
- If you run Windows XP, make sure you have at least .NET Framework 2.0 installed on your system. This can be obtained through Windows Update if you do not already have it.
- Unzip the nbtoolkit.exe file into any location on your computer (for example, C:\NBToolkit). You do not need the source directory.
- Open a command prompt (Win+R, run 'cmd'), and navigate to the folder you placed the exe into (cd C:\NBToolkit).
- Run nbtoolkit.exe with whichever command and parameters you want to use
For Linux users:
- Make sure you have Mono installed on your system. For Fedora/Red Hat users, 'yum install mono-core' will bring down everything you need. Ubuntu/Debian should be similar but for apt.
- Unzip the NBToolkit source directory to some location on your computer.
- Either run nbtoolkit.exe directly in Mono, or use the included nbtoolkit shell script which wraps the call
- Run nbtoolkit with whichever command and parameters you want to use.
For Mac users:
I have no clue but some variation of the Linux instructions should work. Someone please test and correct me.
Usage
Running 'nbtoolkit' or 'nbtoolkit help' will display basic help information and list all supported commands.
Running 'nbtoolkit help <command>' will list additional information and all supported options for that specific command.
Companion Bukkit Plugin
If you run a Bukkit server, I have recently published a plugin that will let you automatically apply a variant of "oregen" (and to a lesser extent, block-replace) to each chunk as it's generated.
http://dev.bukkit.or...r-mods/oreplus/
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
C:\Users\jakeo\Downloads\nbtk2>NBToolKit.exe oregen -b 15 -w "C:\Users\jake\AppData\Roaming\.minecraft\saves\blank" -r 100 --cxa=79 --cxb=92 --cz
a="-76" --cxb="-68" --min=60 -vv --oo=60
Does not add any iron block to those chuck that would be covered , I got the coordinates for Risugami's Sign Tags, what am I doing wrong here?
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
I know -oo does not take anything but it crash program if I do not put one in at least with (windows 7) and in help it list as --oo needing value.
And still does nothing, I get feeling that use of area command does not actually work.
I actually have block coordinate options implemented that will be in the next release, but in the meantime you could just divide by 16 and floor the result.
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
I tried: --block=21 -s 5 --cx=21
and: --block=21 --size=5 --cx=21
but getting this in output:
Perhaps your defaults are overriding/ignoring the parameter?
Oh yeah, in OreGen.cs (lines 61 and 62):
OPT_MIN should be OPT_SIZE ?
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
Just change that and recompile, and hopefully everything else should just work.
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
I suppose I could learn myself, but I'm hoping you can "just push a few buttons" and save me "a few hours of research". :smile.gif:
Edit: Actually I whined for help and I should have just tried first. Visual Studio Express 2010 converted it just fine and right click/Build ftw!
I got it working as I need for my testing I think now...
Glad you got it built successfully though.
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
Testing Procedure
I used three basic worlds:
"Converted" = Generated originally in a 1.1_02 client then converted in a 1.3_01 client. (This was the last version before lapis was added. I used MCNostalgia to revert client.)
"level.dat" = I copied only the level.dat from the original 1.1_02 world before it was converted and then opened a 1.3_01 client and had it convert it. (This was mostly a curiosity test.)
"RandomSeed" = A standard 1.3_01 client generated world. (This is the ideal comparison world, basically what I'd love to see the OreGen function imitate as closely as possible.)
I generated each world and walked to 0,0 without doing anything else then saved the world folder to a dedicated testing folder.
Then, I copied the "Converted" world to a bunch of new folders labelled with the distinguishing parameters I'd use the OreGen function on.
Next, I used the OreGen function to generate lapis in each of the copies of "Converted" with different sizes (Ranging from default of 7 down to 1. I skipped 6 because the default of 7 just seemed obviously too big right off the bat.)
After running the OreGen function, I used c10t to generate statistics with a radius of 5 which processes exactly 81 (9x9) chunks centered about the 0,0 coordinates.
Results at a glance
LapisLazuliOre: Parameters
209: level.dat
220: RandomSeed
313: --block=21 --cx=21
264: --block=21 --rounds=1 --min=0 --max=35 --size=5 --oi=1 --oi=3 --oi=13 --cx=21
258: --block=21 --rounds=1 --min=0 --max=35 --size=4 --oi=1 --oi=3 --oi=13 --cx=21
269: --block=21 --rounds=1 --min=0 --max=35 --size=3 --oi=1 --oi=3 --oi=13 --cx=21
228: --block=21 --rounds=1 --min=0 --max=35 --size=2 --oi=1 --oi=3 --oi=13 --cx=21
164: --block=21 --rounds=1 --min=0 --max=35 --size=1 --oi=1 --oi=3 --oi=13 --cx=21
202: --block=21 --rounds=1 --min=0 --max=35 --size=2 --oi=1
The -cx parameter is pointless in these tests, I just threw it in originally to make sure nothing freaked out. I also adjusted the min and max based on the Wiki documentation. Is it perhaps possible to adjust your routine to do mimic the distribution pattern identified in the Wiki perhaps? Where it indicates lapis is more common near layer 19 then tails off in each direction in frequency? Your min/max is 0/31 for lapis. You might want to adjust that to 4/32 to perhaps match the Wiki's indications of the most common areas it's found. I did 0/35 because I hadn't fully thought it out until now really.
I also realized my initial tests of allowing it to replace dirt and gravel blocks wouldn't exactly mimic ores appearing only in stone as they do with the default algorithm. Not sure why I got that in my head originally, but I don't think it ultimately matters for my basic distribution testing I was doing here.
The RandomSeed and level.dat heightmap renders of lapis seem to "look" different in that the lapis veins seem to be commonly more diagonal and the OreGen function seems to generate them more north/south or east/west in orientation overall. But overall frequency seems to "look" about the same.
--size=2 seems to nail it for the overall count of lapis blocks. I suppose I could run through a lot more tests to ensure reliability, but I'm impressed I held focus long enough to get this data out even, LOL. Maybe someone with more initiative can download the worlds and do some further testing... Or I might be motivated again to do so soon. If you plan on doing multiple updates for this ability, I might even be willing to setup some scripts to automate the testing.
You can peruse/download all my testing materials.
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
Also what exactly? Can you quickly list the ideals/concepts? So hard to choose lol... want... lapis... BADLY! I just want to mimic it as close to default as reasonably possible also.
I also just realized running this with --cx=21 might end up repopulating newly generated chunks that have already been mined clean of lapis... I'm not sure there is any way to determine logic on that though.
I still claim that the algorithm is "good enough" for giving you the resource that's reasonably close to the native algorithm, but erring slightly on the bigger side. Since you've done some experimenting and seem to have found a better approximation with the size parameter, I think you should just go with that (if you're happy enough with the results, anyway!).
I don't expect an update to the tool for at least a few more days (other commitments )
There is also the consideration that you can run the tool now with the existing algorithm, and later on if the native algorithm gets added, you could do a global block-replace to erase all existing Lapis and reseed it. The downside of course is it will affect naturally-generated chunks and you'll again be resetting everything you've mined. But I know some people have taken this route on their maps.
As for the types of generation I want to support, that would include native generation (which produces oblong/blobby deposits), and something along the lines of the current algorithm which generates more of a true vein, but obviously with some improvements such as not being biased to northeast.
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
I would also advise against (or at least make an option) on "improving" the native algorithm. It has bugs in it for sure, but to me I find the charm that is Minecraft in that. I appreciate the nuances no matter how seemingly silly they are.
Yeah I just read that crazy quadrant thread you've been active in lately myself. Mind blowing. Personally I'd like to see this utility keep true to that algorithm.
Reseeding seems even more disastrous of a concept with so much mining that has already taken place. I'm already willing to just say whatever because at least it brings lapis closer. That said, I might be able to define some basic map regions that I know are new/old chunks and limit my seeding efforts as close as possible to "mostly only old chunks".
Due to the offset in the native algorithm though I need to build up more infrastructure in the tool to deal with chunk boundary crossing, so it'll probably take a little longer because of that.
Mods I Develop: Garden Stuff -- Storage Drawers -- Hunger Strike
Tools I Develop: NBTExplorer -- Substrate
Makes sense on more time needed. I'll probably push forward with what you have now. It seems sufficient enough. But I'd still be willing to help test if you need someone to do so. I think a utility like this will come in handy in the future for sure. Having it ready for when I need it again can only benefit myself at least. :smile.gif:
Also, any more thought on including an entity export function perhaps? Output to JSON would be pure hawtness.