Jump to content

  • Curse Sites
Become a Premium Member! Help
Latest News Article

NBToolkit - Map Processing Tools

updated 5.18.13

  • Please log in to reply
323 replies to this topic

#1

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 26 February 2011 - 10:21 AM

NBToolkit

NBToolkit is a set of offline processing utilities for your MC worlds, unified into a single package.  

The oregen utility is an evolution of the original OreGen application released for Beta 1.2 and allows you to create random structured deposits of blocks (typically ores but any block works), which are similar to the native ore deposits created by the MC map generator.  There's a suite of options to control the size and number of deposits generated per chunk, as well as depth.

The replace utility replaces one block type with another in every chunk of your map, and can additionally be used to set block data values.

The purge utility deletes chunks from region files (and the region files themselves, if completely emptied).

The relight utility will clear all the lighting information in your world and recalculate it.  This should fix all the obnoxious lighting errors from the MC map generator, as well as errors introduced by other third party tools.

The dump utility will dump some or all of your world data into a JSON text file for visual inspection or use by another third party tool.

All utilities support a set of common options to limit the number of chunks or blocks that actually get updated.  Such limits include limiting updated chunks to a bounding box defined in chunk coordinates, or updating chunks conditionally on whether they contain or don't contain some block type.  Many more options will be supported in the future to allow for interesting map processing.

Getting Started

Download NBToolkit here
GitHub Project: here

Backup the world directory you plan on executing this tool on.  Although this tool is built on the relatively mature Substrate library, there may be bugs that could result in damaged worlds.

For Windows users:
- If you run Windows XP, make sure you have at least .NET Framework 2.0 installed on your system.  This can be obtained through Windows Update if you do not already have it.
- Unzip the nbtoolkit.exe file into any location on your computer (for example, C:\NBToolkit).  You do not need the source directory.
- Open a command prompt (Win+R, run 'cmd'), and navigate to the folder you placed the exe into (cd C:\NBToolkit).
- Run nbtoolkit.exe with whichever command and parameters you want to use

For Linux users:
- Make sure you have Mono installed on your system.  For Fedora/Red Hat users, 'yum install mono-core' will bring down everything you need.  Ubuntu/Debian should be similar but for apt.
- Unzip the NBToolkit source directory to some location on your computer.
- Either run nbtoolkit.exe directly in Mono, or use the included nbtoolkit shell script which wraps the call
- Run nbtoolkit with whichever command and parameters you want to use.

For Mac users:
I have no clue but some variation of the Linux instructions should work.  Someone please test and correct me.

Usage

Running 'nbtoolkit' or 'nbtoolkit help' will display basic help information and list all supported commands.

Running 'nbtoolkit help <command>' will list additional information and all supported options for that specific command.

Companion Bukkit Plugin

If you run a Bukkit server, I have recently published a plugin that will let you automatically apply a variant of "oregen" (and to a lesser extent, block-replace) to each chunk as it's generated.

http://dev.bukkit.or...r-mods/oreplus/
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

Register or log in to remove.

#2

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 26 February 2011 - 10:22 AM

Reserved
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#3

bigolslabomeat

Posted 27 February 2011 - 10:18 AM

Don't know why there has been no love on this thread yet, but I love this!

command line help told me exactly what I needed to do, ran without error first time, can't tell you if it's worked yet cos I've not had time to go mining in my newly lapis populated world :iapprove:
Yet Another Minecraft Server: http://yams.in
Easiest way to run a Minecraft server on Windows!

#4

Tenosis
    Tenosis

    Tree Puncher

  • Members
  • 16 posts

Posted 28 February 2011 - 04:29 PM

Glad to see these useful tools aren't being forgotten!

#5

omagaalpha

Posted 01 March 2011 - 02:24 PM

when I put in following line
C:\Users\jakeo\Downloads\nbtk2>NBToolKit.exe oregen -b 15 -w "C:\Users\jake\AppData\Roaming\.minecraft\saves\blank" -r 100 --cxa=79 --cxb=92 --cz
a="-76" --cxb="-68" --min=60 -vv --oo=60

Does not add any iron block to those chuck that would be covered , I got the coordinates for Risugami's Sign Tags, what am I doing wrong here?

#6

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 01 March 2011 - 03:22 PM

You repeated --cxb twice.  Also --oo does not take a parameter (were you trying to use --oi instead, to replace a specific block [and only a specific block]?)
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#7

omagaalpha

Posted 01 March 2011 - 03:50 PM

jaquadro said:

You repeated --cxb twice.  Also --oo does not take a parameter (were you trying to use --oi instead, to replace a specific block [and only a specific block]?)
I know -oo does not take anything but it crash program if I do not put one in at least with (windows 7) and in help it list as --oo needing value.

C:\Users\jake\Downloads\nbtk2>NBToolKit.exe oregen -b 14 -w "C:\Users\jake\AppData\Roaming\.minecraft\saves\blank" -r 100 --cxa=79 --cxb=92 --cza=-76 --czb=-68 --min=60 -v -oo=14
r.-1.-1.mcr
r.-1.0.mcr
r.0.-1.mcr
r.0.0.mcr
Done

And still does nothing, I get feeling that use of area command does not actually work.

#8

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 01 March 2011 - 08:56 PM

Are you trying to specify block coordinates, by any chance?  The c* parameters take chunk coordinates ('c' for chunk), and the coordinates you specified are out of range for the discovered regions.

I actually have block coordinate options implemented that will be in the next release, but in the meantime you could just divide by 16 and floor the result.
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#9

EdGruberman
  • Location: Tucson, AZ
  • Minecraft: EdGruberman

Posted 03 March 2011 - 08:06 PM

OreGen seems to be ignoring the -s and --size parameters.

I tried: --block=21 -s 5 --cx=21
and: --block=21 --size=5 --cx=21
but getting this in output:

Quote

Generating 1 size 7 deposits of 21 between 5 and 31

Perhaps your defaults are overriding/ignoring the parameter?

Oh yeah, in OreGen.cs (lines 61 and 62):
				{ "s|size=", "Generates deposits containing roughly up to {VAL} blocks",
					v => OPT_MIN = Convert.ToInt32(v) % 128 },

OPT_MIN should be OPT_SIZE ?

#10

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 03 March 2011 - 08:11 PM

Good catch, it's actually setting the Min depth by mistake.
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#11

EdGruberman
  • Location: Tucson, AZ
  • Minecraft: EdGruberman

Posted 03 March 2011 - 08:14 PM

NINJA'd!

#12

theoutrider
  • Location: Cambridge, UK

Posted 03 March 2011 - 09:54 PM

Running Ubuntu 11.04 here - NBToolkit compiled fine, but whenever I try to run it it claims the world directory given is invalid. This happens with relative and absolute paths, and with both --world= and -w option formats. Any idea what might be going wrong here?

#13

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 03 March 2011 - 09:58 PM

The problem once again appears to be a stupid slip on my part.  TKOptions.cs, line 80, it's trying to test if 'Level.dat' can be found, but in real world directories level.dat is all lower-case.  Windows is indifferent but of course that's a major problem for Linux.

Just change that and recompile, and hopefully everything else should just work.
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#14

EdGruberman
  • Location: Tucson, AZ
  • Minecraft: EdGruberman

Posted 03 March 2011 - 11:12 PM

Any chance of getting you to recompile for the size bug on the Windows exe today?

I suppose I could learn myself, but I'm hoping you can "just push a few buttons" and save me "a few hours of research". :P

Edit: Actually I whined for help and I should have just tried first.  Visual Studio Express 2010 converted it just fine and right click/Build ftw!

I got it working as I need for my testing I think now...

#15

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 03 March 2011 - 11:45 PM

Yeah I would have just published a new version but I have some half-implemented stuff right now that shows up in the options list.

Glad you got it built successfully though.
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#16

EdGruberman
  • Location: Tucson, AZ
  • Minecraft: EdGruberman

Posted 04 March 2011 - 05:35 AM

So, I did some testing tonight.  I think the lapis default size should be adjusted to 2 (instead of 7).


Testing Procedure
I used three basic worlds:
"Converted" = Generated originally in a 1.1_02 client then converted in a 1.3_01 client.  (This was the last version before lapis was added.  I used MCNostalgia to revert client.)
"level.dat" = I copied only the level.dat from the original 1.1_02 world before it was converted and then opened a 1.3_01 client and had it convert it. (This was mostly a curiosity test.)
"RandomSeed" = A standard 1.3_01 client generated world.  (This is the ideal comparison world, basically what I'd love to see the OreGen function imitate as closely as possible.)

I generated each world and walked to 0,0 without doing anything else then saved the world folder to a dedicated testing folder.

Then, I copied the "Converted" world to a bunch of new folders labelled with the distinguishing parameters I'd use the OreGen function on.

Next, I used the OreGen function to generate lapis in each of the copies of "Converted" with different sizes (Ranging from default of 7 down to 1.  I skipped 6 because the default of 7 just seemed obviously too big right off the bat.)

After running the OreGen function, I used c10t to generate statistics with a radius of 5 which processes exactly 81 (9x9) chunks centered about the 0,0 coordinates.


Results at a glance
LapisLazuliOre: Parameters
209: level.dat
220: RandomSeed
313: --block=21 --cx=21
264: --block=21 --rounds=1 --min=0 --max=35 --size=5 --oi=1 --oi=3 --oi=13 --cx=21
258: --block=21 --rounds=1 --min=0 --max=35 --size=4 --oi=1 --oi=3 --oi=13 --cx=21
269: --block=21 --rounds=1 --min=0 --max=35 --size=3 --oi=1 --oi=3 --oi=13 --cx=21
228: --block=21 --rounds=1 --min=0 --max=35 --size=2 --oi=1 --oi=3 --oi=13 --cx=21
164: --block=21 --rounds=1 --min=0 --max=35 --size=1 --oi=1 --oi=3 --oi=13 --cx=21
202: --block=21 --rounds=1 --min=0 --max=35 --size=2 --oi=1

The -cx parameter is pointless in these tests, I just threw it in originally to make sure nothing freaked out.  I also adjusted the min and max based on the Wiki documentation.  Is it perhaps possible to adjust your routine to do mimic the distribution pattern identified in the Wiki perhaps?  Where it indicates lapis is more common near layer 19 then tails off in each direction in frequency?  Your min/max is 0/31 for lapis.  You might want to adjust that to 4/32 to perhaps match the Wiki's indications of the most common areas it's found.  I did 0/35 because I hadn't fully thought it out until now really.

I also realized my initial tests of allowing it to replace dirt and gravel blocks wouldn't exactly mimic ores appearing only in stone as they do with the default algorithm.  Not sure why I got that in my head originally, but I don't think it ultimately matters for my basic distribution testing I was doing here.

The RandomSeed and level.dat heightmap renders of lapis seem to "look" different in that the lapis veins seem to be commonly more diagonal and the OreGen function seems to generate them more north/south or east/west in orientation overall.  But overall frequency seems to "look" about the same.

--size=2 seems to nail it for the overall count of lapis blocks.  I suppose I could run through a lot more tests to ensure reliability, but I'm impressed I held focus long enough to get this data out even, LOL.  Maybe someone with more initiative can download the worlds and do some further testing... Or I might be motivated again to do so soon.  If you plan on doing multiple updates for this ability, I might even be willing to setup some scripts to automate the testing.

You can peruse/download all my testing materials.

#17

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 04 March 2011 - 05:39 AM

I have a little more information about the algorithms available to me now as opposed to when I initially wrote it, so I intend on doing some tweaking (or rather, giving different choices).
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#18

EdGruberman
  • Location: Tucson, AZ
  • Minecraft: EdGruberman

Posted 04 March 2011 - 05:44 AM

mmmmm tweaking.  Do you have any timeline for said tweaking?  I was just debating pushing the buttons on my SMP world for this over the weekend, but if you might have them out soon, it might be worth it to wait?

Also what exactly?  Can you quickly list the ideals/concepts?  So hard to choose lol... want... lapis... BADLY!  I just want to mimic it as close to default as reasonably possible also.

I also just realized running this with --cx=21 might end up repopulating newly generated chunks that have already been mined clean of lapis... I'm not sure there is any way to determine logic on that though.

#19

jaquadro
    jaquadro

    Lapis Lazuli Collector

  • Curse Premium
  • Curse Premium
  • 1053 posts

Posted 04 March 2011 - 06:18 AM

It's true, chunks mined clean of Lapis would have it regenerated.  Personally I wouldn't call that a showstopper but some people are more purist than I am :D

I still claim that the algorithm is "good enough" for giving you the resource that's reasonably close to the native algorithm, but erring slightly on the bigger side.  Since you've done some experimenting and seem to have found a better approximation with the size parameter, I think you should just go with that (if you're happy enough with the results, anyway!).

I don't expect an update to the tool for at least a few more days (other commitments  :) )

There is also the consideration that you can run the tool now with the existing algorithm, and later on if the native algorithm gets added, you could do a global block-replace to erase all existing Lapis and reseed it.  The downside of course is it will affect naturally-generated chunks and you'll again be resetting everything you've mined.  But I know some people have taken this route on their maps.

As for the types of generation I want to support, that would include native generation (which produces oblong/blobby deposits), and something along the lines of the current algorithm which generates more of a true vein, but obviously with some improvements such as not being biased to northeast.
NBTExplorer - NBT editor for Windows and Mac
Substrate - Minecraft map editing library for C#/.NET

#20

EdGruberman
  • Location: Tucson, AZ
  • Minecraft: EdGruberman

Posted 04 March 2011 - 06:40 AM

I so wish chunks had a "Generated" or "Version" tag lol.

I would also advise against (or at least make an option) on "improving" the native algorithm.  It has bugs in it for sure, but to me I find the charm that is Minecraft in that.  I appreciate the nuances no matter how seemingly silly they are.

Yeah I just read that crazy quadrant thread you've been active in lately myself.  Mind blowing.  Personally I'd like to see this utility keep true to that algorithm.

Reseeding seems even more disastrous of a concept with so much mining that has already taken place.  I'm already willing to just say whatever because at least it brings lapis closer.  That said, I might be able to define some basic map regions that I know are new/old chunks and limit my seeding efforts as close as possible to "mostly only old chunks".