Jump to content

  • Curse Sites
Become a Premium Member! Help
Latest News Article

TopoMC - generating Minecraft worlds from topographic data


  • Please log in to reply
382 replies to this topic

#1

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 25 January 2011 - 11:17 PM

Welcome to TopoMC!

The TopoMC project facilitates the construction of superficially realistic Minecraft worlds leveraging USGS, specifically the NED and NLCD datasets.

Major changes:

* TopoMC now generates Anvil worlds with full 256-block heights thanks to @codewarrior0 and his changes to mcedit/pymclevel!  Thank you!

* TopoMC also now runs on Windows!  Kinda.  I think.  It works for me, anyway!  See this for more details!

* The array and world code has been replaced with region-based code which improve accuracy while saving CPU and memory.  The new commands to download, prepare, and build regions are documented below.

* GPGPU support has been added. Yes, your video card can help you build Minecraft worlds!  This latest feature relies on PyOpenCL and its associated dependencies, and is not supported on all systems.

* The landcover code has been rewritten to support the usage of MCEdit schematics as templates for certain areas, specifically croplands (farms) and developed areas.  More information about this feature can be found here.

* pymclevel has been included as a submodule.  This release of pymclevel includes an accelerated NBT module which must be compiled before use, as seen below.  This module can have a significant impact on performance.

* The test dataset has been removed.

* The safehouse has been removed, but the default spawn point is still at the highest point in the dataset.

Before running TopoMC

You will need some additional software installed on your system before TopoMC can run.  On Ubuntu (precise pangolin), the following packages need to be installed:  

git gdal-bin python-scipy python-gdal python-suds python-yaml

Other operating systems use other packaging systems so you're on your own -- the error messages will tell you what's missing, but it's up to you to find it and install it!

How to use TopoMC

The best way to get latitude and longitude is through Google Maps.  Choose your chunk of the planet (still limited to the United States and its possessions, alas) and right-click to place latlong markers in the upper left and lower right of the region you wish to model.  This will give you the maximum and minimum latitude and longitude (Y and X in this model) values.

Next, here's what to do!

1.  Import the pymclevel submodule.  Must be done once before anything else.

jmt@belle:~/git/TopoMC$ git submodule init
jmt@belle:~/git/TopoMC$ git submodule update

2.  (Optional) Compile the accelerated NBT module in pymclevel.

jmt@belle:~/git/TopoMC$ (cd pymclevel && python setup_nbt.py build)

3.  Retrieve the region from the USGS.

jmt@belle:~/git/TopoMC$ ./GetRegion.py --name Provincetown --ymax 42.0901 --xmin -70.2611 --ymin 42.0091 --xmax -70.1100

4.  Prepare the region for processing.

jmt@belle:~/git/TopoMC$ ./PrepRegion.py --name Provincetown

5.  Construct the Minecraft world based on the region.

jmt@belle:~/git/TopoMC$ ./BuildRegion.py --name Provincetown

Geek knobs for GetRegion.py

GetRegion.py has a number of optional arguments not shown above.

* Tile size can be changed.
The default tile size is 256x256, but it can be changed.  The only requirement is that it be a multiple of 16.  An example would be "--tilesize 64".

* Horizontal and vertical scaling can be changed.
The horizontal and vertical scale, both of which default to 6, can be changed independently.  The minimum horizontal scale is 1 with a practical maximum of 30.  The minimum vertical scale is more complex, and essentially depends on the elevation change between the highest point in the region and sea level.  An example scale would be "--scale 1 --vscale 1".  Should the requested scale exceed valid parameters, the software will adjust the scale after informing the user.

* Sealevel and maximum depth can be changed.
Sometimes the minimum vertical scale is too high.  One way to improve this situation is to lower the sealevel from its default of 64 to something lower such as 16 or 8.  The maximum depth should also be lowered as well.  Keep in mind that this may have unexpected effects on ore distribution!  An example sealevel and maximum depth would be "--sealevel 16 --maxdepth 8".

* Elevation can be trimmed!
When lowering the sealevel isn't enough to reach your desired vertical scale, excess elevation can be trimmed.  Elevation is considered excess if it is between sea level and the lowest point on the region.  For example, if a region were selected such that its surface was between 200 and 300 meters above sea level, the 200 meters between sea level and the lowest point on the region could be trimmed.  An example trim would be "--trim 200".  If the trim value requested exceeds the valid limits, the software will adjust the trim value to the maximum allowed after informing the user.

The project can be found at https://github.com/mathuin/TopoMC.  A showcase containing sample worlds is available at http://mathuin.org/TopoMC/.

Register or log in to remove.

#2

Acehalo2
    Acehalo2

    Out of the Water

  • Members
  • 2 posts

Posted 26 January 2011 - 07:27 PM

Very interesting concept!

I would really like to see it once you add snow and trees!
I am currently obsessed with:
Hatsune Miku

#3

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 27 January 2011 - 06:52 AM

Updated text and image to reflect tiling support as well as the need to smooth land-cover values.

Once the land-cover values are smoothed, trees are next on my list!

Jack.

#4

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 30 January 2011 - 09:13 PM

Another update, this time with smoothed land cover and trees!  I need to get a dataset with snow and ice to add support for these features -- any suggestions on locations?

Jack.

#5

azures7
    azures7

    Tree Puncher

  • Members
  • 16 posts

Posted 30 January 2011 - 11:05 PM

There's no download link.

#6

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 30 January 2011 - 11:16 PM

azures7 said:

There's no download link.

The software is on github, as is a repository of datasets.

https://github.com/mathuin/TopoMC

I will update the original post next!

Jack.

#7

maus5000AD

Posted 03 February 2011 - 12:19 AM

How does one go about generating new datasets from USGS heightmap data? I live in a moderately hilly part of the Great Plains in the US, I'd love to build on the topography of my county- one thing the default Minecraft map generator seems to lack is a coherent system of water that makes sense. I mean, it looks very pretty, and it's nice, but I'd like to see more rivers, and oceans aside the lakes and ponds we have now.

Is a Dataset generator in the works? I love programs like this, I used to use one for SimCity 4. Hopefully my hometown will be less boring in Minecraft than in SC!

#8

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 03 February 2011 - 04:07 AM

maus5000AD said:

How does one go about generating new datasets from USGS heightmap data? I live in a moderately hilly part of the Great Plains in the US, I'd love to build on the topography of my county- one thing the default Minecraft map generator seems to lack is a coherent system of water that makes sense. I mean, it looks very pretty, and it's nice, but I'd like to see more rivers, and oceans aside the lakes and ponds we have now.

Is a Dataset generator in the works? I love programs like this, I used to use one for SimCity 4. Hopefully my hometown will be less boring in Minecraft than in SC!

I updated the top post with the URL of the project -- https://github.com/mathuin/TopoMC -- and the project's readme tells exactly how to generate new datasets.  It's possible that my work on porting the Javascript part to Python may break things a little -- if that's the case, and you can't figure it out from context, email me and I'll get back to you ASAP.  Hopefully once it's all one language I will be able to keep it more stable.

Jack.

#9

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 11 February 2011 - 01:26 AM

I spent the past week working on a script which will automatically retrieve a dataset from the USGS servers given latitude and longitude information.  I just put the first working version in the github repository.

Here's an extremely huge example:

./getdataset.py --region CapeCod --ymax 42.13 --ymin 41.49 --xmin -70.75 --xmax -69.83

This will stick the ZIP files in Datasets/CapeCod.  You need to go into that directory and unzip the ZIP files, then run the following command from the Datasets directory:

../../TopoMC-Datasets/warpelev.sh CapeCod

Yes, this means you'll have to check out the other project, but I'll fix that soon.

Anyway!  Once you do that, you can run the following commands from the TopoMC directory:

./BuildImages.py CapeCod
./BuildWorld.py CapeCod --world Worlds/CapeCod

And then you can be the first one on your block with a 1:6 scale model of Cape Cod as your Minecraft world.  Enjoy!

Jack.

#10

Zero_X4
    Zero_X4

    Iron Miner

  • Members
  • 306 posts
  • Location: The Wastes

Posted 11 February 2011 - 11:31 PM

Oh, My, God.

For the above example, about how many mb would that be?
Damm Trolls.

#11

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 12 February 2011 - 06:31 AM

Zero_X4 said:

Oh, My, God.

For the above example, about how many mb would that be?

The download from the USGS is about nine megabytes.  It unzips to about ninety megabytes.  I have yet to convert it into a Minecraft world -- I need to go to the computer store tomorrow and buy more RAM since 4G isn't enough. :-)

Jack.

#12

Zero_X4
    Zero_X4

    Iron Miner

  • Members
  • 306 posts
  • Location: The Wastes

Posted 12 February 2011 - 05:07 PM

How long would this take to create?
Damm Trolls.

#13

Zero_X4
    Zero_X4

    Iron Miner

  • Members
  • 306 posts
  • Location: The Wastes

Posted 12 February 2011 - 05:15 PM

Also, should i want this, where on the github page do i download it?
select the entire thing?
Damm Trolls.

#14

Merfnad
    Merfnad

    Tree Puncher

  • Members
  • 26 posts
  • Location: UmeĆ„, Sweden

Posted 13 February 2011 - 09:22 PM

I really want to use this for a server map, but then I would have to have the underground and ores. I can think of one way that maybe has a chance of giving an acceptable result! I haven't had time to test TopoMC yet, but from the image it seems there are a few levels of sand at the bottom of the map followed by one level of water generated around the landmass, and I don't know how the land looks below the top layer but I'll see if I can generate a world tomorrow.

Here's my idea:

First you make a world with this tool, and then make one of the same size with Minecraft Land Generator. Then open the normally generated world in MCEdit (only editor I have any experience with), and select all blocks from the top of the world to a bit below the ocean surface and delete them. Open the TopoMC generated map and copy everything and save to a file (MCEdit does that to move stuff between worlds). I think the editor supports pretty large amounts of blocks, but if the program/computer can't handle all at once you would have to divide it in parts. Go back to the world with the underground and import the topo world files, with some additional editing I think it would turn out good.

#15

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 13 February 2011 - 11:59 PM

Zero_X4 said:

Also, should i want this, where on the github page do i download it?
select the entire thing?

Yep!  If you've used github before, then you know what to do.

If not, this will probably work on a Linux system with git installed:

$ git clone git://github.com/mathuin/TopoMC.git

Then change to the brand new TopoMC directory and have a blast.

As for your other question for how long to process Cape Cod, I hope to find out tonight. :-)

Jack.

#16

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 14 February 2011 - 12:09 AM

Merfnad said:

I really want to use this for a server map, but then I would have to have the underground and ores. I can think of one way that maybe has a chance of giving an acceptable result! I haven't had time to test TopoMC yet, but from the image it seems there are a few levels of sand at the bottom of the map followed by one level of water generated around the landmass, and I don't know how the land looks below the top layer but I'll see
if I can generate a world tomorrow.

Actually, there's now some simulated bathymetry based on how far from shore the water is.  Also, from the irregular surface down about eight or ten blocks it depends on the land cover, but after that it's stone all the way down.  I'm going to post a new image later today. :-)

Merfnad said:

Here's my idea:

First you make a world with this tool, and then make one of the same size with Minecraft Land Generator. Then open the normally generated world in MCEdit (only editor I have any experience with), and select all blocks from the top of the world to a bit below the ocean surface and delete them. Open the TopoMC generated map and copy everything and save to a file (MCEdit does that to move stuff between worlds). I think the editor supports pretty large amounts of blocks, but if the program/computer can't handle all at once you would have to divide it in parts. Go back to the world with the underground and import the topo world files, with some additional editing I think it would turn out good.

I was hoping to leverage the logic behind http://www.minecraft...f=1022&t=138706 what used to be called OreGen.  It seems pretty straightforward -- find a large bit of stone, figure out what ore works for that level and make a fistful of it.  If you have any thoughts on how to do that, I'm all ears!

Jack.

#17

Zero_X4
    Zero_X4

    Iron Miner

  • Members
  • 306 posts
  • Location: The Wastes

Posted 14 February 2011 - 12:18 AM

Essentially marking Land points with higher probabilities of certain ores?
Like, the black hills in south Dakota would have a higher probability of gold, and the Shenandoah's would have higher probabilities of clay and coal?
Damm Trolls.

#18

Efflixi
    Efflixi

    Carpenter

  • Members
  • 66 posts

Posted 14 February 2011 - 12:28 AM

Bookmarking this, sounds like an incredible setup. I will definitely use this once it becomes easier to use and has a few more features. I program for a living (although nothing that would help me to help you) and don't wanna do that much work to get a map made. :)

#19

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 14 February 2011 - 12:32 AM

Zero_X4 said:

Essentially marking Land points with higher probabilities of certain ores?
Like, the black hills in south Dakota would have a higher probability of gold, and the Shenandoah's would have higher probabilities of clay and coal?

If I were able to make a real server mod that would generate real terrain as you cross it, I would definitely try to do that.  At this point, I would be more interested in approximating the percentages as seen in the graph on this page:  http://www.minecraftwiki.net/wiki/Ore.  In fact, the concepts I'm batting around in my head to convert my wetlands from marshy swamps to small pools would be just as useful creating small nodes of ore.

Tunnels and caverns and dungeons (oh my!), well, no clue there.

Jack.

#20

mathuin
    mathuin

    Zombie Killer

  • Members
  • 197 posts

Posted 14 February 2011 - 12:33 AM

Efflixi said:

Bookmarking this, sounds like an incredible setup. I will definitely use this once it becomes easier to use and has a few more features. I program for a living (although nothing that would help me to help you) and don't wanna do that much work to get a map made. :)

Heh!  I don't blame you -- this bit is hard work and I've only scratched the surface -- literally. :-)

If you have suggestions on features, speak up -- if it's not on the to-do list I posted and I think it's interesting, I'll add it.

Jack.