I wanted to report that on OSX 10.6.6 with a vanilla Python 2.7.1 install, you need to install the following dependencies first before TopoMC will work:
If you've never installed Python packages before, most of them come with a setup.py file. All you need to do is type:
python setup.py install
and it should install the package.
For setuptools, you need to download the .egg file and run it as a shell script:
sh setuptools-xxxx.egg
I'd also like to point out that if you don't know how to download files from github and are unfamiliar with shells, you should wait until this project becomes more user friendly because it is beyond the scope of your technical knowledge.
I have finally added ore. The code is a little complicated and messy but it does work. I basically use the data shown at http://www.minecraftforum.net/viewtopic.php?f=35&t=28299 to figure out about how often I need to generate each kind of ore and about how big each ore vein should be. My own special brand of insanity here is choosing to model an ideal ellipsoid so veins are three-dimensional!
This is only a first pass -- I expect there will be a lot more tweaking for performance if nothing else -- but you should now be able to actually play the game for real now that there's trees on the surface and ore on the ground.
Please give it a try and let me know how it goes...
Here's some preliminary results on adding ore to the standard Block Island image. I'm including the terrain and tree statistics as well to give an idea of the size of the world -- about 2000 chunks or so.
Today's push is for some minor improvements and bug fixes. The biggest change is that another tile type is generated during the BuildImages phase: crust. Crust is the layer between the surface and the stone -- it used to be generated by a random number but now it's smoothed over the course of the map. The performance impact was light so I think it's worth it. This feature request came from DopeGhoti on the IRC channel, so if you don't like it, blame him and me. :-)
The one big bug I'm having trouble fixing is the rotated-rectangles thing. It doesn't show up on Block Island because the affected area is underwater, but it does show up on land. It's due to the reprojection of the elevation map to match the land cover -- I'm looking for a fix. Right now I'm leaning towards "ask for a bigger elevation map" but I need to figure out just how much bigger. Feel free to make suggestions!
Hi !
My girlfriend (who is a developer) tried this week-end to launch your program, but some modules are missing...
This week, we install Linux ! :smile.gif:
I have a question : can you place an option in your script allowing us to change the map size ? (just the height for example, and the width scale automaticly)
Have you an idea for the caves ? :biggrin.gif: You're on it ?
Thanks a lot !! :smile.gif:
(and for your question about ore... sorry, I can't answer you !!)
Hurrah for girlfriends who are developers! If you guys make a list of what you had to install to run TopoMC, I'll put that in the documentation.
There are some commands to change the horizontal and vertical scales -- if you want a really small map, use "--scale 30" but it's very bumpy and not smooth at all, and if you want a flatter map, use "--vscale 10" and the generator will increase in-game height by one cube for every ten meters in real-life. If you want to live large and full-scale and have a seriously powerful computer (or a seriously tiny world in mind), use "--scale 1 --vscale 1". In addition, there's ways to only work with parts of existing datasets. Try "./BuildImages.py -h" and "./BuildWorld.py -h" to see the options.
As for caves, the best idea I've come up with is similar to "Hunt the Wumpus":
* generate a bunch of "rooms" (rounded three-dimensional spaces converted from stone to air)
* make tunnels between the rooms such that each room has three tunnels to it
* put lava and water and crap in the rooms
* connect one room to the surface by a stairway or something
But that's hard so I'm focusing on easier stuff like the new crust code. :-)
Maybe you could make it as an option (false/true) in the script !
Personnaly, I think this a good idea, but maybe people don't want it by default...
I am probably going to make bathymetry, ore placement, and crust mapping as options once the code is feature-complete, but I'm still figuring out what that really means so there's no hurry. If someone's particularly upset with a performance hit, I can expedite that option-making, so don't hesitate to ask.
I downloaded your source and immediately tried it out by using the BlockIsland Datasets that you provided.
BuildImages processed just fine, but when I run BuildWorld, it doesn't seems to work at all...
Traceback (most recent call last):
File "./BuildWorld.py", line 119, in <module>
sys.exit(main(sys.argv))
File "./BuildWorld.py", line 91, in main
peaks = image.processImages(args.region, args.processes)
File "/home/josiah/MineCraft/TopoMC/image.py", line 147, in processImages
peaks = [x for x in results]
File "/usr/lib/python2.6/multiprocessing/pool.py", line 520, in next
raise value
ValueError: list.index(x): x not in list
And I end up with only one level.dat file in world folder. I tried to run it in MineCraft, and it doesn't work.
I downloaded your source and immediately tried it out by using the BlockIsland Datasets that you provided.
BuildImages processed just fine, but when I run BuildWorld, it doesn't seems to work at all...
Traceback (most recent call last):
File "./BuildWorld.py", line 119, in <module>
sys.exit(main(sys.argv))
File "./BuildWorld.py", line 91, in main
peaks = image.processImages(args.region, args.processes)
File "/home/josiah/MineCraft/TopoMC/image.py", line 147, in processImages
peaks = [x for x in results]
File "/usr/lib/python2.6/multiprocessing/pool.py", line 520, in next
raise value
ValueError: list.index(x): x not in list
And I end up with only one level.dat file in world folder. I tried to run it in MineCraft, and it doesn't work.
Yeah, that was a booboo. Do another git pull, and you'll not only have that bug fixed, you'll have some new features. :-)
I chose some places across the country for comparison purposes and determined that the angle of rotation of the rectangle is related to the longitude of the location sampled. If you're curious, check out St. Thomas (48.6937, -97.5030 to 48.5830, -97.3924) for a barely-rotated rectangle, Lubec (44.8790, -67.0760 to 44.8072, -66.9860) for a rotated-left rectangle, and Disneyland (33.81839, -117.92871 to 33.80298, -117.91494) for a rotated-right rectangle.
By the way, the vast beach in the Lubec map is "no data land" -- in this case, Canada -- and I haven't yet written the code to turn it into water. I thought I had, but it doesn't yet work. Fooey.
Ok, i'm stuck, I can't figure out how to use warpelev on windows. Any ideas?
Got it, had to d2u in cygwin.
$ ./warpelev.sh --region badlands
80996710 is not of the correct type
warning: 80996710.zip appears to use backslashes as path separators
No elevation directory found!
Unknown product ID found!
Ok, i'm stuck, I can't figure out how to use warpelev on windows. Any ideas?
Got it, had to d2u in cygwin.
$ ./warpelev.sh --region badlands
80996710 is not of the correct type
warning: 80996710.zip appears to use backslashes as path separators
No elevation directory found!
Unknown product ID found!
Can you rerun the ./GetDataset.py command you ran to get the badlands region and show me the output? Thanks!
I realize something : you take your data on USGC. Does it mean it only work for US data or for the rest of the world too ?
Not only am I limited to the US, I'm limited to the coterminous 48 states. I don't yet support Alaska or Hawaii (the USGS treats them different, to support them I'd have to redo terrain.py which I want to do after I clean up some other code) and I think Puerto Rico, Guam, and other territories are in the same boat as the the 49th and 50th states.
If you can find datasets for your location of choice (elevation and land cover) and documentation for them, I'll see what I can do!
By the way, the vast beach in the Lubec map is "no data land" -- in this case, Canada -- and I haven't yet written the code to turn it into water. I thought I had, but it doesn't yet work. Fooey.
Just pushed the fix for this out there. It also fixed a bug where "ice" wasn't considered "water" with regard to calculating bathymetry. Better now!
The rectangle one is harder, though. I really wish the USGS would fix it for me -- it's their bug I have to work around. Feh. :-)
Not only am I limited to the US, I'm limited to the coterminous 48 states. I don't yet support Alaska or Hawaii (the USGS treats them different, to support them I'd have to redo terrain.py which I want to do after I clean up some other code) and I think Puerto Rico, Guam, and other territories are in the same boat as the the 49th and 50th states.
If you can find datasets for your location of choice (elevation and land cover) and documentation for them, I'll see what I can do!
Jack.
I would really love to see this working for Canada too! Elevation and Landcover data for the country are freely available at http://www.geobase.ca/geobase/en/data/index.html
The digital elevation models are in an ASCII format with .dem file extension, which they claim should be directly compatible with anything that can read USGS Digital Elevation Models. The landcover is in GML or .shp format.
I suspect this might be an absolutely insane amount of work and am not seriously asking you to look at it, but the Soil Landscapes classifications http://sis.agr.gc.ca/cansis/nsdb/slc/intro.html might work to identify block types, water features and biomes too.
I would really love to see this working for Canada too! Elevation and Landcover data for the country are freely available at http://www.geobase.ca/geobase/en/data/index.html
The digital elevation models are in an ASCII format with .dem file extension, which they claim should be directly compatible with anything that can read USGS Digital Elevation Models. The landcover is in GML or .shp format.
I suspect this might be an absolutely insane amount of work and am not seriously asking you to look at it, but the Soil Landscapes classifications http://sis.agr.gc.ca/cansis/nsdb/slc/intro.html might work to identify block types, water features and biomes too.
~JD
I took a look at the Canadian website. They have some interesting data but the formats are incompatible so it'd be a bit of work. Still, it's not impossible - and if you feel like giving it a try, I can help you off-thread. The goal would be to convert the landcover data to raster format to meters (not degrees) ideally at 30m resolution and then warp the elevation data to match the landcover data. If you can do that, I can guarantee we can generate a world for you. :-)
In addition to the format issue, there's the whole "everyone defines land cover differently" thing. I'm already facing that for LC 1992 versus LC 2001 data, and fixing that is on my to-do list. Once that and multiple-chunks are done, I will be ready to deal with new formats and stuff.
You are, of course, welcome to fork the repository and give it a whirl. :-)
Quick question, will you be building worlds with the new 1.3 format soon? Not really important since MC can do that for you.
Retrieving new dataset badlands...
Checking inventory service for coverage.
Inventory service has the following product IDs: L01,ND3
Configured the following layer IDs: L0102HZ,ND302HZ
Requested URLs for layer ID L0102HZ...
Requested URLs for layer ID ND302HZ...
Received download URLs, downloading now!
Requesting download.
request ID is 20110222.161503997.151159020002
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Data
status is Extracting Metadata
status is Extracting Metadata
status is Extracting Metadata
status is Extracting Metadata
status is Extracting Metadata
status is Extracting Metadata
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is Archiving Data Request
status is
status is Retrieving Remote Archive File
downloading 97072762.zip now!
C:\TopoMC>
Quick question, will you be building worlds with the new 1.3 format soon? Not really important since MC can do that for you.
I either have to wait for codewarrior to port pymclevel to 1.3 or port it myself. If it's not done by the weekend I'll take it on as a project.
I am fascinated by the fact that it tells us it's checking for two files but only downloads one. What latitudes and longitudes did you use? I want to run it on Unix and see if there's something about Windows that's making the code act out of order.
Wierd. I used those same settings and here's what I got:
jmt@eeeve:~/TopoMC$ ./GetDataset.py --region Badlands --ymax 43.8 --ymin 43.7 --xmax -102.3 --xmin -102.4
Retrieving new dataset Badlands...
Checking inventory service for coverage.
Inventory service has the following product IDs: L01,ND3
Configured the following layer IDs: L0102HZ,ND302HZ
Requested URLs for layer ID L0102HZ...
Requested URLs for layer ID ND302HZ...
Received download URLs, downloading now!
Requesting download.
request ID is 20110222.185758516.071117239012
status is
status is Retrieving Remote Archive File
downloading 15156911.zip now!
Requesting download.
request ID is 20110222.185833778.071117239012
status is Extracting Data
status is
status is Retrieving Remote Archive File
downloading 40784872.zip now!
It downloaded two files for me, instead of the one for you. :-( Feel free to throw in print statements -- I'm interested to know where along the line it only finds the one file. I don't have Windows here for testing. :-(
setuptools: http://pypi.python.org/pypi/setuptools#files
lxml: http://www.google.com/search?sourceid=c ... F-8&q=lxml
suds: https://fedorahosted.org/suds/
If you've never installed Python packages before, most of them come with a setup.py file. All you need to do is type:
and it should install the package.
For setuptools, you need to download the .egg file and run it as a shell script:
I'd also like to point out that if you don't know how to download files from github and are unfamiliar with shells, you should wait until this project becomes more user friendly because it is beyond the scope of your technical knowledge.
This is only a first pass -- I expect there will be a lot more tweaking for performance if nothing else -- but you should now be able to actually play the game for real now that there's trees on the surface and ore on the ground.
Please give it a try and let me know how it goes...
Jack.
Land cover statistics (527574 total):
417715 (79.18%): Water
35017 (6.64%): Developed/Open-Space
20078 (3.81%): Deciduous Forest
17508 (3.32%): Woody Wetlands
14573 (2.76%): Mixed Forest
11738 (2.22%): Barren Land
3459 (0.66%): Evergreen Forest
3113 (0.59%): Developed/Low-Intensity
2572 (0.49%): Emergent Herbaceous Wetlands
1792 (0.34%): Developed/High-Intensity
5 (0.00%): Cultivated Crops
4 (0.00%): Shrub/Scrub
Tree statistics (1531 total):
774 (50.56%): Redwood
451 (29.46%): Regular
301 (19.66%): Birch
5 (0.33%): Cactus
Ore statistics (988902 total nodes, 115640 total veins):
364958 (19232 veins): Dirt
260227 (37194 veins): Iron
175481 (9248 veins): Gravel
130640 (18673 veins): Coal
30688 (4385 veins): Gold
17937 (17937 veins): Redstone
6740 (6740 veins): Lapis Lazuli
2231 (2231 veins): Diamond
Does this look right? Redstone, Lapis Lazuli, and Diamond are all one-block-per-vein -- is that normal? I don't play the game enough to know. *shame*
Thanks!
Jack.
The one big bug I'm having trouble fixing is the rotated-rectangles thing. It doesn't show up on Block Island because the affected area is underwater, but it does show up on land. It's due to the reprojection of the elevation map to match the land cover -- I'm looking for a fix. Right now I'm leaning towards "ask for a bigger elevation map" but I need to figure out just how much bigger. Feel free to make suggestions!
Jack.
Hurrah for girlfriends who are developers! If you guys make a list of what you had to install to run TopoMC, I'll put that in the documentation.
There are some commands to change the horizontal and vertical scales -- if you want a really small map, use "--scale 30" but it's very bumpy and not smooth at all, and if you want a flatter map, use "--vscale 10" and the generator will increase in-game height by one cube for every ten meters in real-life. If you want to live large and full-scale and have a seriously powerful computer (or a seriously tiny world in mind), use "--scale 1 --vscale 1". In addition, there's ways to only work with parts of existing datasets. Try "./BuildImages.py -h" and "./BuildWorld.py -h" to see the options.
As for caves, the best idea I've come up with is similar to "Hunt the Wumpus":
* generate a bunch of "rooms" (rounded three-dimensional spaces converted from stone to air)
* make tunnels between the rooms such that each room has three tunnels to it
* put lava and water and crap in the rooms
* connect one room to the surface by a stairway or something
But that's hard so I'm focusing on easier stuff like the new crust code. :-)
Jack.
I am probably going to make bathymetry, ore placement, and crust mapping as options once the code is feature-complete, but I'm still figuring out what that really means so there's no hurry. If someone's particularly upset with a performance hit, I can expedite that option-making, so don't hesitate to ask.
Jack.
BuildImages processed just fine, but when I run BuildWorld, it doesn't seems to work at all...
And I end up with only one level.dat file in world folder. I tried to run it in MineCraft, and it doesn't work.
Yeah, that was a booboo. Do another git pull, and you'll not only have that bug fixed, you'll have some new features. :-)
Jack.
I chose some places across the country for comparison purposes and determined that the angle of rotation of the rectangle is related to the longitude of the location sampled. If you're curious, check out St. Thomas (48.6937, -97.5030 to 48.5830, -97.3924) for a barely-rotated rectangle, Lubec (44.8790, -67.0760 to 44.8072, -66.9860) for a rotated-left rectangle, and Disneyland (33.81839, -117.92871 to 33.80298, -117.91494) for a rotated-right rectangle.
By the way, the vast beach in the Lubec map is "no data land" -- in this case, Canada -- and I haven't yet written the code to turn it into water. I thought I had, but it doesn't yet work. Fooey.
Jack.
Got it, had to d2u in cygwin.
$ ./warpelev.sh --region badlands
80996710 is not of the correct type
warning: 80996710.zip appears to use backslashes as path separators
No elevation directory found!
Unknown product ID found!
Can you rerun the ./GetDataset.py command you ran to get the badlands region and show me the output? Thanks!
Jack.
Not only am I limited to the US, I'm limited to the coterminous 48 states. I don't yet support Alaska or Hawaii (the USGS treats them different, to support them I'd have to redo terrain.py which I want to do after I clean up some other code) and I think Puerto Rico, Guam, and other territories are in the same boat as the the 49th and 50th states.
If you can find datasets for your location of choice (elevation and land cover) and documentation for them, I'll see what I can do!
Jack.
Just pushed the fix for this out there. It also fixed a bug where "ice" wasn't considered "water" with regard to calculating bathymetry. Better now!
The rectangle one is harder, though. I really wish the USGS would fix it for me -- it's their bug I have to work around. Feh. :-)
Jack.
I would really love to see this working for Canada too! Elevation and Landcover data for the country are freely available at http://www.geobase.ca/geobase/en/data/index.html
The digital elevation models are in an ASCII format with .dem file extension, which they claim should be directly compatible with anything that can read USGS Digital Elevation Models. The landcover is in GML or .shp format.
I suspect this might be an absolutely insane amount of work and am not seriously asking you to look at it, but the Soil Landscapes classifications http://sis.agr.gc.ca/cansis/nsdb/slc/intro.html might work to identify block types, water features and biomes too.
~JD
I took a look at the Canadian website. They have some interesting data but the formats are incompatible so it'd be a bit of work. Still, it's not impossible - and if you feel like giving it a try, I can help you off-thread. The goal would be to convert the landcover data to raster format to meters (not degrees) ideally at 30m resolution and then warp the elevation data to match the landcover data. If you can do that, I can guarantee we can generate a world for you. :-)
In addition to the format issue, there's the whole "everyone defines land cover differently" thing. I'm already facing that for LC 1992 versus LC 2001 data, and fixing that is on my to-do list. Once that and multiple-chunks are done, I will be ready to deal with new formats and stuff.
You are, of course, welcome to fork the repository and give it a whirl. :-)
Jack.
I either have to wait for codewarrior to port pymclevel to 1.3 or port it myself. If it's not done by the weekend I'll take it on as a project.
I am fascinated by the fact that it tells us it's checking for two files but only downloads one. What latitudes and longitudes did you use? I want to run it on Unix and see if there's something about Windows that's making the code act out of order.
Jack.
Wierd. I used those same settings and here's what I got:
It downloaded two files for me, instead of the one for you. :-( Feel free to throw in print statements -- I'm interested to know where along the line it only finds the one file. I don't have Windows here for testing. :-(
Jack.