Wierd. I used those same settings and here's what I got:
jmt@eeeve:~/TopoMC$ ./GetDataset.py --region Badlands --ymax 43.8 --ymin 43.7 --xmax -102.3 --xmin -102.4
Retrieving new dataset Badlands...
Checking inventory service for coverage.
Inventory service has the following product IDs: L01,ND3
Configured the following layer IDs: L0102HZ,ND302HZ
Requested URLs for layer ID L0102HZ...
Requested URLs for layer ID ND302HZ...
Received download URLs, downloading now!
Requesting download.
request ID is 20110222.185758516.071117239012
status is
status is Retrieving Remote Archive File
downloading 15156911.zip now!
Requesting download.
request ID is 20110222.185833778.071117239012
status is Extracting Data
status is
status is Retrieving Remote Archive File
downloading 40784872.zip now!
It downloaded two files for me, instead of the one for you. :-( Feel free to throw in print statements -- I'm interested to know where along the line it only finds the one file. I don't have Windows here for testing. :-(
Jack.
I guess I don't get what you're asking.
It decided to work after I redownloaded the USGS data and now in cygwin I get
$ ./warpelev.sh --region badlands2
warning: 59766094.zip appears to use backslashes as path separators
warning: 75762051.zip appears to use backslashes as path separators
./warpelev.sh: line 135: gdalinfo: command not found
./warpelev.sh: line 138: gdalwarp: command not found
and gdalinfo and gdalwarp is not available in cygwin.
It decided to work after I redownloaded the USGS data and now in cygwin I get
$ ./warpelev.sh --region badlands2
warning: 59766094.zip appears to use backslashes as path separators
warning: 75762051.zip appears to use backslashes as path separators
./warpelev.sh: line 135: gdalinfo: command not found
./warpelev.sh: line 138: gdalwarp: command not found
and gdalinfo and gdalwarp is not available in cygwin.
http://pypi.python.org/pypi/GDAL/ has a section which discusses Windows. I have never done it myself, but the directions look reasonable so I'd give it a try, since you need not only the Windows binaries but the Python bindings as well. I just don't have Windows at home anymore, and running Cygwin in Wine doesn't work as well as you'd think it would.
The trouble with that I had was getting the libgdal dependencies.
I wish I knew what to tell you, but in your shoes I'd probably install VirtualBox and run Ubuntu in a VM. Probably not very helpful. :-(
There have been some changes to pymclevel that affect TopoMC -- mostly related to names of materials -- so I'm patching that today in the hopes that soon codewarrior will make pymclevel 1.3-friendly. If not, I'll give it a try and I'll want to start from the most recent source so I need to make these changes either way. I'm resisting the urge to add functionality the best I can, but I am adding one requested fix: the bottom layer of the map will be bedrock so nobody else digs through the floor and falls through space. After that, it's back to the todo list!
I have virtualbox installed and probably have ubuntu somewhere on my harddrive so it's not a big deal.
Well i'm having trouble with osgeo.gdal now in step 3. Ubuntu 10.10
problem solved python-gdal
Processing tile at position (0, 1024)...
Traceback (most recent call last):
File "./BuildWorld.py", line 119, in <module>
sys.exit(main(sys.argv))
File "./BuildWorld.py", line 91, in main
peaks = image.processImages(args.region, args.processes)
File "/home/pjstaab/topomc/image.py", line 142, in processImages
peaks = [processImage(imageDirs[region], offset[0], offset[1]) for (offset, size) in imageSets[region]]
File "/home/pjstaab/topomc/image.py", line 122, in processImage
processTerrain([(lcval, real_x, real_z, elevval, bathyval, crustval)])
File "/home/pjstaab/topomc/terrain.py", line 237, in processTerrain
layers(columns)
File "/home/pjstaab/topomc/mcmap.py", line 73, in layers
setBlocksAt(blocks)
File "/home/pjstaab/topomc/mcmap.py", line 105, in setBlocksAt
myBlocks[x & 0xf, z & 0xf, y] = materials.materialNamed(string)
File "/home/pjstaab/topomc/pymclevel/materials.py", line 48, in materialNamed
return self.names.index(name);
ValueError: list.index(x): x not in list
pymclevel has changed its names for certain materials, specifically water and lava. There are other changes to pymclevel which enable MCRegion-style world files and I am currently testing patches to TopoMC. Sometime tonight I hope to push out some code that will require the latest pymclevel and work without world conversions. Thank you for reporting the issue and I am sorry that I did not respond quicker.
So because I pulled the latest pymclevel it was incompatible with topomc?
Yes. I'm very close to checking in a version of TopoMC that works with that version of pymclevel. Once I figure out why I'm generating a thin layer of stone in the clouds, I'll check it in and announce it here. :-)
So like the world has a stone ceiling? Kind of an odd bug. I'm actually writing a program for class right now and it's no fun.
Pretty much! It was a result of a mistake when attempting to add bedrock to the bottom of the map. Fixed now! I pushed out the new version of the code and will update the first post in the thread, then start the big run which rebuilds six or eight worlds for a showcase. Woo!
This has me really excited! Unfortunately, it requires a bit more technical knowledge than I have...
Could I humbly request that you generate a small part of West Virginia and put it up for download if it isn't too much bother? (I really would like to roam the hills around my home!)
Bottom Right 38.825052,-80.093308
Top Left 39.066434,-80.43251
Oh, something finally managed to entice me enough to make me register so I could post!
I'm trying to make an obnoxiously large map now and I guess python does not have enough heap memory.
C:\TopoMC>buildworld.py --region Badlands --world Worlds/Badlands
Creating world from region Badlands
Traceback (most recent call last):
File "C:\TopoMC\BuildWorld.py", line 119, in <module>
sys.exit(main(sys.argv))
File "C:\TopoMC\BuildWorld.py", line 88, in main
mcmap.initWorld(args.world, minX, minZ, maxX, maxZ, processes)
File "C:\TopoMC\mcmap.py", line 64, in initWorld
arrayData[arrayKey] = SharedMemArray(zeros((chunkWidth,chunkWidth,chunkHeight),dtype=uint8))
File "C:\TopoMC\multinumpy.py", line 108, in __init__
self.data = ndarray_to_shmem(arr)
File "C:\TopoMC\multinumpy.py", line 44, in ndarray_to_shmem
arr.size)
File "C:\Python26\lib\multiprocessing\__init__.py", line 241, in RawArray
return RawArray(typecode_or_type, size_or_initializer)
File "C:\Python26\lib\multiprocessing\sharedctypes.py", line 57, in RawArray
return _new_value(type_)
File "C:\Python26\lib\multiprocessing\sharedctypes.py", line 37, in _new_value
wrapper = heap.BufferWrapper(size)
File "C:\Python26\lib\multiprocessing\heap.py", line 190, in __init__
block = BufferWrapper._heap.malloc(size)
File "C:\Python26\lib\multiprocessing\heap.py", line 170, in malloc
(arena, start, stop) = self._malloc(size)
File "C:\Python26\lib\multiprocessing\heap.py", line 92, in _malloc
arena = Arena(length)
File "C:\Python26\lib\multiprocessing\heap.py", line 38, in __init__
self.buffer = mmap.mmap(-1, self.size, tagname=self.name)
WindowsError: [Error 8] Not enough storage is available to process this command
I'm trying to make an obnoxiously large map now and I guess python does not have enough heap memory.
C:\TopoMC>buildworld.py --region Badlands --world Worlds/Badlands
Creating world from region Badlands
Traceback (most recent call last):
File "C:\TopoMC\BuildWorld.py", line 119, in <module>
sys.exit(main(sys.argv))
File "C:\TopoMC\BuildWorld.py", line 88, in main
mcmap.initWorld(args.world, minX, minZ, maxX, maxZ, processes)
File "C:\TopoMC\mcmap.py", line 64, in initWorld
arrayData[arrayKey] = SharedMemArray(zeros((chunkWidth,chunkWidth,chunkHeight),dtype=uint8))
File "C:\TopoMC\multinumpy.py", line 108, in __init__
self.data = ndarray_to_shmem(arr)
File "C:\TopoMC\multinumpy.py", line 44, in ndarray_to_shmem
arr.size)
File "C:\Python26\lib\multiprocessing\__init__.py", line 241, in RawArray
return RawArray(typecode_or_type, size_or_initializer)
File "C:\Python26\lib\multiprocessing\sharedctypes.py", line 57, in RawArray
return _new_value(type_)
File "C:\Python26\lib\multiprocessing\sharedctypes.py", line 37, in _new_value
wrapper = heap.BufferWrapper(size)
File "C:\Python26\lib\multiprocessing\heap.py", line 190, in __init__
block = BufferWrapper._heap.malloc(size)
File "C:\Python26\lib\multiprocessing\heap.py", line 170, in malloc
(arena, start, stop) = self._malloc(size)
File "C:\Python26\lib\multiprocessing\heap.py", line 92, in _malloc
arena = Arena(length)
File "C:\Python26\lib\multiprocessing\heap.py", line 38, in __init__
self.buffer = mmap.mmap(-1, self.size, tagname=self.name)
WindowsError: [Error 8] Not enough storage is available to process this command
I don't think you can blame Python here. :-) How big did you go? I need to set up some instrumentation to see how long certain things take and how much memory they use, and I know the bite I wanted to take out of the Badlands was about five times bigger than my computer wanted to handle. So sad.
Jack.
(guess I'll upgrade sooner rather than later.)
I think I went half a degree by half a degree. Could you just break up world generation into smaller pieces and then merge them all together?
I actually do that but it's the merge-them-all-together part that's causing the memory usage to shoot through the roof. If someone has written something to merge existing worlds together, that might be worth exploring, just keep in mind that the boundaries of those regions are likely to be a little wacky. Hopefully when the modding API comes out I can write a replacement for the terrain generation routine in Minecraft itself, and then this point becomes moot.
This has me really excited! Unfortunately, it requires a bit more technical knowledge than I have...
Could I humbly request that you generate a small part of West Virginia and put it up for download if it isn't too much bother? (I really would like to roam the hills around my home!)
Bottom Right 38.825052,-80.093308
Top Left 39.066434,-80.43251
Oh, something finally managed to entice me enough to make me register so I could post!
Heh. I've grabbed that region, and it's about a thousand pixels on a side. That'll expand to a total of twenty-five million square meters. I'm building it now, if it looks right I'll add it to the showcase and you can grab it from there!
This has me really excited! Unfortunately, it requires a bit more technical knowledge than I have...
Could I humbly request that you generate a small part of West Virginia and put it up for download if it isn't too much bother? (I really would like to roam the hills around my home!)
Bottom Right 38.825052,-80.093308
Top Left 39.066434,-80.43251
Oh, something finally managed to entice me enough to make me register so I could post!
Heh. I've grabbed that region, and it's about a thousand pixels on a side. That'll expand to a total of twenty-five million square meters. I'm building it now, if it looks right I'll add it to the showcase and you can grab it from there!
Jack.
An update: this area is just a little too big for my computer to handle on normal scale. When I set it to --scale 10, I was able to generate a world. It's on my showcase page: http://www.mathuin.org/TopoMC/ You can download it from there, and you can also cruise around the map.
I guess I don't get what you're asking.
It decided to work after I redownloaded the USGS data and now in cygwin I get
$ ./warpelev.sh --region badlands2
warning: 59766094.zip appears to use backslashes as path separators
warning: 75762051.zip appears to use backslashes as path separators
./warpelev.sh: line 135: gdalinfo: command not found
./warpelev.sh: line 138: gdalwarp: command not found
and gdalinfo and gdalwarp is not available in cygwin.
http://pypi.python.org/pypi/GDAL/ has a section which discusses Windows. I have never done it myself, but the directions look reasonable so I'd give it a try, since you need not only the Windows binaries but the Python bindings as well. I just don't have Windows at home anymore, and running Cygwin in Wine doesn't work as well as you'd think it would.
Jack.
I wish I knew what to tell you, but in your shoes I'd probably install VirtualBox and run Ubuntu in a VM. Probably not very helpful. :-(
There have been some changes to pymclevel that affect TopoMC -- mostly related to names of materials -- so I'm patching that today in the hopes that soon codewarrior will make pymclevel 1.3-friendly. If not, I'll give it a try and I'll want to start from the most recent source so I need to make these changes either way. I'm resisting the urge to add functionality the best I can, but I am adding one requested fix: the bottom layer of the map will be bedrock so nobody else digs through the floor and falls through space. After that, it's back to the todo list!
Jack.
Well i'm having trouble with osgeo.gdal now in step 3. Ubuntu 10.10
problem solved python-gdal
pymclevel has changed its names for certain materials, specifically water and lava. There are other changes to pymclevel which enable MCRegion-style world files and I am currently testing patches to TopoMC. Sometime tonight I hope to push out some code that will require the latest pymclevel and work without world conversions. Thank you for reporting the issue and I am sorry that I did not respond quicker.
Jack.
Yes. I'm very close to checking in a version of TopoMC that works with that version of pymclevel. Once I figure out why I'm generating a thin layer of stone in the clouds, I'll check it in and announce it here. :-)
Jack.
Pretty much! It was a result of a mistake when attempting to add bedrock to the bottom of the map. Fixed now! I pushed out the new version of the code and will update the first post in the thread, then start the big run which rebuilds six or eight worlds for a showcase. Woo!
Jack.
Could I humbly request that you generate a small part of West Virginia and put it up for download if it isn't too much bother? (I really would like to roam the hills around my home!)
Oh, something finally managed to entice me enough to make me register so I could post!
I don't think you can blame Python here. :-) How big did you go? I need to set up some instrumentation to see how long certain things take and how much memory they use, and I know the bite I wanted to take out of the Badlands was about five times bigger than my computer wanted to handle. So sad.
Jack.
(guess I'll upgrade sooner rather than later.)
I actually do that but it's the merge-them-all-together part that's causing the memory usage to shoot through the roof. If someone has written something to merge existing worlds together, that might be worth exploring, just keep in mind that the boundaries of those regions are likely to be a little wacky. Hopefully when the modding API comes out I can write a replacement for the terrain generation routine in Minecraft itself, and then this point becomes moot.
Jack.
(and then the USGS shuts me down, heh)
Heh. I've grabbed that region, and it's about a thousand pixels on a side. That'll expand to a total of twenty-five million square meters. I'm building it now, if it looks right I'll add it to the showcase and you can grab it from there!
Jack.
An update: this area is just a little too big for my computer to handle on normal scale. When I set it to --scale 10, I was able to generate a world. It's on my showcase page: http://www.mathuin.org/TopoMC/ You can download it from there, and you can also cruise around the map.
Jack.
(dreaming of a better computer)