This system will not increase lag! The entire point of this system is to avoid the lag caused by higher height limits.
Table of Contents:
Overview
Changes in how things work
Data storage
Changes to world features
Coping with sunlight
Changes to servers
Render/Load distance alterations
Frequently Asked QuestionsChanges to world features
Coping with sunlight
Changes to servers
Render/Load distance alterations
[WIP] The Cubic Chunks Mod!
Gallery
Supporting this suggestion
Special thanks
[spoiler=Tl:dr]Q: Won't this increase lag?
A: Absolutely not. The title of this suggestion and big red letters at the top of this post aren't lying. If you want to know how, then I suggest you actually read this post.
Q: Will this change terrain? Will this break existing worlds?
A: No. Terrain change is not necessary. Existing worlds can be converted fairly easily with a process described later in this post.
Q: How can sunlight or rain work with infinite vertical space?
A: I suggest you read Coping with sunlight, because it's too complex for a tl;dr.
Q: I have a different question but still don't feel like reading this post. What do I do?
A: Too bad, read the FAQ. I put way too much work into this for the entirety of it to be ignored.[/spoiler]
Overview:
In Minecraft, the sky is the limit - literally. It doesn't matter how many thousands of blocks a player has traveled, or what dimension they're in, or even if they're playing in creative or survival, the highest they can ever build is up to a height of 256. Why is that? If Minecraft can have a world that's infinite in the north, in the south, in the east, and in the west, why can't that world be infinite up and down too?
In Minecraft's earliest days, in Classic and Indev, the world was not infinite in any direction. This was because the entire world needed to be generated at the same time, and the entire world needed to be simulated at the same time as well. This led to a conundrum - the bigger the world, the more it lagged.
Notch didn't like this. He knew his players liked to explore and build large creations, so he found a way to make the world truly infinite. When the Minecraft '' class='bbc'>Infdev came out, it brought with it truly infinite worlds. Suddenly, players could travel hundreds of thousands of blocks in any direction, and never encounter a barrier, or become too laggy.
The Infdev update brought about a very large change to Minecraft worlds to accomplish this feat. For the first time, instead every world being just a single huge piece, they were broken up into a two-dimensional grid of pieces, called chunks. Through breaking the world up into pieces, this 'chunk system' enabled infinite worlds by letting Minecraft create new pieces and simulate them only when it needs to.
Why does that not apply to the vertical axis? Because the type of 'chunk system' Minecraft uses right now is a linear one, which, by using only a two-dimensional grid to map out chunks, means that it is impossible for chunks to stack on top of one another, and by extension, meaning that a single chunk must cover the entire vertical space. This brings back the problem that the Infdev update was supposed to eradicate, now only with chunks, instead of an entire world; the bigger a chunk is, the more laggy it is. You can't just increase the height limit and make chunks taller, because it will become laggier, and laggier, and laggier to do it.
That's why, to fix this, Minecraft must change over to a cubic chunk system. Under this system, 163 block chunks are aligned on a three-dimensional grid, completely eliminating maximum height as an aspect of lag.
The immediate benefits:
•Minecraft worlds become as virtually infinite vertically as they are horizontally: The absolute limit being Y = ±30,000,000.
•A large FPS increase: Alpha testers report an FPS increase of 100~200%.
•Increase in running capability: Computers running Minecraft on Tiny render distance will handle only 30% the blocks they do now.
The possible features:
•Spherical render/load distance: Reduce handled blocks by up to 30% by cutting corners made of unneeded chunks.
•Server chunk occlusion/exclusion: Reduce bandwidth usage and defeat hackers by only sending data for visible chunks.
•Three-dimensional biomes: Save biome data per chunk rather than per block column, create volcanoes with magma chambers, underground rivers, tropical skylands floating over icy taigas, and more.
The disadvantages:
•Unloaded gravity-pause: Falling non-player entities and fluids will be forced to pause their fall if they reach unloaded chunks, but will resume falling when those chunks are loaded.
•Slow falling-pause: Players with slower computers and smaller render distances will have falling occasionally paused as they fall into unloaded chunks, until new chunks can be loaded.
The problems:
•Current sunlight and rain calculation methods cannot work with infinite vertical space: The solution to this is described here.
•Current BiomeDecorator cannot work with multiple vertical chunks simultaneously: The BiomeDecorator code must be altered to function correctly with this, or removed.
•Current cave generation method is executed an extra time for each vertical chunk created simultaneously, leading to lag spikes on world generation: Cave generation's method must be altered to suit this system more.
•Current grass/dirt generation algorithm forces additional chunk requests when chunks are loaded, causing chunks to load slower than they should: This algorithm must be replaced with something else.
Changes in how things work:
Obviously, the implementation of this new chunk system will change quite a few things. These changes are mostly either necessary or in the interest of increased efficiency. Such changes are categorized and explained below.
How worlds will be stored:
[spoiler]How the current storage works, and what changes:
Interestingly enough, the current method of storage, the Anvil format, is derived from the storage method that the original Cubic Chunks mod used. The Anvil format stores individual chunk as a series of 163 quasi-cubic chunks. These 'fake' cubic chunks allow for easier reference of specific data, but they still can't be separated from each other, meaning that it fails to reap the full benefits of this system. Even so, the change allowed Mojang to double the maximum height with no performance hit. Chunks are stored in groups of 322, inside 'MCRegion' files, for a total of 1024 chunks.
By nature, cubic chunks does away with the 'quasi-cubic' nonsense. In terms of chunk grouping, instead of using groups of 323 chunks, new "3DRegion" files would contain groups of 163. This means each 3DRegion file contains 4096 chunks, four times as many as MCRegion files. However, each 3DRegion contains only one fourth the amount of blocks. For per-chunk positional metadata, 3DRegion files would use the same number of bits as MCRegion files, after compression. Calculations show that the same area encompassed by a single MCRegion file would consume 64 kilobytes of extra space with 4 3DRegion files, which is nothing.
Converting existing worlds:
Most people are probably wondering something like "But won't this totally destroy all existing worlds?". Absolutely not; conversion could not be simpler. When a non-cubic world is loaded after the implementation of this system, a conversion process will begin and convert the entire world at once(To avoid making chunk loading take longer during play). First, all existing MCRegion files will be divided into quarters to create 3DRegion files. Then, all existing chunks are divided into sixteenths using the quasi-cubic properties to identify boundaries. After that, conversion is done.
The "isEmpty" flag optimization:
A 1-bit flag is added to each chunk, named "isEmpty". If the chunk consists of 100% air blocks, this bit is 1, any other case makes it 0. When the bit is 1, all data for the chunk besides the isEmpty flag is deleted and ignored, which reduces filesize. Empty chunks are never loaded, and locations where they occur are merely simulated as entities reside in them. The chunk will only load when something requires saving inside it.[/spoiler]
Changes to terrain, ores, etcetera:
[spoiler]Terrain:
By default, nothing will change. Small bits of terrain generation code need to be reconfigured to work properly with Cubic Chunks.
Biomes:
By default, nothing will change.
Ore generation:
By default, nothing will change.
Structures:
By default, nothing will change.
The Void:
After conversion to Cubic Chunks, the void and bedrock layer will still exist and generate as they always have. However, the void(Not the bedrock layer!) will not exist as a hard limit and is able to be moved, but not removed, by editing an associated NBT data tag inside a world's level.dat. This feature, that allows for increasing the maximum depth, is intentionally disabled without external programs, to prevent terrain change of any sort. It is intended to be used by experienced mapmakers and world generation mods only.
Superflat settings:
Existing superflat worlds will not change. However, new superflat worlds will gain a new decoration parameter, 'void'. Inclusion of this parameter will cause the void to form below the lowest defined layer. Exclusion of it will cause all layers below the lowest defined layer to copy the settings of that layer.[/spoiler]
Coping with sunlight:
[spoiler]There used to be a solution here, but it wasn't deemed good enough by Jeb. Suggest solutions in this thread.[/spoiler]
Changes to servers:
[spoiler]Settings:
There's a setting inside the Server.properties file called 'max-build-height'. The setting makes it impossible for any player to place or remove blocks above that height.
With the implementation of Cubic Chunks, a new setting named "maximum-generation-depth" would be added. The void, bedrock layers, and magma layers will generate normally at and above the Y level designated by the value of this setting.
Chunk occlusion/exclusion:
Using the raytracing methods already available in the code and used for explosion calculations, servers can identify which chunks are visible to a player, within safe assumptions, and only send the data for those chunks. This both reduces bandwidth usage, and cripples the usefulness of X-Ray cheats.[/spoiler]
Render/Load distance alterations:
[spoiler]After the implementation of Cubic Chunks, view distances' radii will apply to the vertical axis too. This reduces handled blocks in the cases of tiny and short render distances, and increases them in the cases of normal and far render distances. This can be optimized by utilizing a spherical render distance instead of a cubic one, which would reduce handled blocks in all distances except Far.[/spoiler]
Frequently Asked Questions:
[spoiler=FAQ]Q: This is impossible.
A: No it's not. See below.
Q: Is this available as a mod?
A: Not yet! But it will be!
Q: I like X-ray! What if I don't want it to be broken?
A: First of all, breaking X-ray hacks will only be possible to do in multiplayer. That said, the system that would break X-ray would be possible to disable by the server owner. If the owner doesn't disable the system, then they don't want you using X-ray, and you should not be doing what the server owner doesn't want in the first place.
Q: I play on a PvP/Anarchy/Raid/Faction server. Won't this system let people pillar up into the sky and create a base thousands of blocks in the air and never be found?
A: No.
Q: I like Minecraft's current height limits. What if I don't want to have an infinite sky or infinite underground?
A: If this system is added, all worlds will not automatically gain an infinite underground. As stated below, the Void will remain in all worlds, even after the conversion to Cubic Chunks. The ability to remove the Void will simply be there. As for infinite space in the sky, the current build limit is over one hundred blocks above any terrain that vanilla Minecraft can possibly generate. It is ENTIRELY your decision on whether or not you take advantage of this height. If you play on a server, like stated above, the server owner can set a maximum build height. If s/he doesn't, then don't play on their server - you don't play on servers where the server owners allow things you don't like. Why would you play on an anarchy server if you hate being stolen from and killed?
Q: Will this affect Redstone at all?
A: No. This system will simply make it possible to make larger redstone circuitry than before.
Q: Won't this break existing worlds?
A: No. Existing worlds can be easily converted by dividing each MCRegion file into 4 pieces, then slicing the existing 256 block-high chunks inside them into 16 individual chunks.
Q: Won't this affect mods? Won't mod authors have a hard time updating their mods?
A: The answer to this question depends solely on the answer to the following two questions: Do parts of the modification code rely on chunk data/metadata? Does the mod author want to take advantage of the features of the new chunk system? If the answers to the first and second question are both "No", then updating a mod to this system should be very easy and quick. If the answer to the first question is "Yes", then those parts of the code will need to be rewritten somewhat, but in most cases, the changes should be fairly quick and easy. The only time that it should be hard to update a mod to this system, is if the answer to the second question is "Yes".
Q: Won't this require a total rewrite of the mod API if that's released first?
A: No. Whether or not even a small part of the mod API needs to be rewritten depends on the way that it is implemented and whether or not there are API inclusions for chunk handling and other chunk-related behavior.
Q: Could a player fall into unloaded chunks if chunks aren't loaded fast enough?
A: No, they could not, and for several reasons. Minecraft has a terminal velocity, though it might not seem like it. This velocity is slower than it should take to load new chunks below the player. In cases with exceptionally slow computers, even if the player did manage to reach an unloaded chunk, their fall would be paused until that chunk can be loaded.
Q: What would happen when water, sand, or a mob falls into an unloaded chunk?
A: Nothing. The water/sand/mob would freeze in place until the chunk is loaded and it can continue moving. You can already see this same thing happening on the horizontal axis.
Q: What will happen to the Void?
A: It will still exist, along with all its effects. The only difference is that the Void is no longer a hard limit and it can be moved. After the conversion to Cubic Chunks, the Void's location will be stored in a world's ' class='bbc'>level.dat, and this location can be changed with NBT editing tools. When and where the Void exists, chunks will cease to generate.
Q: Will this affect terrain?
A: No. However, terrain generators will gain the ability to use infinite height.
Q: Will this affect ore generation?
A: No. Ore is a part of terrain generation. As stated above, terrain will not be changed.
Q: Won't all current terrain generators be incompatible with this system and need to be rewritten?
A: No. Terrain generators work independently of chunks. When a chunk is generated for the first time, it calls the terrain generator and receives a specific section of the resultant terrain to save inside itself. Because of this, some custom terrain generators can generate steep terrain all the way to Y256, where you can experience a large, flat cut-off. Since there are no chunks above Y256 to call the terrain generator for terrain, no terrain exists there.
Q: What would happen if there's a huge solid ceiling so far above you that it is unloaded? Wouldn't you just see the sky, just with everything being completely dark?
A: Yes. This already happens on the horizontal axis, and it is an issue with sky rendering, not this chunk system. As such, this has nothing to do with this suggestion. Please do not post about this.
Q: If you go deep underground, will your plants grow/ores smelt/animals grow?
A: No, because those chunks would be unloaded, just as if you had walked far away. This is a flaw with any chunk system, regardless of shape. It is a necessary evil that allows Minecraft to have infinite worlds. The only way to fix this would be to introduce a separate new system that works with chunks as they are loaded and unloaded. This suggestion deals with the chunk system itself, and not sister processes. Because of that, that is outside of the scope of this suggestion. Please do not post about this.[/spoiler]
[WIP] The Cubic Chunks Mod! (Tall Worlds Mod):
Cuchaz has taken it upon himself to bring us the glorious Cubic Chunks, since Mojang refuses to do so.
Cuchaz is using a API of his own creation to help assist in the making of this mod, and he's quite far along, as seen in these two tech demo videos:
[spoiler=T-Demo 1: Vertical chunk loading][/spoiler]
[spoiler=T-Demo 2: Broken height cap and no lag!][/spoiler]
With the basic functionality in place - a complete overhaul of the basic chunk system, and height limit removed - this whole concept can already be considered proven. What remains is making sure everything else functions correctly under the new chunk system. In any case, stay tuned for future updates if it interests you(If it doesn't, then you are the weakest link - goodbye!).
You can follow the mod's development in much more depth in its very own topic!
Gallery:
[spoiler=A mountainside with an experimental engine using Cubic Chunks designed by Nocte. 960 block view radius, and 30 FPS.]
[/spoiler][spoiler=A different view of the mountainside with the same engine by Nocte. This time, with 1600 block view radius and 15 FPS.]
[/spoiler][spoiler="A video demonstrating Nocte's engine."]
Support & Submission to Mojang:
If you support this, hit the rep button in the bottom-left corner of this post. It is the only good way of accurately measuring support here.
If you wish, you can put the following banner, courtesy of laz2727, into your signature. It helps to attract support from all parts of the forum!
[url=http://www.minecraftforum.net/topic/1707097-cubic-chunks-infinite-height-elimination-of-x-ray-and-more-60-supporters/page__st__0][img]http://img833.imageshack.us/img833/443/hov.png[/img][/url]
Please help us get word out of this suggestion! Share this with your friends, with Minecraft celebrities if you're familiar with them, or even with Mojangsters like Jeb or Dinnerbone! (Do not share this with Notch. Notch doesn't work with Minecraft anymore.)
The purpose of this suggestion is to have Cubic Chunks implemented in Vanilla. Being available as a modification does not fulfill that purpose. The modification featured in this suggestion is to act as a proof-of-concept only(Note: Its being featured here is to act as a proof-of-concept. The modification itself is on its way to becoming a fully fledged modification).
Special thanks:
Cuchaz, for taking Barteks' proof and running with it, to give us a truly functional Cubic Chunks mod.
Barteks2x, for updating the Cubic Chunks mod to 1.6.2, proving that it is possible.
Azraile, for posting the original suggestion and allowing me to take ownership of it.
Nocte, for helping resolve flaws and designing Hexahedra.
MineCrak, for a large amount of valuable insight and enthusiasm into the topic of Cubic Chunks.
aaronfranke, for helping resolve flaws.
PanJouda, for creating the original banner.
Flexico, for creating the predecessor to the current banner.
laz2727, for creating the current banner.
Robinton, for creating the original Cubic Chunks mod.
The_Watchman13, for answering all those stupid questions so I don't have to.
Note: Many calculations and information can be found among the many posts of this topic. There are too many for me to cite here, but if you wish, you can search for them yourself.
Let's get some wood! Now that I think about it I'm not sure why I didn't just make an axe... 
I mined 21 stone in this exposed area.
I made some stone tools. Who needs wood tools anyway? =P
I treecapitated some trees with my new axe! I think I'm almost ready to build my house.
I think I'll make my first house in that biome!
Finally sheep. I think I might actually survive my first night.
I found a cave on my way to the spruce biome. I might use it for mining.
I started work on my tree house! It's going to have two rooms. A bedroom and a main room.
I put in a door and chopped in some window space.
A beautiful sunset from atop my house!
I got part of the roof partially finished. I'm going to cover them with stairs after a while. But for now this will do.
You know how the title says this is a "tree" house? Here's why! I ended up removing the actual trees. =P
Here's the completed house from the outside!
And to end off this update, two images from the inside! I hope you enjoyed it! =D
Yes! I found a cave closer to my house to explore.
I found 8 iron right off the bat.
That entire area was coal with a few iron.
A new cave! This one looks promising. I'll go in as soon as I smelt my iron. 
I made some iron tools and an iron chestplate. Let's go caving!
After searching for about 5 minutes, I finally found it again. Let's see what this cave has to offer!
A bat! I've halfway completed one of my goals.
You have arrived at facepalm city. At least we found a bat....
Let's go caving!
This sure is a nice welcome!
An abandoned mine shaft!
Before going in I made some decent stuff to wear.
This room is..A bit weird. Random bookshelves anyone?
Some gold! 
This is my complete inventory at the moment.
I thought I made dungeons LESS common...
Hi, zombies! I come in peace!
They didn't come in peace. D: I opened the door and got rushed by: two baby zombies and five normal ones. Ow.
Because of this I'm going to uninstall this mod and roguelike dungeons will take it's place. I didn't change difficulty to cheat, I just think that some of these things are WAY to OP.The Success! (2.3)
Time to explore the mineshaft!
Worst. Loot. Ever.
I hate how infernal creepers always do that before I can kill them. 
This is more like it!
An infernal zombie was kind enough to drop...An iron hoe. Woopie.
At least it dropped an enderpearl!
Somehow a squid spawned here and died. lol
On a side note, between updates I made some iron furnaces.
I also went and made some ore doubling upgrades!
I decided to start a farm. It will start off by having melons and wheat! I'll add other things later.
Here is the farm so far, I like the stone torches. They look pretty cool.
For this modpack I'm going to start using the Crafteryada texture pack. It's still in beta so some things are the default textures.
Infernal spider. I didn't know I ahd particles turned off, but that's what it is!
I made an add-on to my house for the Nether portal. I had some problems making it look good due to the location of the farm. I think this setup looks alright.
I made the portal! Next update I will venture into the depths..
1
Must find new pants.
1
/thread
1
14
XD I was just pointing out that the fact that he put that in there seemed a little strange to me. Like he felt that he needed to assure every single person that read his post (all 15 of them) that he was not, in fact, forever alone.
9
-pauses to wish he had a girlfriend-*
1
I'd support that completely.
SNARK IS MY LIFE.
Plus, when you feed trolls, you get to play with them. No harm in that.
1
1
1
3