So, I downloaded the FTB launcher and FTB: Ultimate on my new computer, and after about one minute of game play, not loading many chunks, the game's console starts spitting "out of memory" errors at me and the FPS dropped from about 100 to 0. My old computer was able to run this just fine. The base game of Minecraft runs without any problems. I have 12 gigs of ram. I can supply other info if needed.
If there's already a thread for this problem, sorry but at the moment, I didn't care and wasn't about to search for one.
You need to assign more RAM to Minecraft via the FTB Launcher. Be default it uses only 1.5GB I think. Just don't give it too much. 2-4GB should do fine, more then 8 is useless.
I think in this case the problem is due to running Minecraft on 32 bit Java; I've seen this same error myself when I used the default 1 GB allocation and try to use "Far" render distance in vanilla 1.6.4 (at least); the debug screen doesn't indicate any memory problems but it starts to lag then freezes and and then shows "Minecraft has run out of memory" with GL out of memory errors in the console; interestingly enough, if I reduce the memory allocated to 512 MB it runs without problems - the issue is that 32 bit Java can only use 2 GB of process space, which includes not just the JVM itself but also memory required by Open GL, which is where the problem is occurring (not enough memory for Open GL); as seen in Task Manager, the java process uses a lot more memory than indicated in-game (after playing for several hours with 512 MB allocated total process usage is over 1 GB).
Of course, with a big modpack like FTB you can't allocate less memory without running out of heap space so upgrading to 64 bit Java is the only practical solution. Also, as HD texture packs require more memory for textures this can also be an issue. Note that when downloading Java some browsers will try to download the 32 bit version even if you have a 64 bit OS because they are 32 bit themselves (32 bit software is still very common because it is compatible with 32/64 bit OSs and they don't need any of the 64 bit advantages) you'll need to make sure you get the 64 bit (x64) version; here is a page with all Java versions available; note that you may want to keep 32 bit Java on your system; you can ensure Minecraft starts with the right version by adding -d64 to the JVM arguments line in the profile editor.
Also, for big modpacks I'd recommend adding -XX:MaxPermSize=128m as well to avoid out of memory problems with permgen space (which again causes some people to try to fix it by allocating more heap space but that isn't the correct solution).
If there's already a thread for this problem, sorry but at the moment, I didn't care and wasn't about to search for one.
I think in this case the problem is due to running Minecraft on 32 bit Java; I've seen this same error myself when I used the default 1 GB allocation and try to use "Far" render distance in vanilla 1.6.4 (at least); the debug screen doesn't indicate any memory problems but it starts to lag then freezes and and then shows "Minecraft has run out of memory" with GL out of memory errors in the console; interestingly enough, if I reduce the memory allocated to 512 MB it runs without problems - the issue is that 32 bit Java can only use 2 GB of process space, which includes not just the JVM itself but also memory required by Open GL, which is where the problem is occurring (not enough memory for Open GL); as seen in Task Manager, the java process uses a lot more memory than indicated in-game (after playing for several hours with 512 MB allocated total process usage is over 1 GB).
Of course, with a big modpack like FTB you can't allocate less memory without running out of heap space so upgrading to 64 bit Java is the only practical solution. Also, as HD texture packs require more memory for textures this can also be an issue. Note that when downloading Java some browsers will try to download the 32 bit version even if you have a 64 bit OS because they are 32 bit themselves (32 bit software is still very common because it is compatible with 32/64 bit OSs and they don't need any of the 64 bit advantages) you'll need to make sure you get the 64 bit (x64) version; here is a page with all Java versions available; note that you may want to keep 32 bit Java on your system; you can ensure Minecraft starts with the right version by adding -d64 to the JVM arguments line in the profile editor.
Also, for big modpacks I'd recommend adding -XX:MaxPermSize=128m as well to avoid out of memory problems with permgen space (which again causes some people to try to fix it by allocating more heap space but that isn't the correct solution).
TheMasterCaver's First World - possibly the most caved-out world in Minecraft history - includes world download.
TheMasterCaver's World - my own version of Minecraft largely based on my views of how the game should have evolved since 1.6.4.
Why do I still play in 1.6.4?