(Forum admins: If this properly belongs in the server administration section, please move it. As it also affects normal client/internal server, I'm putting it in normal discussion).
EDIT:
The short version:
1. The default garbage collector is a worst-case fit to minecraft. 2. Optifine, at least 147 D5, has a memory leak that will cause the client to never be happy. not confirmed; actually seems to be caused by something else.
3. A launcher for servers can be found at http://pastebin.com/rY3gPVGE
-- this will also provide log files to quickly see how memory can be adjusted for your server's needs. This launcher uses Concurrent-Mark-Sweep. That link is for version 1.7.1, release 1.
4. Client scripts are harder, as most "launchers" (that handle your login/password) want to define their own memory management, and will not say "Use what was used to run me". ("minecraft.exe" on windows systems will, but window's "cmd" lacks the features to run that script). An offline launcher based on that server script is trivial; command flags to give to "smart launchers" are harder. Coming soon.
5. Switching to G1GC, in initial testing, consumes significantly more memory while providing some performance improvement. Further testing is on my ToDo list.
What's the oddball in this list:
1. Easter Bunny
2. Fast Java Programs
3. Santa Clause
4. Benevolent Government
** The default Java garbage collector (https://blogs.oracle..._monitoring_and, http://www.oracle.co...00744-en-in.pdf) is optimized for total throughput using all cores regardless of system load. It has no concern about responsiveness (latency). Not only did testing last night match every case of client lag with an increase in "small GC" activity, when a full GC was triggered it was often longer than 30 seconds (forcing a disconnect) and was 95 seconds in the worst case).
Background: Java -- both Sun/Oracle's, and Apple's, in Java 6, have at least three different garbage collectors, all of which have tunable options. The default garbage collector is "parallel new", which is optimized for total throughput of large heap spaces (> 1gb). In comparison, "parallel old" is regarded as a better choice for smaller heap spaces (but must be explicitly requested), "single-threaded old" is better for small heap spaces on single core CPU's, and either "concurrent mark sweep" or "incremental" is better on large heap, multi-core cpu's.
In other words, the default garbage collector is the worst match for minecraft except possibly on large memory single core systems.
===
(http://www.oracle.co...00744-en-in.pdf has steps to take to measure your system's actual memory usage, and then adjust the VM's parameters for your system.)
If you know how to add the flag -XX:+PrintCommandLineFlags to your minecraft (client or server), then please help me collect data here. Follow along, test out your system, and lets get some ideas on how Mojang can set default settings to improve minecraft performance. If you don't know how, then I will apologize in advance: I have higher priorities (aka "don't have the time") than getting your data. I could tell you that if you use Magic Launcher to use the "Advanced" tab, and enter it there. I could tell you that if you run a server, you are either currently using a script to start it, or if you are using the windows' EXE server then you can get the jar server and script from Mojang's web site. But the level of hand holding you will need after that is more time then I'm willing to spend on this.
===
So here is what I did.
(Mod list: Client and server: 1.3.2 versions:
Forge
Mystcraft (linking books/portals were used; new ages were not)
Extra Biomes (was not used in this test)
Twilight Forest (was not used in this test)
Superior Enchanting (was not used in this test)
Client only:
Optifine (B3)
MineVideo
Too Many Items
Saturation Bar
What's My Light Level (33, saturation bar compatibility)
Durability Warning
Server only:
Ruins
Clay Soil
FallFix
More village biomes and Village logic addon
Water Propagation Fix
Larga-isso-enderman ("drop that block, enderman")
Custom Ore Gen (was used; new chunks generated)
None of these makes major demands on the system (the ones that would were not used), so I think these are representative mods.
Step 1: On the server: -d32 -server -XX:+UseParallelOldGC -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log (32 bit is recommended by oracle for 2.5GB or smaller heaps.)
On the client: -XX:+UseParallelOldGC -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log (NB: This leaves it in 64 bit mode for far vision support)
Note that Oracle specifically states to start with parallel old GC for collecting data, and go from there. It also says to start by tossing more memory than you need and see what gets used.
Step 2:
In order to get useful information, you need to trigger a full garbage collect from out of memory. The client does multiple full garbage collects when you disconnect / reconnect to a server. I consistently saw a total of 3, and the first and third both had large amounts of memory refunded. (One on disconnect; two on connection).
The client seems to need a full GC every 300-1000 seconds. Larger render distance resulted in more frequent GC's (kinda obvious, but a good sanity test). With 8 threads on an i7 4+4 @2.2 GHz, the observed times were 12.4 sec (during play), 4 sec (manual disconnect/reconnect), 77 seconds (server DC'd me for being non responsive), 61 seconds (ditto), and 95 seconds (last one, after the server itself went unresponsive).
The server took about 3 hours to finally run out; it needed 61 seconds to garbage collect.
Step 3: Look at the numbers
The server:
12642.340: [Full GC [PSYoungGen: 6841K->0K(419776K)] [ParOldGen: 850825K->96766K(853376K)] 857666K->96766K(1273152K) [PSPermGen: 19671K->19592K(39424K)], 30.4059153 secs] [Times: user=0.68 sys=1.43, real=30.41 secs]
Heap
PSYoungGen total 419776K, used 98290K [0000000040970000, 000000005aa10000, 000000005aa10000)
eden space 412928K, 23% used [0000000040970000,000000004696c8e8,0000000059cb0000)
from space 6848K, 0% used [0000000059cb0000,0000000059cb0000,000000005a360000)
to space 6848K, 0% used [000000005a360000,000000005a360000,000000005aa10000)
ParOldGen total 853376K, used 96766K [000000000c810000, 0000000040970000, 0000000040970000)
object space 853376K, 11% used [000000000c810000,000000001268fa10,0000000040970000)
PSPermGen total 39424K, used 19601K [0000000008810000, 000000000ae90000, 000000000c810000)
object space 39424K, 49% used [0000000008810000,0000000009b34570,000000000ae90000)
So the total space after collecting is about 120MB for one client, and the old gen is about 100 MB. Perm is about 20 MB (mostly classes)
Perm hits a max of about 45 MB, and old is around ... well, that's actually hard to say. Note that it kept going up, and was over 300 MB at the end. I'm going to guess that it will be happy around 400-450 MB.
According to page 25 of Oracle's slide show:
Initial Heap Configuration
Rule of thumb
– Set -Xms and -Xmx to 3x to 4x of LDS
– Set both -XX:PermSize and -XX:MaxPermSize to around 1.2x to 1.5x the max perm gen size
Set the generation sizes accordingly
– Young gen should be around 1/3 - 1/4 of the heap size
Young gen should be around 1x to 1.5x of LDS
Old gen should be around 2x to 3x LDS – Eg. for LDS of 512m
-Xmn786m -Xms2g -Xms2g
If heap size is around or smaller than 1.5x LDS
– Might be too tight for GC
– GC might be excessive and overwhelms the application
Now, we can draw some conclusions here:
#1: There seems to be a serious memory leak in the client.
The increase in "OldGen" from around 150-170 MB to 230 MB corresponds to the change from "Short + 32" render distance to "far" render distance. The final jump at the end is a mystery to me -- this is AFTER the server timed out and the client DC'd.
#2: Oracle recommends putting the max size at least 3 times the "long-lived data" size -- which goes well over 1 GB. Yes, minecraft may run in less, but for garbage collection performance, it needs more.
#3: As soon as you go over 1 GB, parallel old is no longer recommended. For low latency, oracle recommends either concurrent mark/sweep or incremental. They explicitly say NOT to use parallel new, yet that is what Minecraft is using.
#4: If you want to keep a 1 GB footprint for minecraft, at 400 MB of LDS, you are looking at (from that slide, not yet tested)
-Xms1g -Xmx1g -XX:PermSize65m -XX:MaxPermSize65m -Xmn400m
===
Tuning:
Looking at slide 30-32, tuning the young gen: -Xmn seems to defines the young gen size. Old gen is "left over". (Page 32, "old gen should be smaller than ..." seems to be a typo -- larger seems to fit better. To speed up responsiveness, the young gen should be smaller -- so dropping -Xmn down to 100m-200m may be a performance improvement here.
Page 38:
Choice of GC
General rule of thumb
– Latency more critical than throughput → CMS
< 1GB heap → ParallelOldGC might be able to meet desired latencies
Either CMS/incremental, or parallel Old with reduced young gen size. But definitely NOT the default settings.
===
Migration to CMS (page 52):
Normally, no compaction in the old gen – Potential for fragmentation
– Compaction only done by full GC
Increase old gen size by at least 20% to 30% – Accommodate fragmentation
So if we want to increase the old gen, and keep the young gen small, and keep a 1 GB footprint, these numbers (client) look like -Xms1g -Xmx1g -XX:PermSize65m -XX:MaxPermSize65m -Xmn100m
That's 10% for young, and about 800mb for the old.
Then, add either -Xincgc or -XX:+UseConcMarkSweepGC
Additional testing to come.
===
What I want to see: If you made it this far, understand what I have said, and are able to adjust your flags, please try this. See what kind of numbers give what kind of performance. Test with the memory you allocate on your server.
Now, an important question: How many garbage collection threads? Especially with the concurrent collector.
On my i7 4+4, the cpu will internally overclock if fewer cores are in use. One for the server, one for the client, and then, if the OS decides that there is not enough load, it will not use cores 5-8, and the client and server will run faster.
That implies dropping the number of GC threads down from 8 (physical limit) to 2.
-XX:ParallelGCThreads=8 change to
-XX:ParallelGCThreads=2
Finally: These numbers are for one client. I have no idea how this scales to larger servers, except to think that as the number of clients go up, and the amount of memory used goes up, the parallel new collector is even more a mismatch.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
I'm willing to do this, but first I would like a concise explanation of what each of those flags does, why you chose the numbers you did for the numeric ones, how they're supposed to help, and potential drawbacks. I'm already running -Xincgc and my launcher is set to a 4GB heap size, but the rest of this is getting into pretty deep technical territory.
Rollback Post to RevisionRollBack
"Sometimes, I just wanna give up, say 'I'm done with this mess' and go to bed. But you know what; you can't shrug off your responsibilities. You got to pull yourself up and meet the challenges head on. That's the only way you're gonna get ahead in life."
You don't need any GC or allocation changes for a client.
You do if you use a 256x256 texture pack and Single Player Commands. WorldEdit is like the RAM equivalent of McDonald's food.
Rollback Post to RevisionRollBack
"Sometimes, I just wanna give up, say 'I'm done with this mess' and go to bed. But you know what; you can't shrug off your responsibilities. You got to pull yourself up and meet the challenges head on. That's the only way you're gonna get ahead in life."
Well, the issue is that the default settings are not optimal for a program like minecraft.
And, I gave a nice technical PDF slideshow from Oracle about these.
It turns out that some of the flags are not present in the current version, and frankly, I cannot decipher the output from the incremental GC.
Fundamentally, the real questions are:
1. How much memory do you need to give to each of the memory buckets for Minecraft to run well
2. How much total memory do you need to give minecraft
given that minecraft won't be the only program running on your machine, and has to play well with other programs.
Equally, if having larger memory allocated to the young generation results in longer garbage collection, then at some point you are going to hurt responsiveness.
If the incremental GC reduces the "halting" interruption -- and it does -- then how to you adjust the buckets to prevent/reduce full collections because the incremental collector could not finish on time, while preventing it from running constantly because it's never fully satisfied.
And, just how much additional stuff can you add into minecraft and be happy with the memory you have for it. For example, since changing the render distance seriously affects memory consumption of the client, how far can you see for a given memory allotment? If you have a tradeoff between render distance and better texture pack, what is it? If you are running a server, how many people can you handle, and can you have both internet server and your own client happy on the same machine? How much is the effect of multiple people if they are in different areas -- increasing the server-side memory demand without changing the client's demand on your system.
Etc.
The big point is: Lacking any measured numbers, we have people saying "Toss more", "Don't toss too much", some people pointing out that if you have nothing else running, then tossing more memory up to the physical limit does help, but not looking at the whole "big collection" -- as I mentioned, if there's too much accumulated garbage, you get garbage collections longer than 30 seconds, and the default collection strategy does promote junk into the long-term storage. Letting that get too big is a problem.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
I usually respond to posts like the OP, but I've come to realize that responding in detail is essentially stooping to their level of idiocy at which point they beat me with their extensive experience.
EDIT: Though it's important to note, I said <like> the OP. After reading it it seems a lot more informed than most that I see. Kudos and good luck on your research to find the proper value
(and similar for the client)
(Yes, that's different from what I had the last time I tried using the CMS collector. Turns out I was mixing two different garbage collectors, which is why I couldn't understand the results.)
That mean, broken down:
1. Use the newer incremental GC (yes, there are two, but the second has not been updated since 1.4.2)
2. Start with 300MB, go up to 2G.
3. Start with 200MB for "eden + 2 survivors"
4. Go up to 600 MB for that.
5. Start with 300MB for tenured objects
6. Logging.
7. 45 MB allocated for classes and JVM overhead initially.
Output options (what the JVM actually is using from that)
-XX:InitialHeapSize=314572800 -XX:MaxHeapSize=2147483648 -XX:MaxNewSize=629145600 -XX:MaxTenuringThreshold=6 -XX:NewSize=209715200 -XX:OldPLABSize=16 -XX:OldSize=314572800 -XX:PermSize=47185920 -XX:+PrintCommandLineFlags -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+TraceClassUnloading -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
The biggest surprise: Even with what looks like very large buckets, desired "survivor" space is only 10mb, and a nether portal trip overwhelms the system. At a minimum of 20 MB, and the worst observed behavior was three full GC's on the client.
Client at that time:
{Heap before GC invocations=133 (full 2):
par new generation total 184320K, used 164317K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 100% used [0000000006010000, 0000000010010000, 0000000010010000)
from space 20480K, 2% used [0000000011410000, 00000000114877c0, 0000000012810000)
to space 20480K, 0% used [0000000010010000, 0000000010010000, 0000000011410000)
concurrent mark-sweep generation total 116336K, used 96518K [000000002b810000, 00000000329ac000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19156K [0000000086010000, 0000000088d10000, 000000008a010000)
2544.828: [GC 2544.828: [ParNew
Desired survivor size 10485760 bytes, new threshold 1 (max 6)
- age 1: 20237680 bytes, 20237680 total
- age 2: 48368 bytes, 20286048 total
- age 3: 182920 bytes, 20468968 total
- age 4: 22184 bytes, 20491152 total
- age 5: 18208 bytes, 20509360 total
- age 6: 21184 bytes, 20530544 total
: 164317K->20480K(184320K), 0.0630760 secs] 260836K->131008K(300656K), 0.0631392 secs] [Times: user=0.06 sys=0.03, real=0.07 secs]
Heap after GC invocations=134 (full 2):
par new generation total 184320K, used 20480K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 0% used [0000000006010000, 0000000006010000, 0000000010010000)
from space 20480K, 100% used [0000000010010000, 0000000011410000, 0000000011410000)
to space 20480K, 0% used [0000000011410000, 0000000011410000, 0000000012810000)
concurrent mark-sweep generation total 116336K, used 110528K [000000002b810000, 00000000329ac000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19156K [0000000086010000, 0000000088d10000, 000000008a010000)
}
2544.892: [GC [1 CMS-initial-mark: 110528K(116336K)] 134292K(300656K), 0.2133963 secs] [Times: user=0.00 sys=0.00, real=0.21 secs]
2545.105: [CMS-concurrent-mark-start]
2545.406: [CMS-concurrent-mark: 0.300/0.301 secs] [Times: user=0.10 sys=0.02, real=0.30 secs]
2545.406: [CMS-concurrent-preclean-start]
2545.407: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
2545.407: [CMS-concurrent-abortable-preclean-start]
2546.182: [CMS-concurrent-abortable-preclean: 0.070/0.775 secs] [Times: user=0.28 sys=0.02, real=0.78 secs]
2546.182: [GC[YG occupancy: 107006 K (184320 K)]2546.183: [Rescan (parallel) , 0.0052912 secs]2546.188: [weak refs processing, 0.0001272 secs] [1 CMS-remark: 110528K(116336K)] 217534K(300656K), 0.0055215 secs] [Times: user=0.03 sys=0.00, real=0.00 secs]
2546.188: [CMS-concurrent-sweep-start]
2546.311: [CMS-concurrent-sweep: 0.123/0.123 secs] [Times: user=0.03 sys=0.00, real=0.13 secs]
2546.311: [CMS-concurrent-reset-start]
2546.569: [CMS-concurrent-reset: 0.258/0.258 secs] [Times: user=0.02 sys=0.02, real=0.25 secs]
{Heap before GC invocations=134 (full 3):
par new generation total 184320K, used 184320K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 100% used [0000000006010000, 0000000010010000, 0000000010010000)
from space 20480K, 100% used [0000000010010000, 0000000011410000, 0000000011410000)
to space 20480K, 0% used [0000000011410000, 0000000011410000, 0000000012810000)
concurrent mark-sweep generation total 140220K, used 84130K [000000002b810000, 00000000340ff000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19165K [0000000086010000, 0000000088d10000, 000000008a010000)
2547.680: [GC 2547.680: [ParNew
Desired survivor size 10485760 bytes, new threshold 1 (max 6)
- age 1: 20957512 bytes, 20957512 total
: 184320K->20480K(184320K), 0.6270064 secs] 268450K->154833K(324540K), 0.6270783 secs] [Times: user=0.11 sys=0.07, real=0.63 secs]
Heap after GC invocations=135 (full 3):
par new generation total 184320K, used 20480K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 0% used [0000000006010000, 0000000006010000, 0000000010010000)
from space 20480K, 100% used [0000000011410000, 0000000012810000, 0000000012810000)
to space 20480K, 0% used [0000000010010000, 0000000010010000, 0000000011410000)
concurrent mark-sweep generation total 140220K, used 134353K [000000002b810000, 00000000340ff000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19165K [0000000086010000, 0000000088d10000, 000000008a010000)
}
2548.308: [GC [1 CMS-initial-mark: 134353K(140220K)] 158173K(324540K), 0.0097999 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
2548.318: [CMS-concurrent-mark-start]
2548.361: [CMS-concurrent-mark: 0.042/0.043 secs] [Times: user=0.15 sys=0.00, real=0.05 secs]
2548.361: [CMS-concurrent-preclean-start]
2548.363: [CMS-concurrent-preclean: 0.001/0.002 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
2548.363: [CMS-concurrent-abortable-preclean-start]
2551.321: [CMS-concurrent-abortable-preclean: 0.733/2.958 secs] [Times: user=1.95 sys=0.03, real=2.96 secs]
2551.321: [GC[YG occupancy: 105728 K (184320 K)]2551.321: [Rescan (parallel) , 0.0094801 secs]2551.331: [weak refs processing, 0.0227833 secs] [1 CMS-remark: 134353K(140220K)] 240081K(324540K), 0.0324341 secs] [Times: user=0.05 sys=0.00, real=0.03 secs]
2551.354: [CMS-concurrent-sweep-start]
2551.374: [CMS-concurrent-sweep: 0.020/0.020 secs] [Times: user=0.04 sys=0.00, real=0.02 secs]
2551.374: [CMS-concurrent-reset-start]
2551.381: [CMS-concurrent-reset: 0.007/0.007 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
2551.687: [GC [1 CMS-initial-mark: 129270K(215452K)] 242523K(399772K), 0.0474287 secs]
[Times: user=0.05 sys=0.00, real=0.05 secs]
2551.735: [CMS-concurrent-mark-start]
2551.776: [CMS-concurrent-mark: 0.040/0.041 secs] [Times: user=0.11 sys=0.00, real=0.04 secs]
2551.776: [CMS-concurrent-preclean-start]
2551.777: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
2551.777: [CMS-concurrent-abortable-preclean-start]
{Heap before GC invocations=135 (full 5):
par new generation total 184320K, used 184320K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 100% used [0000000006010000, 0000000010010000, 0000000010010000)
from space 20480K, 100% used [0000000011410000, 0000000012810000, 0000000012810000)
to space 20480K, 0% used [0000000010010000, 0000000010010000, 0000000011410000)
concurrent mark-sweep generation total 215452K, used 129270K [000000002b810000, 0000000038a77000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19169K [0000000086010000, 0000000088d10000, 000000008a010000)
2555.983: [GC 2555.983: [ParNew2556.040: [CMS-concurrent-abortable-preclean: 1.070/4.263 secs] [Times: user=2.17 sys=0.05, real=4.26 secs]
Desired survivor size 10485760 bytes, new threshold 6 (max 6)
- age 1: 6221624 bytes, 6221624 total
: 184320K->20480K(184320K), 0.2185113 secs] 313590K->168054K(399772K), 0.2185718 secs] [Times: user=0.06 sys=0.03, real=0.22 secs]
Heap after GC invocations=136 (full 5):
par new generation total 184320K, used 20480K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 0% used [0000000006010000, 0000000006010000, 0000000010010000)
from space 20480K, 100% used [0000000010010000, 0000000011410000, 0000000011410000)
to space 20480K, 0% used [0000000011410000, 0000000011410000, 0000000012810000)
concurrent mark-sweep generation total 215452K, used 147574K [000000002b810000, 0000000038a77000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19169K [0000000086010000, 0000000088d10000, 000000008a010000)
}
2556.203: [GC[YG occupancy: 23625 K (184320 K)]2556.203: [Rescan (parallel) , 0.0041555 secs]2556.208: [weak refs processing, 0.0000141 secs] [1 CMS-remark: 147574K(215452K)] 171200K(399772K), 0.0042775 secs] [Times: user=0.02 sys=0.00, real=0.00 secs]
2556.208: [CMS-concurrent-sweep-start]
2556.223: [CMS-concurrent-sweep: 0.015/0.015 secs] [Times: user=0.04 sys=0.00, real=0.02 secs]
2556.223: [CMS-concurrent-reset-start]
2556.230: [CMS-concurrent-reset: 0.007/0.007 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
{Heap before GC invocations=136 (full 5):
par new generation total 184320K, used 184320K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 100% used [0000000006010000, 0000000010010000, 0000000010010000)
from space 20480K, 100% used [0000000010010000, 0000000011410000, 0000000011410000)
to space 20480K, 0% used [0000000011410000, 0000000011410000, 0000000012810000)
concurrent mark-sweep generation total 245656K, used 147391K [000000002b810000, 000000003a7f6000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19169K [0000000086010000, 0000000088d10000, 000000008a010000)
2596.718: [GC 2596.718: [ParNew
Desired survivor size 10485760 bytes, new threshold 6 (max 6)
- age 1: 483816 bytes, 483816 total
- age 2: 5772808 bytes, 6256624 total
: 184320K->8020K(184320K), 0.0265022 secs] 331711K->155412K(429976K), 0.0265498 secs] [Times: user=0.01 sys=0.01, real=0.03 secs]
Heap after GC invocations=137 (full 5):
par new generation total 184320K, used 8020K [0000000006010000, 0000000012810000, 000000002b810000)
eden space 163840K, 0% used [0000000006010000, 0000000006010000, 0000000010010000)
from space 20480K, 39% used [0000000011410000, 0000000011be5090, 0000000012810000)
to space 20480K, 0% used [0000000010010000, 0000000010010000, 0000000011410000)
concurrent mark-sweep generation total 245656K, used 147391K [000000002b810000, 000000003a7f6000, 0000000086010000)
concurrent-mark-sweep perm gen total 46080K, used 19169K [0000000086010000, 0000000088d10000, 000000008a010000)
}
Server at that time:
{Heap before GC invocations=138 (full 5):
par new generation total 184320K, used 130316K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 79% used [77ae00000, 782c90ec8, 784e00000)
from space 20480K, 3% used [784e00000, 784eb2310, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 177748K, used 112515K [7a0600000, 7ab395000, 7fae00000)
concurrent-mark-sweep perm gen total 68108K, used 41120K [7fae00000, 7ff083000, 800000000)
2542.493: [Full GC (System) 2542.493: [CMS: 112515K->105934K(177748K), 1.9739913 secs] 242831K->105934K(362068K), [CMS Perm : 41120K->41038K(68108K)], 1.9740847 secs] [Times: user=0.25 sys=0.06, real=1.97 secs]
Heap after GC invocations=139 (full 6):
par new generation total 184320K, used 0K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 0% used [784e00000, 784e00000, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 177748K, used 105934K [7a0600000, 7ab395000, 7fae00000)
concurrent-mark-sweep perm gen total 68400K, used 41038K [7fae00000, 7ff0cc000, 800000000)
}
{Heap before GC invocations=139 (full 6):
par new generation total 184320K, used 163474K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 99% used [77ae00000, 784da4958, 784e00000)
from space 20480K, 0% used [784e00000, 784e00000, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 177748K, used 105934K [7a0600000, 7ab395000, 7fae00000)
concurrent-mark-sweep perm gen total 68400K, used 41055K [7fae00000, 7ff0cc000, 800000000)
2548.482: [GC 2548.482: [ParNew
Desired survivor size 10485760 bytes, new threshold 1 (max 6)
- age 1: 20575688 bytes, 20575688 total
: 163474K->20480K(184320K), 0.0457367 secs] 269409K->134103K(362068K), 0.0457886 secs]
[Times: user=0.05 sys=0.01, real=0.04 secs]
Heap after GC invocations=140 (full 6):
par new generation total 184320K, used 20480K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 100% used [786200000, 787600000, 787600000)
to space 20480K, 0% used [784e00000, 784e00000, 786200000)
concurrent mark-sweep generation total 177748K, used 113623K [7a0600000, 7ab395000, 7fae00000)
concurrent-mark-sweep perm gen total 68400K, used 41055K [7fae00000, 7ff0cc000, 800000000)
}
{Heap before GC invocations=140 (full 6):
par new generation total 184320K, used 184320K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 100% used [77ae00000, 784e00000, 784e00000)
from space 20480K, 100% used [786200000, 787600000, 787600000)
to space 20480K, 0% used [784e00000, 784e00000, 786200000)
concurrent mark-sweep generation total 177748K, used 113623K [7a0600000, 7ab395000, 7fae00000)
concurrent-mark-sweep perm gen total 68400K, used 41055K [7fae00000, 7ff0cc000, 800000000)
2551.429: [GC 2551.429: [ParNew
Desired survivor size 10485760 bytes, new threshold 1 (max 6)
- age 1: 20947232 bytes, 20947232 total
: 184320K->20480K(184320K), 0.0833066 secs] 297943K->162252K(362068K), 0.0833638 secs] [Times: user=0.06 sys=0.03, real=0.08 secs]
Heap after GC invocations=141 (full 6):
par new generation total 184320K, used 20480K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 100% used [784e00000, 786200000, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 177748K, used 141772K [7a0600000, 7ab395000, 7fae00000)
concurrent-mark-sweep perm gen total 68400K, used 41055K [7fae00000, 7ff0cc000, 800000000)
}
2551.512: [GC [1 CMS-initial-mark: 141772K(177748K)] 163213K(362068K), 0.0025717 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
2551.515: [CMS-concurrent-mark-start]
2551.565: [CMS-concurrent-mark: 0.049/0.050 secs] [Times: user=0.17 sys=0.04, real=0.05 secs]
2551.565: [CMS-concurrent-preclean-start]
2551.566: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
2551.566: [CMS-concurrent-abortable-preclean-start]
2552.353: [CMS-concurrent-abortable-preclean: 0.081/0.786 secs] [Times: user=1.12 sys=0.60, real=0.79 secs]
2552.354: [GC[YG occupancy: 117160 K (184320 K)]2552.354: [Rescan (parallel) , 0.0251490 secs]2552.379: [weak refs processing, 0.0000184 secs] [1 CMS-remark: 141772K(177748K)] 258933K(362068K), 0.0252459 secs] [Times: user=0.15 sys=0.01, real=0.03 secs]
2552.379: [CMS-concurrent-sweep-start]
2552.396: [CMS-concurrent-sweep: 0.016/0.016 secs] [Times: user=0.04 sys=0.01, real=0.01 secs]
2552.396: [CMS-concurrent-reset-start]
2552.399: [CMS-concurrent-reset: 0.003/0.003 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
{Heap before GC invocations=141 (full 7):
par new generation total 184320K, used 184320K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 100% used [77ae00000, 784e00000, 784e00000)
from space 20480K, 100% used [784e00000, 786200000, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 234640K, used 140784K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41055K [7fae00000, 7ff0fc000, 800000000)
2552.873: [GC 2552.873: [ParNew
Desired survivor size 10485760 bytes, new threshold 1 (max 6)
- age 1: 12116256 bytes, 12116256 total
: 184320K->18961K(184320K), 0.8652316 secs] 325104K->180117K(418960K), 0.8652876 secs] [Times: user=0.04 sys=0.04, real=0.87 secs]
Heap after GC invocations=142 (full 7):
par new generation total 184320K, used 18961K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 92% used [786200000, 7874847f0, 787600000)
to space 20480K, 0% used [784e00000, 784e00000, 786200000)
concurrent mark-sweep generation total 234640K, used 161155K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41055K [7fae00000, 7ff0fc000, 800000000)
}
{Heap before GC invocations=142 (full 7):
par new generation total 184320K, used 182749K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 99% used [77ae00000, 784df2d58, 784e00000)
from space 20480K, 92% used [786200000, 7874847f0, 787600000)
to space 20480K, 0% used [784e00000, 784e00000, 786200000)
concurrent mark-sweep generation total 234640K, used 161155K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41056K [7fae00000, 7ff0fc000, 800000000)
2555.566: [GC 2555.566: [ParNew
Desired survivor size 10485760 bytes, new threshold 1 (max 6)
- age 1: 12919488 bytes, 12919488 total
: 182749K->18719K(184320K), 0.1499257 secs] 343904K->191168K(418960K), 0.1499711 secs] [Times: user=0.04 sys=0.02, real=0.15 secs]
Heap after GC invocations=143 (full 7):
par new generation total 184320K, used 18719K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 91% used [784e00000, 786047ef0, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 234640K, used 172448K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41056K [7fae00000, 7ff0fc000, 800000000)
}
{Heap before GC invocations=143 (full 7):
par new generation total 184320K, used 182559K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 100% used [77ae00000, 784e00000, 784e00000)
from space 20480K, 91% used [784e00000, 786047ef0, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 234640K, used 172448K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41057K [7fae00000, 7ff0fc000, 800000000)
2560.395: [GC 2560.395: [ParNew
Desired survivor size 10485760 bytes, new threshold 6 (max 6)
- age 1: 1991688 bytes, 1991688 total
: 182559K->12861K(184320K), 0.1364792 secs] 355008K->196631K(418960K), 0.1365323 secs] [Times: user=0.02 sys=0.02, real=0.14 secs]
Heap after GC invocations=144 (full 7):
par new generation total 184320K, used 12861K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 62% used [786200000, 786e8f530, 787600000)
to space 20480K, 0% used [784e00000, 784e00000, 786200000)
concurrent mark-sweep generation total 234640K, used 183770K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41057K [7fae00000, 7ff0fc000, 800000000)
}
{Heap before GC invocations=144 (full 7):
par new generation total 184320K, used 176701K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 100% used [77ae00000, 784e00000, 784e00000)
from space 20480K, 62% used [786200000, 786e8f530, 787600000)
to space 20480K, 0% used [784e00000, 784e00000, 786200000)
concurrent mark-sweep generation total 234640K, used 183770K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41057K [7fae00000, 7ff0fc000, 800000000)
2564.074: [GC 2564.074: [ParNew
Desired survivor size 10485760 bytes, new threshold 6 (max 6)
- age 1: 596448 bytes, 596448 total
- age 2: 1618424 bytes, 2214872 total
: 176701K->4027K(184320K), 0.0058398 secs] 360471K->187797K(418960K), 0.0058944 secs] [Times: user=0.01 sys=0.01, real=0.01 secs]
Heap after GC invocations=145 (full 7):
par new generation total 184320K, used 4027K [77ae00000, 787600000, 7a0600000)
eden space 163840K, 0% used [77ae00000, 77ae00000, 784e00000)
from space 20480K, 19% used [784e00000, 7851eed78, 786200000)
to space 20480K, 0% used [786200000, 786200000, 787600000)
concurrent mark-sweep generation total 234640K, used 183770K [7a0600000, 7aeb24000, 7fae00000)
concurrent-mark-sweep perm gen total 68592K, used 41057K [7fae00000, 7ff0fc000, 800000000)
}
Things to note:
1. The survivor buckets are 20 MiB, and are full AFTER the GC. This is "normal" but undesirable with the CMS collector -- the goal of the system is to start a CMS early enough that it will finish before the survivor buckets fill up.
For a nether portal, the system is going to load a bunch of new mini chunks.
It will probably (with far vision, or range 15 server distance) load 31x31 chunks of 16x16x16 minichunks x8 height x2 bytes per block -- that's about 62 MB.
Yea.
First conclusion: If your server is bottlenecked, turn distance down from 15 (far) to 7 *9* (normal).
EDIT: "9" should be "normal". "8" is the actual display distance, and to tick the blocks at 8 chunks distance, they need to be able to affect blocks slightly past them.
2. Moving around normally loads one strip of chunks on a regular basis. That's 1x31 x16x16x16 x5 (average height of 80) x2 or about 1.2MiB (40KiB*31, or 1240MiB). That fits will enough in the 10 MiB desired survior space without triggering unwanted promotion. But that does imply that the desired survivor space needs to be around 6 to 12 MiB per player if you are doing any moving -- which means survivor buckets of 12 to 24 MiB.
Note that you cannot (as far as I can tell) specify the size of the survivor buckets -- you have to specify the new size and survivor ratios. Yucky.
If you are moving around a construction/base area, you can be moving more than 16 blocks in each direction trivially -- we move more than 32 blocks around the inside of our base, and it was not built with chunk aligned insides. That means in our base, I'm running around a 4x4 chunk area.
Which in turn means that the outer 4 strips in all 4 directions are being reloaded and flushed.
That's about 16x27 (31 length would give overlap) x16 x16x16 x6 (we built in extreme hills) x2 of data being flushed as I move around.
That's over 20MiB of stuff being thrown away regularly.
But it doesn't trigger constant full GC's -- the GC system seems to be able to handle movement just fine.
So the survivor buckets don't need to be that large. Other than going through portals, 20 MiB buckets were working just fine, and are probably larger than needed.
3. Java's "I'll reduce the size of my buckets and return unused memory to the OS" is worthless. Turns out that Java wants 70% free space in any bucket before it shrinks that bucket.
That will only happen at shutdown.
Current recommendations:
1. Use the incremental collector. While the throughput collector does have a flag to say "Keep collections shorter than Nms", that is a "Opps, we took too long, adjust our thresholds a little and try again" behavior.
2. Allocate memory based on the assumption of 16 MiB survivor buckets per player. (Technically: group of players in the same location -- this is dominated by movement loading new chunks, and if the players are in the same location they will load like a single player)
If you have the standard ratio of 6:1 for eden versus survivor, that's 8*16 = 128MiB of "new" space per player you want to support.
So for a 12 person PvP battle, or for 12 parties/factions with small bases, you want -XX:NewSize=1536m.
Equally, if you are going to be playing underground in caves, or have players that cannot use "far" outdoors and want a balanced fight, lower the server view distance.
3. The client generally needed a little over 200MiB of long-term space:
Those were from portal hopping. That was peak memory usage, and it isn't clear that the 350MB usage was really all used or just some strange temporary (remember, the CMS collector can miss temporary garbage and collect it later).
But for around 200m of long-term memory, 350m of "old" working space, 128m of "new", and 42m of "perm", that's a client memory usage of about 500m to have enough memory to keep garbage collection at a minimum. Less than 500m will have excessive GC; more than 500m will reduce memory available to other programs (such as Firefox -- the bigger memory hog :-). And, if you are using a lot of portals, you'll want to add more.
So that's 512m for people who don't use portals a lot, or 640m for people who do use portals a lot.
That -- along with incremental GC -- is the best recommendation for people who want minecraft to multitask with other programs while playing on a server (as opposed to locally).
Note also: That's based on using the 64 bit java for "far" support. If you are not using far, you need a lot less (32 bit will take less memory AND you have less to deal with).
4. The server:
Well, it turns out that "-d32 -server" gives slightly different output.
(yes, I left the server running overnight after logging off.)
The server seems to have an 80m overhead when no players are logged on!
About the only thing I can conclude from this is a minimum of 180m per player for old storage, plus 128m for "new" (again, that's per party -- dominated by movement), plus another 90-180m for temporary/change in the long-term, or about 400m+ per group plus a smaller amount per player, plus another 80m for overhead on your server.
Without testing the integrated client, the "predictable" is that they can share "new" space nicely, but the "world" will be loaded twice -- once on the client side, once on the server side -- and you're looking at 640m (client) plus 250m for server data, or 900m for minimal garbage collection while keeping memory usage under control.
Small machines (2g or less, definitely for 1g systems) really want to consider disabling far vision.
This is now at the limit of my knowledge of tweaking java's GC. Anyone want to push farther?
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
So it turns out that "-XX:+UseParNewGC" is not a new version of "-XX:UseParallelGC".
ParNewGC does parallel collection of the young generation, and is designed to work with the CMS collection of the old/tenured generation.
Additionally: -XX:+CMSParallelRemarkEnabled -- speeds up concurrent remarking, and -XX:CMSFullGCsBeforeCompaction=1 -- makes sure that if CMS gets part/most of the way through when memory fills up, that work is not wasted, but finished.
Still to determine: How to reduce full collections (generally: larger memory), and how to reduce memory (generally: More collections) together :-).
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Please keep in mind that decreasing the view-range setting on a server will affect mob spawn rates. Try it out by setting it to 7 (instead of the default 10), and watch mob spawns pretty much go up in smoke. They will still spawn, but much, much reduced.
Rollback Post to RevisionRollBack
Looking for a friendly community? Look no further, and join the BlockStackers! We run a Vanilla whitelisted server, a Feed The Beast server, and a Survival server with some twists.
Please keep in mind that decreasing the view-range setting on a server will affect mob spawn rates. Try it out by setting it to 7 (instead of the default 10), and watch mob spawns pretty much go up in smoke. They will still spawn, but much, much reduced.
That, I was not aware of.
As I understand it, "9" means 9 chunks radius from your chunk. "Tiny" view is radus 2 chunks, "short" is radius 4 chunks, "normal" is radius 8 chunks, and normal vanilla behavior is to load stuff at radius 9 chunks and tick (and display) blocks within radius 8 chunks; as blocks can affect blocks slightly more distant when they tick, that next group of chunks has to be loaded. But this means that "9" is equivalent to "normal" vanilla single player range.
Going down to less than 9, I thought only affected what was sent to the client, not what the server actually worked with. If that's the case, then drop it down to 9, and not lower.
Am I the only one who thinks this is a server bug?
Collecting "new" memory is fast. No matter how big you make "new", the speed of collection depends on the number of "live" objects in it that are being copied.
Collecting "tenured" objects is slow. The bigger you make "tenured", the _SLOWER_ it will be.
Fast behavior: Make tenured as big as you need -- 150% of actual usage is fine -- but no bigger. Make "new" as big as you can afford the memory for.
Once you make "new" big, the next tuning is "how many times do I copy a new object, and give it a chance to die, before I commit it to tenured space and ignore it for a long time".
Current testing, which seems to work well for 1-2 players that are mostly together: 100m of new, survivor ratio of 6 (gives "survivor" size of 13m), max copying of 4. Then the "-Xmx" can be 500m and the server is happy. VERY happy.
Next test is "stress" -- twilight forest and mystcraft (it already passes the nether test).
Client side is still being tweaked/tested before I make any recommendations.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Me neither, but I found out the hard way when I dropped it to 7 from 10, suddenly mobs just stopped spawning. Well, they still spawn, just, they turn into very rare little creatures. I found out it had to do with the view range after doing a reset on the configuration and noticing all of a sudden things were back to normal. Seeing as the only thing I changed was the view range and the difficulty level, tinkering around lead to the strange finding that view-range is linked to mob spawns, somehow.
As I understand it, "9" means 9 chunks radius from your chunk. "Tiny" view is radus 2 chunks, "short" is radius 4 chunks, "normal" is radius 8 chunks, and normal vanilla behavior is to load stuff at radius 9 chunks and tick (and display) blocks within radius 8 chunks; as blocks can affect blocks slightly more distant when they tick, that next group of chunks has to be loaded. But this means that "9" is equivalent to "normal" vanilla single player range.
Pretty much yes, actually the default is 10 chunks, according to the wiki (I think), the irony being that while it's supposedly sending less to the client, you still end up with the same number of chunks in the chunk cache (hit F3 and see the multiplayerchunkcache value), so either it sends them slower, or it ignores this setting for some reason.
Going down to less than 9, I thought only affected what was sent to the client, not what the server actually worked with. If that's the case, then drop it down to 9, and not lower.
Am I the only one who thinks this is a server bug?
In theory yes, practically it seems that mob spawning and mob spawn rates are tied into what's sent to any given client for some odd reason. Probably has something to do with a mob being an entity in a chunk, instead of not being associated in that way at all. Load less chunks, see less entities, get lower spawn rates because the spawn 'find a spot to spawn stuff' selection algorithm goes funny.
I'm not sure if it's a bug, I'd like to think it is...
Rollback Post to RevisionRollBack
Looking for a friendly community? Look no further, and join the BlockStackers! We run a Vanilla whitelisted server, a Feed The Beast server, and a Survival server with some twists.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
But it looks to be a case of being able to say "Every 1/20th of a second, spend up to N milliseconds collecting as much garbage as you can". With compaction (which CMS cannot provide).
I lack the knowledge of how to adjust G1's tuning knobs; I lack the knowledge of what those knobs even are. Just that they exist. And, apparently, it isn't even clear that J6 has a complete/good implementation; that may require J7.
"If G1 works out as we expect, it will become our low-pause collector in place of "ParNew" + "CMS". And if you're about to ask when will it be ready, please don't be offended by my dead silence.".
"With G1 you can specify a time slice NNN and the maximum amount of time to be spent on GC in that time slice MMM. G1 will then schedule it's work such that for any NNN time slice no more than MMM will be spent doing GC. You can clearing over specify this goal (MMM = 0 will never work) but G1 tries to provide the application with at least NNN - MMM per time slice."
Now, granted that's not quite what we want -- we want "Every 1/20th of a second, the main loop begins; take as much time as you want once the main loop has finished; that time available to you will change each time through the loop". But it's a big step -- it's the first indication that this is a real-time garbage collector (which is confirmed in other comments on that page).
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
A status update. Just when I thought I had this solved, I tested just sitting around in twilight forest, with DynamicMap running.
Desired survivor size 6553600 bytes, new threshold 1 (max 4)
- age 1: 11097320 bytes, 11097320 total
It's not even close to running out of memory -- but it's constantly filling up with temporaries, triggering almost constant background CMS garbage collections.
Both TF and Dynmap want lots of temp memory, and I think Dynmap is worse -- I don't recall having any issues with just TF. I suspect TF is representative of the "nasty" mods -- mods with lots of creatures and blocks.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
So it looks like Dynamic Map will be one of the worst case add-ins. Normal behavior does not generate map tiles so frequently as to cause problems, however, a full render will be massive.
This add-in wants 200 MB survivor spaces (70-90 MB of temporary stuff in the first surviving generation is common). Tossing 600 MB at eden (to keep new generation GC's down), results in mostly behaved memory consumption (Yes, that's 1 GB for new memory space). At full speed rendering, it still overwhelms the tenured GC's ability to keep memory free, and goes up to 600 MB of tenured space used (in fairness, that was the limit in my test -- I don't know where it would actually max).
Giving it a throttle lowers the tenured space used to a maximum (allocated) of just over 300 MB, and a 200 MB actual usage. Since the recommendation for tenured size is 150% or more of actual, that's not a bad match, and that's only during a full render with a complicated set of maps.
At the moment, it looks like 200 MB for tenured is more than sufficient for a small (3 player) server (add 100 MB for dynamic maps). New/Eden/Survivor, on the other hand, will vary significantly based on mods.
Draft zero of a tuning walk-through should be ready in about 2-3 weeks.
In the meantime, here is my current startup command:
Yes, that's calling for the 32 bit server -- Oracle actually says that the 32 bit version works fine for heaps up to about 2.5 GB, and will consume less memory than the 64 bit. "MaxHeapFreeRatio" is supposed to return unused memory to the operating system, but it has no effect on Macintosh.
Other than that, the startup is calling for 12.5 MB survivor spaces -- which is overkill/waste more than half the time. The goal here is to give "eden" enough room that new collections happen infrequently enough for objects to go "dead" before survivor space fills up or 4 collections occur. And, for a 1-3 person server, that works. Single player should be similar.
Again, this is with Twilight Forest -- which spams giant trees, massive leaf canopies, and large amounts of peaceful animals. High block count, high passive mob count. Vanilla will probably be less except in a giant extreme hills environment.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Garbage collector definitions. Should be familiar.
If you have 8 cores, *you do not want incremental*.
Change that second line to
-XX:ParallelGCThreads=4 \
and then experiment with raising it up from 4 to 8.
Partner:
So I removed this and it had a huge negative impact on the server. With this line removed, entities warp around and the tick ms is at an average of 40ms at all times, with the line added, there is no warping of entities and the server ms is an average of 8ms.
Me:
Very interesting. That's the opposite of what Oracle recommends.
Adding to my toolkit ...
===
Bottom line: Incremental garbage collection (not CMS, but breaking CMS into pieces), which Oracle only recommends for single or 2-core systems, actually was better on an 8-core system.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
So, what does that mean at a high level for a player like me? I'm well-versed in programming (java at that) but not nearly as technical as you. Would changing any of these improve performance at all? Note that I have 8GB allocated to Minecraft; I know I don't need that much but I play with tons of mods and HD textures, so it's good to have extra--and I have 32gb ram on my system anyways. I realize these findings probably only affect memory usage, which is obviously not a problem for me, but I'm just curious to see if they do have an impact on performance. I should also note that I'm using a 4 core processor.
So, what does that mean at a high level for a player like me? I'm well-versed in programming (java at that) but not nearly as technical as you. Would changing any of these improve performance at all? Note that I have 8GB allocated to Minecraft; I know I don't need that much but I play with tons of mods and HD textures, so it's good to have extra--and I have 32gb ram on my system anyways. I realize these findings probably only affect memory usage, which is obviously not a problem for me, but I'm just curious to see if they do have an impact on performance. I should also note that I'm using a 4 core processor.
This is all about performance.
Java, by default, assumes that you want a throughput collector, and you have relatively low long-term data storage. This is a great assumption for dedicated servers that serve web pages. This is a bad assumption for minecraft.
Minecraft wants fast responsiveness, and the default garbage collector cannot provide it.
Minecraft wants several hundred MB of long-term data storage (at least 200 MB, more with texture packs, more if you have a single player game with both server and client in one package, more if you have a server with lots of players, more if those players are not in the same place, etc.)
Minecraft is two single-threaded operations. While Java can do some bookkeeping work on the other cores, the majority of the code can only use two CPU's -- so if you have more cores, you want to let Java do more stuff on those cores at the same time. Hence the desire to use concurrent garbage collection if possible (so garbage collection doesn't slow the system down)
The default allocation of "new vs old" cannot fit all cases. The more memory you have available for minecraft, the more you want into new. Having the tenured space get bigger and bigger only causes the full collections to get less frequent and longer. Having more space for "new" results in less time spent in collections. Java wants to use a system based on percent of memory for this, percent of memory for that, and it cannot match what minecraft wants/needs for all memory sizes.
And, in my case, I want to have a voice chat server, voice chat client (and recorder), screen recorder, multiplayer server, and client, all running on the same box -- so I can't just say "Give it all the memory I have", I have to figure out how much it needs / how much to allocate where.
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
I don't have any clue on larger servers (more people) yet.
Give me a couple of days, and I should have some baseline numbers to use to start making recommendations. Currently have on my to-do list a server log from a server with over 100 mods, a decent player base, and a small memory leak.
As a simple case of "number from nothing", try a tenured size of 320 MB plus 30 MB per person, and then make "new" as big as you can without exhausting physical memory. Vanilla only needs about 45 MB for perm, but with lots of mods it can overfill 90 MB and run out of memory -- if you get errors there, try raising perm to 150 (should be overkill). (Perm is used for bytecode and compiled code).
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Brief summary: Memory is divided into approximately 2,000 cells (tunable); each cell may contain any type (survivor, eden, tenured, etc) as needed. This permits memory allocation to vary based on loads. Cells that are unused are very fast to collect; collector determines which cells are fastest to collect and collects those. Minimum size of a cell is 1 MB; recommended for heap sizes of 6 GB or larger.
The primary tuning control is the desired application pause time; the default is 1/5th second. First approximation for minecraft would be to target 1/40th of a second (half a tick).
I expect to play with this in detail, probably in february.
Waiting for feedback from someone with a decent-sized server with memory issues before making any recommendations; expect something no later than wednesday. Script for launching a server with easy access to memory tuning parameters is done; modifying it to launch a client is "in progress".
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
EDIT:
The short version:
1. The default garbage collector is a worst-case fit to minecraft.
2. Optifine, at least 147 D5, has a memory leak that will cause the client to never be happy.not confirmed; actually seems to be caused by something else.3. A launcher for servers can be found at http://pastebin.com/rY3gPVGE
-- this will also provide log files to quickly see how memory can be adjusted for your server's needs. This launcher uses Concurrent-Mark-Sweep. That link is for version 1.7.1, release 1.
4. Client scripts are harder, as most "launchers" (that handle your login/password) want to define their own memory management, and will not say "Use what was used to run me". ("minecraft.exe" on windows systems will, but window's "cmd" lacks the features to run that script). An offline launcher based on that server script is trivial; command flags to give to "smart launchers" are harder. Coming soon.
5. Switching to G1GC, in initial testing, consumes significantly more memory while providing some performance improvement. Further testing is on my ToDo list.
What's the oddball in this list:
1. Easter Bunny
2. Fast Java Programs
3. Santa Clause
4. Benevolent Government
** The default Java garbage collector (https://blogs.oracle..._monitoring_and, http://www.oracle.co...00744-en-in.pdf) is optimized for total throughput using all cores regardless of system load. It has no concern about responsiveness (latency). Not only did testing last night match every case of client lag with an increase in "small GC" activity, when a full GC was triggered it was often longer than 30 seconds (forcing a disconnect) and was 95 seconds in the worst case).
Background: Java -- both Sun/Oracle's, and Apple's, in Java 6, have at least three different garbage collectors, all of which have tunable options. The default garbage collector is "parallel new", which is optimized for total throughput of large heap spaces (> 1gb). In comparison, "parallel old" is regarded as a better choice for smaller heap spaces (but must be explicitly requested), "single-threaded old" is better for small heap spaces on single core CPU's, and either "concurrent mark sweep" or "incremental" is better on large heap, multi-core cpu's.
In other words, the default garbage collector is the worst match for minecraft except possibly on large memory single core systems.
===
(http://www.oracle.co...00744-en-in.pdf has steps to take to measure your system's actual memory usage, and then adjust the VM's parameters for your system.)
If you know how to add the flag -XX:+PrintCommandLineFlags to your minecraft (client or server), then please help me collect data here. Follow along, test out your system, and lets get some ideas on how Mojang can set default settings to improve minecraft performance. If you don't know how, then I will apologize in advance: I have higher priorities (aka "don't have the time") than getting your data. I could tell you that if you use Magic Launcher to use the "Advanced" tab, and enter it there. I could tell you that if you run a server, you are either currently using a script to start it, or if you are using the windows' EXE server then you can get the jar server and script from Mojang's web site. But the level of hand holding you will need after that is more time then I'm willing to spend on this.
===
So here is what I did.
(Mod list: Client and server: 1.3.2 versions:
Forge
Mystcraft (linking books/portals were used; new ages were not)
Extra Biomes (was not used in this test)
Twilight Forest (was not used in this test)
Superior Enchanting (was not used in this test)
Client only:
Optifine (B3)
MineVideo
Too Many Items
Saturation Bar
What's My Light Level (33, saturation bar compatibility)
Durability Warning
Server only:
Ruins
Clay Soil
FallFix
More village biomes and Village logic addon
Water Propagation Fix
Larga-isso-enderman ("drop that block, enderman")
Custom Ore Gen (was used; new chunks generated)
None of these makes major demands on the system (the ones that would were not used), so I think these are representative mods.
Step 1: On the server: -d32 -server -XX:+UseParallelOldGC -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log (32 bit is recommended by oracle for 2.5GB or smaller heaps.)
On the client: -XX:+UseParallelOldGC -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log (NB: This leaves it in 64 bit mode for far vision support)
Note that Oracle specifically states to start with parallel old GC for collecting data, and go from there. It also says to start by tossing more memory than you need and see what gets used.
Step 2:
In order to get useful information, you need to trigger a full garbage collect from out of memory. The client does multiple full garbage collects when you disconnect / reconnect to a server. I consistently saw a total of 3, and the first and third both had large amounts of memory refunded. (One on disconnect; two on connection).
The client seems to need a full GC every 300-1000 seconds. Larger render distance resulted in more frequent GC's (kinda obvious, but a good sanity test). With 8 threads on an i7 4+4 @2.2 GHz, the observed times were 12.4 sec (during play), 4 sec (manual disconnect/reconnect), 77 seconds (server DC'd me for being non responsive), 61 seconds (ditto), and 95 seconds (last one, after the server itself went unresponsive).
The server took about 3 hours to finally run out; it needed 61 seconds to garbage collect.
Step 3: Look at the numbers
The server:
So the total space after collecting is about 120MB for one client, and the old gen is about 100 MB. Perm is about 20 MB (mostly classes)
For the client, the numbers are:
Perm hits a max of about 45 MB, and old is around ... well, that's actually hard to say. Note that it kept going up, and was over 300 MB at the end. I'm going to guess that it will be happy around 400-450 MB.
According to page 25 of Oracle's slide show:
Now, we can draw some conclusions here:
#1: There seems to be a serious memory leak in the client.
The increase in "OldGen" from around 150-170 MB to 230 MB corresponds to the change from "Short + 32" render distance to "far" render distance. The final jump at the end is a mystery to me -- this is AFTER the server timed out and the client DC'd.
#2: Oracle recommends putting the max size at least 3 times the "long-lived data" size -- which goes well over 1 GB. Yes, minecraft may run in less, but for garbage collection performance, it needs more.
#3: As soon as you go over 1 GB, parallel old is no longer recommended. For low latency, oracle recommends either concurrent mark/sweep or incremental. They explicitly say NOT to use parallel new, yet that is what Minecraft is using.
#4: If you want to keep a 1 GB footprint for minecraft, at 400 MB of LDS, you are looking at (from that slide, not yet tested)
-Xms1g -Xmx1g -XX:PermSize65m -XX:MaxPermSize65m -Xmn400m
===
Tuning:
Looking at slide 30-32, tuning the young gen: -Xmn seems to defines the young gen size. Old gen is "left over". (Page 32, "old gen should be smaller than ..." seems to be a typo -- larger seems to fit better. To speed up responsiveness, the young gen should be smaller -- so dropping -Xmn down to 100m-200m may be a performance improvement here.
Page 38:
Either CMS/incremental, or parallel Old with reduced young gen size. But definitely NOT the default settings.
===
Migration to CMS (page 52):
So if we want to increase the old gen, and keep the young gen small, and keep a 1 GB footprint, these numbers (client) look like -Xms1g -Xmx1g -XX:PermSize65m -XX:MaxPermSize65m -Xmn100m
That's 10% for young, and about 800mb for the old.
Then, add either -Xincgc or -XX:+UseConcMarkSweepGC
Additional testing to come.
===
What I want to see: If you made it this far, understand what I have said, and are able to adjust your flags, please try this. See what kind of numbers give what kind of performance. Test with the memory you allocate on your server.
Now, an important question: How many garbage collection threads? Especially with the concurrent collector.
On my i7 4+4, the cpu will internally overclock if fewer cores are in use. One for the server, one for the client, and then, if the OS decides that there is not enough load, it will not use cores 5-8, and the client and server will run faster.
That implies dropping the number of GC threads down from 8 (physical limit) to 2.
-XX:ParallelGCThreads=8 change to
-XX:ParallelGCThreads=2
Finally: These numbers are for one client. I have no idea how this scales to larger servers, except to think that as the number of clients go up, and the amount of memory used goes up, the parallel new collector is even more a mismatch.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Instead of (wrong): -XX:PermSize65m -XX:MaxPermSize65m
Use -XX:PermSize=65m -XX:MaxPermSize=65m
Testing now ...
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
You do if you use a 256x256 texture pack and Single Player Commands. WorldEdit is like the RAM equivalent of McDonald's food.
And, I gave a nice technical PDF slideshow from Oracle about these.
It turns out that some of the flags are not present in the current version, and frankly, I cannot decipher the output from the incremental GC.
Fundamentally, the real questions are:
1. How much memory do you need to give to each of the memory buckets for Minecraft to run well
2. How much total memory do you need to give minecraft
given that minecraft won't be the only program running on your machine, and has to play well with other programs.
Equally, if having larger memory allocated to the young generation results in longer garbage collection, then at some point you are going to hurt responsiveness.
If the incremental GC reduces the "halting" interruption -- and it does -- then how to you adjust the buckets to prevent/reduce full collections because the incremental collector could not finish on time, while preventing it from running constantly because it's never fully satisfied.
And, just how much additional stuff can you add into minecraft and be happy with the memory you have for it. For example, since changing the render distance seriously affects memory consumption of the client, how far can you see for a given memory allotment? If you have a tradeoff between render distance and better texture pack, what is it? If you are running a server, how many people can you handle, and can you have both internet server and your own client happy on the same machine? How much is the effect of multiple people if they are in different areas -- increasing the server-side memory demand without changing the client's demand on your system.
Etc.
The big point is: Lacking any measured numbers, we have people saying "Toss more", "Don't toss too much", some people pointing out that if you have nothing else running, then tossing more memory up to the physical limit does help, but not looking at the whole "big collection" -- as I mentioned, if there's too much accumulated garbage, you get garbage collections longer than 30 seconds, and the default collection strategy does promote junk into the long-term storage. Letting that get too big is a problem.
So what numbers work? For what kind of usage?
Can we find a pattern? Some guide for users?
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
EDIT: Though it's important to note, I said <like> the OP. After reading it it seems a lot more informed than most that I see. Kudos and good luck on your research to find the proper value
So additional research.
http://www.oracle.co...g-5-138395.html
http://www.oracle.co...go5-140223.html
http://www.oracle.co...jsp-140102.html
http://engineering.l...on-web-services
http://www.petefreit...icles/gctuning/
And had an email conversation with the author of the original PDF slideshow, who confirmed a couple of typos.
Last night's testing:
Input options:
java \
-d32 -server \
-XX:+UseConcMarkSweepGC \
-Xms300m -Xmx2g \
-XX:NewSize=200m \
-XX:MaxNewSize=600m \
-XX:OldSize=300m \
-XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:MaxTenuringThreshold=6 \
-XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log \
-XX:PermSize=45m \
-jar server-myst-ebxl.jar nogui main
(and similar for the client)
(Yes, that's different from what I had the last time I tried using the CMS collector. Turns out I was mixing two different garbage collectors, which is why I couldn't understand the results.)
That mean, broken down:
1. Use the newer incremental GC (yes, there are two, but the second has not been updated since 1.4.2)
2. Start with 300MB, go up to 2G.
3. Start with 200MB for "eden + 2 survivors"
4. Go up to 600 MB for that.
5. Start with 300MB for tenured objects
6. Logging.
7. 45 MB allocated for classes and JVM overhead initially.
Output options (what the JVM actually is using from that)
-XX:InitialHeapSize=314572800 -XX:MaxHeapSize=2147483648 -XX:MaxNewSize=629145600 -XX:MaxTenuringThreshold=6 -XX:NewSize=209715200 -XX:OldPLABSize=16 -XX:OldSize=314572800 -XX:PermSize=47185920 -XX:+PrintCommandLineFlags -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+TraceClassUnloading -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
The biggest surprise: Even with what looks like very large buckets, desired "survivor" space is only 10mb, and a nether portal trip overwhelms the system. At a minimum of 20 MB, and the worst observed behavior was three full GC's on the client.
Client at that time:
Server at that time:
Things to note:
1. The survivor buckets are 20 MiB, and are full AFTER the GC. This is "normal" but undesirable with the CMS collector -- the goal of the system is to start a CMS early enough that it will finish before the survivor buckets fill up.
For a nether portal, the system is going to load a bunch of new mini chunks.
It will probably (with far vision, or range 15 server distance) load 31x31 chunks of 16x16x16 minichunks x8 height x2 bytes per block -- that's about 62 MB.
Yea.
First conclusion: If your server is bottlenecked, turn distance down from 15 (far) to
7*9* (normal).EDIT: "9" should be "normal". "8" is the actual display distance, and to tick the blocks at 8 chunks distance, they need to be able to affect blocks slightly past them.
2. Moving around normally loads one strip of chunks on a regular basis. That's 1x31 x16x16x16 x5 (average height of 80) x2 or about 1.2MiB (40KiB*31, or 1240MiB). That fits will enough in the 10 MiB desired survior space without triggering unwanted promotion. But that does imply that the desired survivor space needs to be around 6 to 12 MiB per player if you are doing any moving -- which means survivor buckets of 12 to 24 MiB.
Note that you cannot (as far as I can tell) specify the size of the survivor buckets -- you have to specify the new size and survivor ratios. Yucky.
If you are moving around a construction/base area, you can be moving more than 16 blocks in each direction trivially -- we move more than 32 blocks around the inside of our base, and it was not built with chunk aligned insides. That means in our base, I'm running around a 4x4 chunk area.
Which in turn means that the outer 4 strips in all 4 directions are being reloaded and flushed.
That's about 16x27 (31 length would give overlap) x16 x16x16 x6 (we built in extreme hills) x2 of data being flushed as I move around.
That's over 20MiB of stuff being thrown away regularly.
But it doesn't trigger constant full GC's -- the GC system seems to be able to handle movement just fine.
So the survivor buckets don't need to be that large. Other than going through portals, 20 MiB buckets were working just fine, and are probably larger than needed.
3. Java's "I'll reduce the size of my buckets and return unused memory to the OS" is worthless. Turns out that Java wants 70% free space in any bucket before it shrinks that bucket.
That will only happen at shutdown.
Current recommendations:
1. Use the incremental collector. While the throughput collector does have a flag to say "Keep collections shorter than Nms", that is a "Opps, we took too long, adjust our thresholds a little and try again" behavior.
2. Allocate memory based on the assumption of 16 MiB survivor buckets per player. (Technically: group of players in the same location -- this is dominated by movement loading new chunks, and if the players are in the same location they will load like a single player)
If you have the standard ratio of 6:1 for eden versus survivor, that's 8*16 = 128MiB of "new" space per player you want to support.
So for a 12 person PvP battle, or for 12 parties/factions with small bases, you want -XX:NewSize=1536m.
Equally, if you are going to be playing underground in caves, or have players that cannot use "far" outdoors and want a balanced fight, lower the server view distance.
3. The client generally needed a little over 200MiB of long-term space:
Note the timestamps on these:
But for around 200m of long-term memory, 350m of "old" working space, 128m of "new", and 42m of "perm", that's a client memory usage of about 500m to have enough memory to keep garbage collection at a minimum. Less than 500m will have excessive GC; more than 500m will reduce memory available to other programs (such as Firefox -- the bigger memory hog :-). And, if you are using a lot of portals, you'll want to add more.
So that's 512m for people who don't use portals a lot, or 640m for people who do use portals a lot.
That -- along with incremental GC -- is the best recommendation for people who want minecraft to multitask with other programs while playing on a server (as opposed to locally).
Note also: That's based on using the 64 bit java for "far" support. If you are not using far, you need a lot less (32 bit will take less memory AND you have less to deal with).
4. The server:
Well, it turns out that "-d32 -server" gives slightly different output.
There are full collections, they just output differently. If you know how to read them:
"Minor" cleanings:
The server seems to have an 80m overhead when no players are logged on!
About the only thing I can conclude from this is a minimum of 180m per player for old storage, plus 128m for "new" (again, that's per party -- dominated by movement), plus another 90-180m for temporary/change in the long-term, or about 400m+ per group plus a smaller amount per player, plus another 80m for overhead on your server.
Without testing the integrated client, the "predictable" is that they can share "new" space nicely, but the "world" will be loaded twice -- once on the client side, once on the server side -- and you're looking at 640m (client) plus 250m for server data, or 900m for minimal garbage collection while keeping memory usage under control.
Small machines (2g or less, definitely for 1g systems) really want to consider disabling far vision.
This is now at the limit of my knowledge of tweaking java's GC. Anyone want to push farther?
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
So it turns out that "-XX:+UseParNewGC" is not a new version of "-XX:UseParallelGC".
ParNewGC does parallel collection of the young generation, and is designed to work with the CMS collection of the old/tenured generation.
Additionally: -XX:+CMSParallelRemarkEnabled -- speeds up concurrent remarking, and -XX:CMSFullGCsBeforeCompaction=1 -- makes sure that if CMS gets part/most of the way through when memory fills up, that work is not wasted, but finished.
Still to determine: How to reduce full collections (generally: larger memory), and how to reduce memory (generally: More collections) together :-).
More testing will follow, probably thursday.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
That, I was not aware of.
As I understand it, "9" means 9 chunks radius from your chunk. "Tiny" view is radus 2 chunks, "short" is radius 4 chunks, "normal" is radius 8 chunks, and normal vanilla behavior is to load stuff at radius 9 chunks and tick (and display) blocks within radius 8 chunks; as blocks can affect blocks slightly more distant when they tick, that next group of chunks has to be loaded. But this means that "9" is equivalent to "normal" vanilla single player range.
Going down to less than 9, I thought only affected what was sent to the client, not what the server actually worked with. If that's the case, then drop it down to 9, and not lower.
Am I the only one who thinks this is a server bug?
===
More on memory: a quick update:
Java 6 options (found it finally): http://www.oracle.co...jsp-140102.html
Java 6 garbage collection optimization: http://www.oracle.co...g-6-140523.html, http://www.slideshar...llection-tuning
The big key:
Collecting "new" memory is fast. No matter how big you make "new", the speed of collection depends on the number of "live" objects in it that are being copied.
Collecting "tenured" objects is slow. The bigger you make "tenured", the _SLOWER_ it will be.
Fast behavior: Make tenured as big as you need -- 150% of actual usage is fine -- but no bigger. Make "new" as big as you can afford the memory for.
Once you make "new" big, the next tuning is "how many times do I copy a new object, and give it a chance to die, before I commit it to tenured space and ignore it for a long time".
Current testing, which seems to work well for 1-2 players that are mostly together: 100m of new, survivor ratio of 6 (gives "survivor" size of 13m), max copying of 4. Then the "-Xmx" can be 500m and the server is happy. VERY happy.
Next test is "stress" -- twilight forest and mystcraft (it already passes the nether test).
Client side is still being tweaked/tested before I make any recommendations.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Me neither, but I found out the hard way when I dropped it to 7 from 10, suddenly mobs just stopped spawning. Well, they still spawn, just, they turn into very rare little creatures. I found out it had to do with the view range after doing a reset on the configuration and noticing all of a sudden things were back to normal. Seeing as the only thing I changed was the view range and the difficulty level, tinkering around lead to the strange finding that view-range is linked to mob spawns, somehow.
Pretty much yes, actually the default is 10 chunks, according to the wiki (I think), the irony being that while it's supposedly sending less to the client, you still end up with the same number of chunks in the chunk cache (hit F3 and see the multiplayerchunkcache value), so either it sends them slower, or it ignores this setting for some reason.
In theory yes, practically it seems that mob spawning and mob spawn rates are tied into what's sent to any given client for some odd reason. Probably has something to do with a mob being an entity in a chunk, instead of not being associated in that way at all. Load less chunks, see less entities, get lower spawn rates because the spawn 'find a spot to spawn stuff' selection algorithm goes funny.
I'm not sure if it's a bug, I'd like to think it is...
http://www.minecraftforum.net/topic/63836-making-your-server-lag-less-by-tuning-java-settings/
Dates from 2010, and includes Evil Seph.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
https://blogs.oracle.../our_collectors
One of the guys behind the collectors talks about them. And this is the first description of "garbage first" I could find.
Sadly, the details appear to be behind a paywall -- http://portal.acm.or....cfm?id=1029879 -- and I don't have access.
But it looks to be a case of being able to say "Every 1/20th of a second, spend up to N milliseconds collecting as much garbage as you can". With compaction (which CMS cannot provide).
I lack the knowledge of how to adjust G1's tuning knobs; I lack the knowledge of what those knobs even are. Just that they exist. And, apparently, it isn't even clear that J6 has a complete/good implementation; that may require J7.
"If G1 works out as we expect, it will become our low-pause collector in place of "ParNew" + "CMS". And if you're about to ask when will it be ready, please don't be offended by my dead silence.".
"With G1 you can specify a time slice NNN and the maximum amount of time to be spent on GC in that time slice MMM. G1 will then schedule it's work such that for any NNN time slice no more than MMM will be spent doing GC. You can clearing over specify this goal (MMM = 0 will never work) but G1 tries to provide the application with at least NNN - MMM per time slice."
Now, granted that's not quite what we want -- we want "Every 1/20th of a second, the main loop begins; take as much time as you want once the main loop has finished; that time available to you will change each time through the loop". But it's a big step -- it's the first indication that this is a real-time garbage collector (which is confirmed in other comments on that page).
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Desired survivor size 6553600 bytes, new threshold 1 (max 4)
- age 1: 11097320 bytes, 11097320 total
It's not even close to running out of memory -- but it's constantly filling up with temporaries, triggering almost constant background CMS garbage collections.
Both TF and Dynmap want lots of temp memory, and I think Dynmap is worse -- I don't recall having any issues with just TF. I suspect TF is representative of the "nasty" mods -- mods with lots of creatures and blocks.
More tuning will come.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
This add-in wants 200 MB survivor spaces (70-90 MB of temporary stuff in the first surviving generation is common). Tossing 600 MB at eden (to keep new generation GC's down), results in mostly behaved memory consumption (Yes, that's 1 GB for new memory space). At full speed rendering, it still overwhelms the tenured GC's ability to keep memory free, and goes up to 600 MB of tenured space used (in fairness, that was the limit in my test -- I don't know where it would actually max).
Giving it a throttle lowers the tenured space used to a maximum (allocated) of just over 300 MB, and a 200 MB actual usage. Since the recommendation for tenured size is 150% or more of actual, that's not a bad match, and that's only during a full render with a complicated set of maps.
At the moment, it looks like 200 MB for tenured is more than sufficient for a small (3 player) server (add 100 MB for dynamic maps). New/Eden/Survivor, on the other hand, will vary significantly based on mods.
Draft zero of a tuning walk-through should be ready in about 2-3 weeks.
In the meantime, here is my current startup command:
Yes, that's calling for the 32 bit server -- Oracle actually says that the 32 bit version works fine for heaps up to about 2.5 GB, and will consume less memory than the 64 bit. "MaxHeapFreeRatio" is supposed to return unused memory to the operating system, but it has no effect on Macintosh.
Other than that, the startup is calling for 12.5 MB survivor spaces -- which is overkill/waste more than half the time. The goal here is to give "eden" enough room that new collections happen infrequently enough for objects to go "dead" before survivor space fills up or 4 collections occur. And, for a 1-3 person server, that works. Single player should be similar.
Again, this is with Twilight Forest -- which spams giant trees, massive leaf canopies, and large amounts of peaceful animals. High block count, high passive mob count. Vanilla will probably be less except in a giant extreme hills environment.
For the client, my current startup is
(That goes on the advanced tab in magic launcher)
It is not nearly as optimized, but it works.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Me:
-XX:+UseConcMarkSweepGC -XX:+UseParNewGC \
-XX:+CMSIncrementalPacing -XX:ParallelGCThreads=2 \
Garbage collector definitions. Should be familiar.
If you have 8 cores, *you do not want incremental*.
Change that second line to
-XX:ParallelGCThreads=4 \
and then experiment with raising it up from 4 to 8.
Partner:
So I removed this and it had a huge negative impact on the server. With this line removed, entities warp around and the tick ms is at an average of 40ms at all times, with the line added, there is no warping of entities and the server ms is an average of 8ms.
Me:
Very interesting. That's the opposite of what Oracle recommends.
Adding to my toolkit ...
===
Bottom line: Incremental garbage collection (not CMS, but breaking CMS into pieces), which Oracle only recommends for single or 2-core systems, actually was better on an 8-core system.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
So, what does that mean at a high level for a player like me? I'm well-versed in programming (java at that) but not nearly as technical as you. Would changing any of these improve performance at all? Note that I have 8GB allocated to Minecraft; I know I don't need that much but I play with tons of mods and HD textures, so it's good to have extra--and I have 32gb ram on my system anyways. I realize these findings probably only affect memory usage, which is obviously not a problem for me, but I'm just curious to see if they do have an impact on performance. I should also note that I'm using a 4 core processor.
Click the picture!
-Derek Shunia
This is all about performance.
Java, by default, assumes that you want a throughput collector, and you have relatively low long-term data storage. This is a great assumption for dedicated servers that serve web pages. This is a bad assumption for minecraft.
Minecraft wants fast responsiveness, and the default garbage collector cannot provide it.
Minecraft wants several hundred MB of long-term data storage (at least 200 MB, more with texture packs, more if you have a single player game with both server and client in one package, more if you have a server with lots of players, more if those players are not in the same place, etc.)
Minecraft is two single-threaded operations. While Java can do some bookkeeping work on the other cores, the majority of the code can only use two CPU's -- so if you have more cores, you want to let Java do more stuff on those cores at the same time. Hence the desire to use concurrent garbage collection if possible (so garbage collection doesn't slow the system down)
The default allocation of "new vs old" cannot fit all cases. The more memory you have available for minecraft, the more you want into new. Having the tenured space get bigger and bigger only causes the full collections to get less frequent and longer. Having more space for "new" results in less time spent in collections. Java wants to use a system based on percent of memory for this, percent of memory for that, and it cannot match what minecraft wants/needs for all memory sizes.
And, in my case, I want to have a voice chat server, voice chat client (and recorder), screen recorder, multiplayer server, and client, all running on the same box -- so I can't just say "Give it all the memory I have", I have to figure out how much it needs / how much to allocate where.
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
Give me a couple of days, and I should have some baseline numbers to use to start making recommendations. Currently have on my to-do list a server log from a server with over 100 mods, a decent player base, and a small memory leak.
As a simple case of "number from nothing", try a tenured size of 320 MB plus 30 MB per person, and then make "new" as big as you can without exhausting physical memory. Vanilla only needs about 45 MB for perm, but with lots of mods it can overfill 90 MB and run out of memory -- if you get errors there, try raising perm to 150 (should be overkill). (Perm is used for bytecode and compiled code).
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?
http://www.oracle.co...rted/index.html
Brief summary: Memory is divided into approximately 2,000 cells (tunable); each cell may contain any type (survivor, eden, tenured, etc) as needed. This permits memory allocation to vary based on loads. Cells that are unused are very fast to collect; collector determines which cells are fastest to collect and collects those. Minimum size of a cell is 1 MB; recommended for heap sizes of 6 GB or larger.
The primary tuning control is the desired application pause time; the default is 1/5th second. First approximation for minecraft would be to target 1/40th of a second (half a tick).
I expect to play with this in detail, probably in february.
Waiting for feedback from someone with a decent-sized server with memory issues before making any recommendations; expect something no later than wednesday. Script for launching a server with easy access to memory tuning parameters is done; modifying it to launch a client is "in progress".
* Promoting this week: Captive Minecraft 4, Winter Realm. Aka: Vertical Vanilla Viewing. Clicky!
* My channel with Mystcraft, and general Minecraft Let's Plays: http://www.youtube.com/user/Keybounce.
* See all my video series: http://www.minecraftforum.net/forums/minecraft-editions/minecraft-editions-show-your/2865421-keybounces-list-of-creation-threads
(In regard to a mod that gives realistic animal genetics):
Would you really rather have bees that make diamonds and oil with magical genetic blocks?
... did I really ask that?