#!/bin/bash # This line is for starting from mac os icon double click cd "$( dirname "$0" )" # V1.7.1. Minor tweaks: # - compile threshold lowered from 3,000 to 300. Oracle's default for servers is 15,000; # my testing fails to show any benefit from this. Faster startup, quicker initial block # breaking, etc. # - Initial memory allocation lowered # - Clarification to the comments at the end where you select "favor extra space for tenured" # or "favor extra space for new" memory. # - MinHeapFreeRatio=21 instead of 20 -- solves a borderline issue where 80% of memory was # used, 20% was free, and repeated full CMS collections were run without expanding memory # allocation to java. # - Keep one old GC.log file around. # - Changed from "Release Candidate" to "Official first release" :-). # V1.7: Borderline case existed where memory allocation would be in error. # Example: Survivor copy of 90, 90% => Survivor space of 100. # Desired Eden of 250. # Would attempt to allocate 200 for eden, 200 for two survivors. # That's 400 for new, even if "MaxNew" was 350. # Right behavior in this case? # No clear answer. # What's more important: making sure you have as much eden as you ask for, # or limiting your "new" memory? # # Answer used here: "new" memory is more important than tenured, # as long as tenured is big enough. So, if that happens, we force maxnew up to fit. # When this happens, a warning is printed. Pay attention to your logs when this # happens -- tenured overflow results in very large amounts of full collections # because small collections cannot clear eden. ## V1.6. This time the 64 bit flag works. # Again: This is throwing "too much" memory at eden/survivor. # Pay attention to the GC log. Sections like # 1552.959: [CMS-concurrent-reset-start] # 1552.965: [CMS-concurrent-reset: 0.005/0.005 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] # {Heap before GC invocations=17 (full 1): # par new generation total 180224K, used 146721K [0000000009000000, 0000000016c00000, 0000000025200000) # eden space 135168K, 100% used [0000000009000000, 0000000011400000, 0000000011400000) # from space 45056K, 25% used [0000000014000000, 0000000014b48750, 0000000016c00000) # to space 45056K, 0% used [0000000011400000, 0000000011400000, 0000000014000000) # concurrent mark-sweep generation total 102400K, used 63696K [0000000025200000, 000000002b600000, 000000003b000000) # concurrent-mark-sweep perm gen total 24940K, used 21197K [000000003b000000, 000000003c85b000, 0000000044600000) # The "CMS-concurrent-reset" means it has finished a full collection. # The "concurrent mark-sweep generation" line has the tenured allocation # In this case, 63 MB of long-term data. # Sections like # 56.089: [GC 56.089: [ParNew # Desired survivor size 41523608 bytes, new threshold 4 (max 4) # - age 1: 17033216 bytes, 17033216 total # - age 2: 22458216 bytes, 39491432 total # - age 3: 14552 bytes, 39505984 total # - age 4: 136648 bytes, 39642632 total # 17 MB of temporary space was recently allocated; just under 40 MB of temporaries are "surviving" # and not immediately useless. (This comes from startup; almost all of that will be valid for the life # of the program run.) # Age 4 overflows (age 4's total larger than the max, or worse, "new threshold 2" or other small number) # means the potential for tenured space to grow unnecessarily. This kills long-term performance. ## V1.5-corrected: 64 bit version, and ** LOTS OF MEMORY ALLOCATION ** # This is to give information for data collection. # Determine how much your system uses, and then adjust numbers down to avoid waste # # Specifically: By making the memory pools large, we see how much is used. # Then, we can determine what to cut them down to. # # This will probably be the last CMS version; G1GC next. java=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Commands/java ## V1.4: Java is now customizable (see the above line), thank you oracle for a ## java (1.7) that does not support 32 bit servers for reduced memory consumption. ## V1.3: The -XX:SoftRefLRUPolicyMSPerMB=0 flag got lost! Back in now. ## This flag defaults to 1000, and can cause memory leaks. ## V1.2: We now play with -XX:TargetSurvivorRatio=n to reduce waste in new, permitting more ## space to be used # Configurables: # -d32 is for heap size up to 2.5gb. (NB: apparently only 1.5 gb on microsoft windows?) # Change to "-d64 XX:+UseCompressedOops" if you use more. # ** Mention that flag specifically, do not rely on it being autoset. # ** Known and documented JVM bug -- https://forums.oracle.com/forums/thread.jspa?messageID=10017916 JVM_SIZE="-d32 -server" # JVM_SIZE="-d64 -XX:+UseCompressedOops -server" # CMSInitiatingOccupancyFraction: Determine how frequently to do a full CMS # Special: New "Most important". This primarily affects long-term growth # of the heap. The percentage of used space before CMS starts a collection. # 95 is sufficient for general stuff. 85 is useful for things that suddenly # need a lot. # Dynamic maps, in particular, no longer needs 125 MB of survivor -- it can # get by with 60-75. It can go much lower, but then the garbage collections # need to be started sooner, or else it will never have enough memory and # always grow the heap. # # To clarify: This is obsolete -- completely -- in G1GC. # This needs to be low enough that sudden spurts of temporary memory trigger # garbage collection first. # This should be re-worked as a "MB safety level" -- for example, if you have # 300 MB of tenured, and want at least 30 MB free. But Java don't work that way. # As tenured increases, this will also increase the "keep free" level. CMSInitiatingOccupancyFraction=80 # Memory tuning: # Command line controls total heap, and "new". "Tenured" is the difference. # Bigger "new": Less frequent collections. # These numbers are in "Megabytes", the java "m" suffix. # The rule of memory tuning: # SurvivorSpace * (SurvivorRatio + 2) = New # ("SurvivorSpace" is twice the actual surviving threshold.) # SurvivorSpace * SurvivorRatio = Eden. # Two additional survivor spaces are used to copy surviving objects across minor collections. # MAX: Maximum heap space used. # Does not include permanent (byte/compiled code) # Does not include JVM overhead MAX=550 # Tenured: Desired long-term storage space # Will vary based on mods, and "loaded chunks" # -- how many parties of players close to each other. # # Starting assumption: 250 for 1 person, plus 50 per group # of players near each other. # # That is a guess. Please report what numbers work for your server. # # ** More testing. # Multiple people in the same area: No detectable difference between 1 and 2 people. # Actual space needed varied from (LDS minimum) 55 (1.5.1) to 75 (1.4.7, dynamic maps) # The old guess of 250 was way too high. # This will need to be higher on servers with more players. Tenured=100 # Most important tuning number. Survivor copy size. # Making this higher: Fewer full collections, but more wasted space. # During startup, expect this to overflow frequently. # Actual space allocated is 2 spaces, each one twice this size. # "waste/overhead" will be two about to three times this number. # *** Maximum of 1/6rd of "new" # Pay attention to the tenuring distribution logs. # *** This should be enough for generation 3 99%+ of the time. *** # ** TOO SMALL WILL KILL YOUR GARBAGE COLLECTION ** # (worse, it will cause tenured space to grow.) # ** TOO BIG WILL WASTE SPACE ** # (this is a minor issue overall.) # # To clarify: You can easily use 12 here if you stay primarily in one place. # You can even use less, but the memory savings are not significant. # Biggest usage I've seen: dimension travel (see below) and dynamic maps with # low angle (20 degree) maps. # # *** Recommendations # If you do a lot of dimension travel -- including using a nether hub to get # around, or lots of Mystcraft books (two links to go from A to B in the same # world), then you want 65 MB if you can afford it, or 53 MB minimum. That's # for one person; groups will be higher. # (tested in a world with forests at the hub zone; may be higher for extreme hills, # jungles, or jungle-hills (expected to be the worst case.)) # # If you do a lot of high speed rail travel over large distances, a minimum of # 40 MB. Ditto if you use a lot of flight mode movement. More if you have a # speed boost. # # If you do very little travel -- if you are just building in one spot -- then # 15 MB is plenty. # # If you are testing a new mod pack, and have no clue what you need, start at 100. # If you are working with a multiplayer server, I don't know what to advise -- # test and tell me. # # More testing: 147, dynamic maps, ocean area: 30 # 151, jungles/desert/forest: 38 # Probably are good at 32 here # Defaulting to 35 MB, works well on a server with extreme hills and no massive dimension # travel. Again: Changing dimension once every few minutes won't bother anything. Rapid # changing multiple times a minute will. SurvivorCopySize=35 # Survivor target ratio. Java defaults to 50%, which wastes a lot of space. If you know how much # you need (see below), you can set this value higher; this gives less waste and "better performance". TargetSurvivorRatio=90 ## Notes on "SurvivorCopySize": # Flying around in creative mode, in already generated chunks will want # at least 30-35, preferrably 40 meg. # Standing around, single player, can be happy with less than 1. # Even in Mystcraft, with massive amounts of decay everywhere (implies lots of block # updates), 95% of the time 1 meg suffices. # Moving around a little, doing basic building/digging, about 3. # # The rule: You want to see "new threshold 4 (max 4)" most of the time. # The total value at age three -- # - age 3: 36712 bytes, 5897520 total # should be less than this almost all of the time, at least after startup. # # 12 meg is more than enough for one person with EBXL, Mystcraft, Twilight Forest, # and Custom Ore Gen. Even in EBXL's extreme jungle with Mystcraft's decay littering the ground. # # The single biggest factor is chunks loaded; that will depend more on parties than on players, # and the speed at which they move. Adjust to your server, and your mods. Remember, teleportation # will raise this value very rapidly. # # To clarify the problem: An overflow here forces memory into tenured space. If tenured is # nearly full -- lets say Java will start a garbage collection soon -- then this can force # more space to be allocated to tenured. # # Forcing more space into tenured is the disaster that this is all about preventing. # Tenured space is more expensive to collect, and Java does not release this back to the # operating system when it does not need it. This means, on systems that are doing more than # just a minecraft server (say, a client, a browser, perhaps voice chat software, etc), this # will impact your whole system's performance. # # A secondary effect of overflow is forcing earlier/more frequent full collections. # For single/dual processors, this may be the more dominant factor. # For 4/8 core processors, this is minor. # Second most important tuning. Eden. # Making this bigger means less frequent small collections. # General rule: Make this as big as your memory can handle. # Must be at least 2x Survivor. Java requires it to be # an integer multiple of Survivor. # # If you have no idea what to use here, then right at the end # there are a pair of commended lines that will calculate # something for you. That calculation uses Max, Tenured, and # copy, and assumes everything else is new/Eden. Result is a # bigger footprint (should always use max or close to it) and # the biggest eden space available. # # Specifying a value here will reduce the starting eden, and # give space for tenured to grow later. desiredEden=30 # Summary: Approximately desiredEden, plus 2 times Survivor, # plus 130, will be used by java to start the heap. Up to a max of MAX. # Script will attempt to ensure at least Tenured space exist; # should exit with a message if it cannot. # # In theory, Java will allocate extra space to new or tenured as needed. # In practice, I've never seen it increase "new". # # See the bottom of the config section for more. # If your shell cannot do math, replace these with an appropriate constant MaxNew=$(($MAX - $Tenured)) ## Survivor=$((2 * $SurvivorCopySize)) ## Working with survivor target. "2" is for 50%. For 90%, it's much closer to 1. ## What we want is 100 / target percentage, as the ratio instead of 2. ## For integer only shell math, we re-write as (100 * survivor) / target, which gives us ## close integer to the desired result -- as close as we can get in the shell. Survivor=$(( ($SurvivorCopySize * 100 ) / $TargetSurvivorRatio )) ## Equally, the "3" in sanity test is from 3 bins -- two survivors, one eden. ## But that does NOT change here -- it's still the sanity test lower limit. sanityTest=$((3 * $Survivor)) if [ $sanityTest -gt $MaxNew ] then echo Memory config error >& 2 exit 1 fi # We cannot use more than MaxNew. # The idea: # 1. Find the multiple of Survivor that is bigger than S and less than MN. # 2. Determine survivor ratio from that. Subtract 2 (java.) # 3. Specify -Xmn for new, and survivor ratio, to set eden and new. # "New" will be Eden plus 2* Survivor. # MaxRatio -- what the ratio is if we use all of maxnew. MaxRatio=$(( ($MaxNew / $Survivor) - 2 )) # DesiredRatio -- what the ratio is based on declared eden space # There is no "-2" here -- this will allocate eden plus 2* survivor. desiredRatio=$(( ($desiredEden / $Survivor) )) # SurvivorSpace * (SurvivorRatio + 2) = New # Now check for "desired Eden". If survivor is not an exact multiple of DE, # then we have just rounded down. Test for this, and if so, see if we can # raise it up (watch out for maxnew) ## TODO! FIXME! This is a cheap approximation if ( [ $(( $desiredRatio + 1 )) -le $MaxRatio ] ) then desiredRatio=$(( $desiredRatio + 1 )) fi desiredNew=$(($Survivor * ($desiredRatio + 2) )) biggerNew=$(($Survivor * ($MaxRatio + 2) )) echo Debug: Max ratio $MaxRatio, desiredRatio $desiredRatio echo Debug: biggerNew $biggerNew, should be less than MaxNew $MaxNew echo Debug: desired eden $desiredEden, survivor $Survivor, actual new $desiredNew # desiredNew: Gives an eden up to, not bigger, than desiredEden. # biggerNew: Gives an eden at least as big as desiredEden. # FIXME: DesiredNew / ratio should be smallest at least as big as desiredEden # This means, if less, then add 1 to ratio and add to new. # # "Bigger" assigns ALL non-tenured memory to new. # Q: Desired numbers? Bigger/Max numbers? # Choose one of these pairs # Use the user specified Eden. Starting memory footprint will be smaller. # Tenured can grow if needed. NEW=$desiredNew RATIO=$desiredRatio # If you have no idea what values to use for Eden, then this will determine # eden values on it's own. This will use your specified tenured, max, and # copy values, and assign close to the maximum footprint and eden space. # NEW=$biggerNew # RATIO=$MaxRatio ### 1.7 bug fix! Turns out that borderline cases could fail. if ( [ $NEW -gt $MaxNew ] ) then MaxNew=$NEW ## CAUTION! "Start" below may fail. ## If you get here, then you need to watch out for tenured overflow echo "** Launcher adjusted MaxNew; make sure tenured space is big enough" fi START=$(($NEW + 90)) # Experimal behavior: Starting up, and loading one person into a world, takes # about 70-90 MB of tenured space to have Java "happy". Less just results in repeated # full collections while Java raises the amount of allocated memory. (Range depends on # mods.) ## TESTME: Does "MaxNewSize" matter if we have adaptive GC boundary? Does it hurt? # A few more notes ... # -XX:+UseAdaptiveGCBoundary -- apparently, adjust the boundary between new and tenured as needed. # Nice to see; did not know about it before. # Sadly, it seems to have no effect. # -XX:+CMSIncrementalMode: Tells the garbage collector to break the job into many small parts. # May result in better performance. Essential on systems with few cores. # -XX:CompileThreshold: Tells Java when to convert bytecode into compiled code. # Server defaults to something like 15,000, which is appropriate for something that will # run for months, and horrible for something that will run for a few hours. # Non-server defaults to ... 1000? 1500? Much lower. # "-f" prevents an error message on the first run mv -f GC.log GC.log.old exec $java \ $JVM_SIZE \ -Xms${START}m -Xmx${MAX}m \ -XX:+UseAdaptiveGCBoundary \ -XX:TargetSurvivorRatio=$TargetSurvivorRatio \ -XX:NewSize=${NEW}m -XX:MaxNewSize=${MaxNew}m \ -XX:SurvivorRatio=$RATIO \ -XX:CompileThreshold=300 \ -XX:+UseConcMarkSweepGC -XX:+UseParNewGC \ -XX:CMSInitiatingOccupancyFraction=$CMSInitiatingOccupancyFraction \ \ -XX:SoftRefLRUPolicyMSPerMB=0 \ -XX:MaxPermSize=150m \ -XX:+UseParNewGC \ -XX:MaxHeapFreeRatio=25 \ -XX:MinHeapFreeRatio=21 \ -XX:+DisableExplicitGC \ -XX:MaxTenuringThreshold=4 \ -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution \ -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log \ -jar new_server.jar nogui 147Main # The last word of that exec statement -- '147Main' -- is just something that shows up in # the process list, so I can tell which process is which server (each copy of this script # has a different name in that field).