SHOW:
|
|
- or go back to the newest paste.
1 | - | #!/bin/bash |
1 | + | #!/bin/bash |
2 | # This line is for starting from mac os icon double click | |
3 | cd "$( dirname "$0" )" | |
4 | ||
5 | # V1.7: Borderline case existed where memory allocation would be in error. | |
6 | # Example: Survivor copy of 90, 90% => Survivor space of 100. | |
7 | # Desired Eden of 250. | |
8 | # Would attempt to allocate 200 for eden, 200 for two survivors. | |
9 | # That's 400 for new, even if "MaxNew" was 350. | |
10 | ||
11 | # Right behavior in this case? | |
12 | # No clear answer. | |
13 | # What's more important: making sure you have as much eden as you ask for, | |
14 | # or limiting your "new" memory? | |
15 | # | |
16 | # Answer used here: "new" memory is more important than tenured, | |
17 | # as long as tenured is big enough. So, if that happens, we force maxnew up to fit. | |
18 | # When this happens, a warning is printed. Pay attention to your logs when this | |
19 | # happens -- tenured overflow results in very large amounts of full collections | |
20 | # because small collections cannot clear eden. | |
21 | ||
22 | ## V1.6. This time the 64 bit flag works. | |
23 | # Again: This is throwing "too much" memory at eden/survivor. | |
24 | # Pay attention to the GC log. Sections like | |
25 | ||
26 | # 1552.959: [CMS-concurrent-reset-start] | |
27 | # 1552.965: [CMS-concurrent-reset: 0.005/0.005 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] | |
28 | # {Heap before GC invocations=17 (full 1): | |
29 | # par new generation total 180224K, used 146721K [0000000009000000, 0000000016c00000, 0000000025200000) | |
30 | # eden space 135168K, 100% used [0000000009000000, 0000000011400000, 0000000011400000) | |
31 | # from space 45056K, 25% used [0000000014000000, 0000000014b48750, 0000000016c00000) | |
32 | # to space 45056K, 0% used [0000000011400000, 0000000011400000, 0000000014000000) | |
33 | # concurrent mark-sweep generation total 102400K, used 63696K [0000000025200000, 000000002b600000, 000000003b000000) | |
34 | # concurrent-mark-sweep perm gen total 24940K, used 21197K [000000003b000000, 000000003c85b000, 0000000044600000) | |
35 | ||
36 | # The "CMS-concurrent-reset" means it has finished a full collection. | |
37 | # The "concurrent mark-sweep generation" line has the tenured allocation | |
38 | # In this case, 63 MB of long-term data. | |
39 | ||
40 | # Sections like | |
41 | ||
42 | # 56.089: [GC 56.089: [ParNew | |
43 | # Desired survivor size 41523608 bytes, new threshold 4 (max 4) | |
44 | # - age 1: 17033216 bytes, 17033216 total | |
45 | # - age 2: 22458216 bytes, 39491432 total | |
46 | # - age 3: 14552 bytes, 39505984 total | |
47 | # - age 4: 136648 bytes, 39642632 total | |
48 | ||
49 | # 17 MB of temporary space was recently allocated; just under 40 MB of temporaries are "surviving" | |
50 | # and not immediately useless. (This comes from startup; almost all of that will be valid for the life | |
51 | # of the program run.) | |
52 | ||
53 | # Age 4 overflows (age 4's total larger than the max, or worse, "new threshold 2" or other small number) | |
54 | # means the potential for tenured space to grow unnecessarily. This kills long-term performance. | |
55 | ||
56 | ## V1.5-corrected: 64 bit version, and ** LOTS OF MEMORY ALLOCATION ** | |
57 | # This is to give information for data collection. | |
58 | # Determine how much your system uses, and then adjust numbers down to avoid waste | |
59 | # | |
60 | # Specifically: By making the memory pools large, we see how much is used. | |
61 | # Then, we can determine what to cut them down to. | |
62 | # | |
63 | # This will probably be the last CMS version; G1GC next. | |
64 | ||
65 | - | #JVM_SIZE="-d32 -server" |
65 | + | |
66 | - | JVM_SIZE="-d64 -XX:+UseCompressedOops -server" |
66 | + | |
67 | ## V1.4: Java is now customizable (see the above line), thank you oracle for a | |
68 | ## java (1.7) that does not support 32 bit servers for reduced memory consumption. | |
69 | ||
70 | ## V1.3: The -XX:SoftRefLRUPolicyMSPerMB=0 flag got lost! Back in now. | |
71 | ## This flag defaults to 1000, and can cause memory leaks. | |
72 | ||
73 | ## V1.2: We now play with -XX:TargetSurvivorRatio=n to reduce waste in new, permitting more | |
74 | ## space to be used | |
75 | ||
76 | # Configurables: | |
77 | # -d32 is for heap size up to 2.5gb. (NB: apparently only 1.5 gb on microsoft windows?) | |
78 | # Change to "-d64 XX:+UseCompressedOops" if you use more. | |
79 | # ** Mention that flag specifically, do not rely on it being autoset. | |
80 | # ** Known and documented JVM bug -- https://forums.oracle.com/forums/thread.jspa?messageID=10017916 | |
81 | ||
82 | JVM_SIZE="-d32 -server" | |
83 | # JVM_SIZE="-d64 -XX:+UseCompressedOops -server" | |
84 | ||
85 | # CMSInitiatingOccupancyFraction: Determine how frequently to do a full CMS | |
86 | # Special: New "Most important". This primarily affects long-term growth | |
87 | # of the heap. The percentage of used space before CMS starts a collection. | |
88 | # 95 is sufficient for general stuff. 85 is useful for things that suddenly | |
89 | # need a lot. | |
90 | # Dynamic maps, in particular, no longer needs 125 MB of survivor -- it can | |
91 | # get by with 60-75. It can go much lower, but then the garbage collections | |
92 | # need to be started sooner, or else it will never have enough memory and | |
93 | # always grow the heap. | |
94 | # | |
95 | # To clarify: This is obsolete -- completely -- in G1GC. | |
96 | # This needs to be low enough that sudden spurts of temporary memory trigger | |
97 | # garbage collection first. | |
98 | # This should be re-worked as a "MB safety level" -- for example, if you have | |
99 | # 300 MB of tenured, and want at least 30 MB free. But Java don't work that way. | |
100 | # As tenured increases, this will also increase the "keep free" level. | |
101 | - | MAX=3000 |
101 | + | |
102 | ||
103 | # Memory tuning: | |
104 | # Command line controls total heap, and "new". "Tenured" is the difference. | |
105 | # Bigger "new": Less frequent collections. | |
106 | ||
107 | # These numbers are in "Megabytes", the java "m" suffix. | |
108 | ||
109 | # The rule of memory tuning: | |
110 | # SurvivorSpace * (SurvivorRatio + 2) = New | |
111 | - | Tenured=450 |
111 | + | |
112 | # SurvivorSpace * SurvivorRatio = Eden. | |
113 | # Two additional survivor spaces are used to copy surviving objects across minor collections. | |
114 | ||
115 | # MAX: Maximum heap space used. | |
116 | - | # Dynamic maps wants this at least 100, preferrably 125. |
116 | + | |
117 | # Does not include JVM overhead | |
118 | MAX=1300 | |
119 | ||
120 | # Tenured: Desired long-term storage space | |
121 | # Will vary based on mods, and "loaded chunks" | |
122 | # -- how many parties of players close to each other. | |
123 | # | |
124 | - | ## SurvivorCopySize=26 |
124 | + | |
125 | - | ## 26 is too small -- memory consumption keeps going up with dynamic maps |
125 | + | |
126 | # | |
127 | - | # To clarify: You can easily use 12 here if you do not use dynamic maps. |
127 | + | |
128 | Tenured=500 | |
129 | - | # You can use less if you use lowres maps. |
129 | + | |
130 | - | # The big memory consumer is low-angle maps; the 30 degree maps of high res, |
130 | + | |
131 | - | # or the 20 degree maps I use on my server are the big memory consumers. |
131 | + | |
132 | # During startup, expect this to overflow frequently. | |
133 | # Actual space allocated is 2 spaces, each one twice this size. | |
134 | # "waste/overhead" will be two about to three times this number. | |
135 | # *** Maximum of 1/6rd of "new" | |
136 | # Pay attention to the tenuring distribution logs. | |
137 | # *** This should be enough for generation 3 95%+ of the time. *** | |
138 | # ** TOO SMALL WILL KILL YOUR GARBAGE COLLECTION ** | |
139 | # (worse, it will cause tenured space to grow.) | |
140 | # ** TOO BIG WILL WASTE SPACE ** | |
141 | # (this is a minor issue overall.) | |
142 | # | |
143 | - | # Even in Mystcraft, with massive amounts of decay everywhere, 95% of the time 1 meg suffices. |
143 | + | # To clarify: You can easily use 12 here if you stay primarily in one place. |
144 | # You can even use less, but the memory savings are not significant. | |
145 | # Biggest usage I've seen: dimension travel (see below) and dynamic maps with | |
146 | # low angle (20 degree) maps. | |
147 | # | |
148 | # *** Recommendations | |
149 | - | # should be less than this 95% of the time. |
149 | + | # If you do a lot of dimension travel -- including using a nether hub to get |
150 | # around, or lots of Mystcraft books (two links to go from A to B in the same | |
151 | # world), then you want 65 MB if you can afford it, or 53 MB minimum. That's | |
152 | # for one person; groups will be higher. | |
153 | # (tested in a world with forests at the hub zone; may be higher for extreme hills, | |
154 | - | # and the speed at which they move. Adjust to your server, and your mods. |
154 | + | # jungles, or jungle-hills (expected to be the worst case.)) |
155 | # | |
156 | - | # Single player won't need that much. Really. |
156 | + | # If you do a lot of high speed rail travel over large distances, a minimum of |
157 | # 40 MB. Ditto if you use a lot of flight mode movement. More if you have a | |
158 | # speed boost. | |
159 | # | |
160 | # If you do very little travel -- if you are just building in one spot -- then | |
161 | - | # Must be at least 2x SurvivorCopySize. Java requires it to be |
161 | + | # 15 MB is plenty. |
162 | - | # an integer multiple of that value. |
162 | + | |
163 | # If you are testing a new mod pack, and have no clue what you need, start at 100. | |
164 | # If you are working with a multiplayer server, I don't know what to advise -- | |
165 | # test and tell me. | |
166 | ||
167 | - | # plus 100, will be used by java to start the heap. Up to a max of MAX. |
167 | + | # SurvivorCopySize=65 |
168 | SurvivorCopySize=100 | |
169 | ||
170 | # Survivor target ratio. Java defaults to 50%, which wastes a lot of space. If you know how much | |
171 | # you need (see below), you can set this value higher; this gives less waste and "better performance". | |
172 | ||
173 | TargetSurvivorRatio=90 | |
174 | ||
175 | ## Notes on "SurvivorCopySize": | |
176 | # Flying around in creative mode, in already generated chunks will want | |
177 | # at least 30-35, preferrably 40 meg. | |
178 | # Standing around, single player, can be happy with less than 1. | |
179 | # Even in Mystcraft, with massive amounts of decay everywhere (implies lots of block | |
180 | # updates), 95% of the time 1 meg suffices. | |
181 | # Moving around a little, doing basic building/digging, about 3. | |
182 | # | |
183 | # The rule: You want to see "new threshold 4 (max 4)" most of the time. | |
184 | # The total value at age three -- | |
185 | # - age 3: 36712 bytes, 5897520 total | |
186 | # should be less than this 98% of the time. | |
187 | # 12 meg is more than enough for one person with EBXL, Mystcraft, Twilight Forest, | |
188 | # and Custom Ore Gen. Even in EBXL's extreme jungle with Mystcraft's decay littering the ground. | |
189 | # | |
190 | # The single biggest factor is chunks loaded; that will depend more on parties than on players, | |
191 | # and the speed at which they move. Adjust to your server, and your mods. Remember, teleportation | |
192 | # will raise this value very rapidly. | |
193 | # | |
194 | # To clarify the problem: An overflow here forces memory into tenured space. If tenured is | |
195 | # nearly full -- lets say Java will start a garbage collection soon -- then this can force | |
196 | # more space to be allocated to tenured. | |
197 | # | |
198 | # Forcing more space into tenured is the disaster that this is all about preventing. | |
199 | # Tenured space is more expensive to collect, and Java does not release this back to the | |
200 | # operating system when it does not need it. This means, on systems that are doing more than | |
201 | # just a minecraft server (say, a client, a browser, perhaps voice chat software, etc), this | |
202 | # will impact your whole system's performance. | |
203 | # | |
204 | # A secondary effect of overflow is forcing earlier/more frequent full collections. | |
205 | # For single/dual processors, this may be the more dominant factor. | |
206 | # For 4/8 core processors, this is minor. | |
207 | ||
208 | # Second most important tuning. Eden. | |
209 | # Making this bigger means less frequent small collections. | |
210 | # General rule: Make this as big as your memory can handle. | |
211 | # Must be at least 2x Survivor. Java requires it to be | |
212 | # an integer multiple of Survivor. | |
213 | ||
214 | desiredEden=250 | |
215 | ||
216 | # Summary: Approximately desiredEden, plus 2 times Survivor, | |
217 | # plus 130, will be used by java to start the heap. Up to a max of MAX. | |
218 | # Script will attempt to ensure at least Tenured space exist; | |
219 | # should exit with a message if it cannot. | |
220 | # | |
221 | # In theory, Java will allocate extra space to new or tenured as needed. | |
222 | # In practice, I've never seen it increase "new". | |
223 | # | |
224 | # See the bottom of the config section for more. | |
225 | ||
226 | # If your shell cannot do math, replace these with an appropriate constant | |
227 | ||
228 | MaxNew=$(($MAX - $Tenured)) | |
229 | ||
230 | ## Survivor=$((2 * $SurvivorCopySize)) | |
231 | ## Working with survivor target. "2" is for 50%. For 90%, it's much closer to 1. | |
232 | ## What we want is 100 / target percentage, as the ratio instead of 2. | |
233 | ## For integer only shell math, we re-write as (100 * survivor) / target, which gives us | |
234 | ## close integer to the desired result -- as close as we can get in the shell. | |
235 | ||
236 | Survivor=$(( ($SurvivorCopySize * 100 ) / $TargetSurvivorRatio )) | |
237 | ||
238 | ## Equally, the "3" in sanity test is from 3 bins -- two survivors, one eden. | |
239 | ## But that does NOT change here -- it's still the sanity test lower limit. | |
240 | ||
241 | sanityTest=$((3 * $Survivor)) | |
242 | - | # New space is small -- specified eden. |
242 | + | |
243 | then | |
244 | echo Memory config error >& 2 | |
245 | exit 1 | |
246 | - | # Tenured is small -- specified tenured space. |
246 | + | |
247 | - | ## Should Not Be Needed -- "NewSize" and "MaxNewSize" specified separately. |
247 | + | |
248 | - | # In theory, Java should now adjust new as neeed. |
248 | + | |
249 | - | #NEW=$biggerNew |
249 | + | |
250 | - | #RATIO=$MaxRatio |
250 | + | |
251 | # 1. Find the multiple of Survivor that is bigger than S and less than MN. | |
252 | # 2. Determine survivor ratio from that. Subtract 2 (java.) | |
253 | # 3. Specify -Xmn for new, and survivor ratio, to set eden and new. | |
254 | ||
255 | # "New" will be Eden plus 2* Survivor. | |
256 | ||
257 | # MaxRatio -- what the ratio is if we use all of maxnew. | |
258 | MaxRatio=$(( ($MaxNew / $Survivor) - 2 )) | |
259 | # DesiredRatio -- what the ratio is based on declared eden space | |
260 | # There is no "-2" here -- this will allocate eden plus 2* survivor. | |
261 | desiredRatio=$(( ($desiredEden / $Survivor) )) | |
262 | ||
263 | # SurvivorSpace * (SurvivorRatio + 2) = New | |
264 | ||
265 | # Now check for "desired Eden". If survivor is not an exact multiple of DE, | |
266 | # then we have just rounded down. Test for this, and if so, see if we can | |
267 | # raise it up (watch out for maxnew) | |
268 | ||
269 | ## TODO! FIXME! This is a cheap approximation | |
270 | if ( [ $(( $desiredRatio + 1 )) -le $MaxRatio ] ) | |
271 | then desiredRatio=$(( $desiredRatio + 1 )) | |
272 | fi | |
273 | ||
274 | desiredNew=$(($Survivor * ($desiredRatio + 2) )) | |
275 | biggerNew=$(($Survivor * ($MaxRatio + 2) )) | |
276 | ||
277 | echo Debug: Max ratio $MaxRatio, desiredRatio $desiredRatio | |
278 | echo Debug: biggerNew $biggerNew, should be less than MaxNew $MaxNew | |
279 | echo Debug: desired eden $desiredEden, survivor $Survivor, actual new $desiredNew | |
280 | ||
281 | # desiredNew: Gives an eden up to, not bigger, than desiredEden. | |
282 | # biggerNew: Gives an eden at least as big as desiredEden. | |
283 | # FIXME: DesiredNew / ratio should be smallest at least as big as desiredEden | |
284 | - | -jar new_server.jar nogui 147test |
284 | + | |
285 | # | |
286 | - | # The last word of that exec statement -- '147test' -- is just something that shows up in |
286 | + | |
287 | ||
288 | # Q: Desired numbers? Bigger/Max numbers? | |
289 | ||
290 | # Choose one of these pairs | |
291 | ||
292 | # New space is sufficient -- specified eden. | |
293 | NEW=$desiredNew | |
294 | RATIO=$desiredRatio | |
295 | ||
296 | # borderline case: Not quite the desired eden, we ran into maxnew. | |
297 | # Disable these; we'd rather raise maxNew for Minecraft. | |
298 | ## NEW=$biggerNew | |
299 | ## RATIO=$MaxRatio | |
300 | ||
301 | ### 1.7 bug fix! Turns out that borderline cases could fail. | |
302 | if ( [ $NEW -gt $MaxNew ] ) | |
303 | then | |
304 | MaxNew=$NEW | |
305 | ## CAUTION! "Start" below may fail. | |
306 | ## If you get here, then you need to watch out for tenured overflow | |
307 | echo "** Launcher adjusted MaxNew; make sure tenured space is big enough" | |
308 | fi | |
309 | ||
310 | START=$(($NEW + 130)) | |
311 | # Experimal behavior: Starting up, and loading one person into a world, takes | |
312 | # 130 MB of tenured space to have Java "happy". Less just results in repeated | |
313 | # full collections while Java raises the amount of allocated memory. | |
314 | ||
315 | ## TESTME: Does "MaxNewSize" matter if we have adaptive GC boundary? Does it hurt? | |
316 | ||
317 | # A few more notes ... | |
318 | ||
319 | # -XX:+UseAdaptiveGCBoundary -- apparently, adjust the boundary between new and tenured as needed. | |
320 | # Nice to see; did not know about it before. | |
321 | # Sadly, it seems to have no effect. | |
322 | ||
323 | # -XX:+CMSIncrementalMode: Tells the garbage collector to break the job into many small parts. | |
324 | # May result in better performance. Essential on systems with few cores. | |
325 | ||
326 | # -XX:CompileThreshold: Tells Java when to convert bytecode into compiled code. | |
327 | # Server defaults to something like 15,000, which is appropriate for something that will | |
328 | # run for months, and horrible for something that will run for a few hours. | |
329 | # Non-server defaults to ... 1000? 1500? Much lower. | |
330 | ||
331 | exec $java \ | |
332 | $JVM_SIZE \ | |
333 | -Xms${START}m -Xmx${MAX}m \ | |
334 | -XX:NewSize=${NEW}m -XX:MaxNewSize=${MaxNew}m \ | |
335 | -XX:+UseAdaptiveGCBoundary \ | |
336 | -XX:SurvivorRatio=$RATIO \ | |
337 | -XX:TargetSurvivorRatio=$TargetSurvivorRatio \ | |
338 | -XX:CompileThreshold=3000 \ | |
339 | -XX:CMSInitiatingOccupancyFraction=$CMSInitiatingOccupancyFraction \ | |
340 | \ | |
341 | -XX:SoftRefLRUPolicyMSPerMB=0 \ | |
342 | -XX:MaxPermSize=150m \ | |
343 | -XX:+UseConcMarkSweepGC -XX:+UseParNewGC \ | |
344 | -XX:MaxHeapFreeRatio=20 \ | |
345 | -XX:MinHeapFreeRatio=15 \ | |
346 | -XX:+DisableExplicitGC \ | |
347 | -XX:MaxTenuringThreshold=4 \ | |
348 | -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution \ | |
349 | -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log \ | |
350 | -jar new_server.jar nogui 151myst10.3 | |
351 | ||
352 | # The last word of that exec statement -- '151myst10.3' -- is just something that shows up in | |
353 | # the process list, so I can tell which process is which server (each copy of this script | |
354 | # has a different name in that field). |