View difference between Paste ID: 8LYm2EMj and duLuWGJJ
SHOW: | | - or go back to the newest paste.
1-
#!/bin/bash 
1+
#!/bin/bash
2
# This line is for starting from mac os icon double click
3
cd "$( dirname "$0" )" 
4
5
# V1.7: Borderline case existed where memory allocation would be in error.
6
# Example: Survivor copy of 90, 90% => Survivor space of 100.
7
# Desired Eden of 250.
8
# Would attempt to allocate 200 for eden, 200 for two survivors.
9
# That's 400 for new, even if "MaxNew" was 350.
10
11
# Right behavior in this case?
12
# No clear answer.
13
# What's more important: making sure you have as much eden as you ask for,
14
# or limiting your "new" memory?
15
#
16
# Answer used here: "new" memory is more important than tenured,
17
# as long as tenured is big enough. So, if that happens, we force maxnew up to fit.
18
# When this happens, a warning is printed. Pay attention to your logs when this
19
# happens -- tenured overflow results in very large amounts of full collections
20
# because small collections cannot clear eden.
21
22
## V1.6. This time the 64 bit flag works.
23
# Again: This is throwing "too much" memory at eden/survivor.
24
# Pay attention to the GC log. Sections like
25
26
# 1552.959: [CMS-concurrent-reset-start]
27
# 1552.965: [CMS-concurrent-reset: 0.005/0.005 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
28
# {Heap before GC invocations=17 (full 1):
29
#  par new generation   total 180224K, used 146721K [0000000009000000, 0000000016c00000, 0000000025200000)
30
#   eden space 135168K, 100% used [0000000009000000, 0000000011400000, 0000000011400000)
31
#   from space 45056K,  25% used [0000000014000000, 0000000014b48750, 0000000016c00000)
32
#   to   space 45056K,   0% used [0000000011400000, 0000000011400000, 0000000014000000)
33
#  concurrent mark-sweep generation total 102400K, used 63696K [0000000025200000, 000000002b600000, 000000003b000000)
34
#  concurrent-mark-sweep perm gen total 24940K, used 21197K [000000003b000000, 000000003c85b000, 0000000044600000)
35
36
# The "CMS-concurrent-reset" means it has finished a full collection.
37
# The "concurrent mark-sweep generation" line has the tenured allocation
38
# In this case, 63 MB of long-term data.
39
40
# Sections like
41
42
# 56.089: [GC 56.089: [ParNew
43
# Desired survivor size 41523608 bytes, new threshold 4 (max 4)
44
# - age   1:   17033216 bytes,   17033216 total
45
# - age   2:   22458216 bytes,   39491432 total
46
# - age   3:      14552 bytes,   39505984 total
47
# - age   4:     136648 bytes,   39642632 total
48
49
# 17 MB of temporary space was recently allocated; just under 40 MB of temporaries are "surviving"
50
# and not immediately useless. (This comes from startup; almost all of that will be valid for the life
51
# of the program run.)
52
53
# Age 4 overflows (age 4's total larger than the max, or worse, "new threshold 2" or other small number)
54
# means the potential for tenured space to grow unnecessarily. This kills long-term performance.
55
56
## V1.5-corrected: 64 bit version, and ** LOTS OF MEMORY ALLOCATION **
57
# This is to give information for data collection.
58
# Determine how much your system uses, and then adjust numbers down to avoid waste
59
#
60
# Specifically: By making the memory pools large, we see how much is used.
61
# Then, we can determine what to cut them down to.
62
#
63
# This will probably be the last CMS version; G1GC next.
64
65-
#JVM_SIZE="-d32 -server"
65+
66-
JVM_SIZE="-d64 -XX:+UseCompressedOops -server"
66+
67
## V1.4: Java is now customizable (see the above line), thank you oracle for a
68
## java (1.7) that does not support 32 bit servers for reduced memory consumption. 
69
70
## V1.3: The -XX:SoftRefLRUPolicyMSPerMB=0 flag got lost! Back in now.
71
## This flag defaults to 1000, and can cause memory leaks.
72
73
## V1.2: We now play with -XX:TargetSurvivorRatio=n to reduce waste in new, permitting more
74
## space to be used
75
76
# Configurables:
77
# -d32 is for heap size up to 2.5gb. (NB: apparently only 1.5 gb on microsoft windows?)
78
# Change to "-d64 XX:+UseCompressedOops" if you use more.
79
# ** Mention that flag specifically, do not rely on it being autoset.
80
# ** Known and documented JVM bug -- https://forums.oracle.com/forums/thread.jspa?messageID=10017916
81
82
JVM_SIZE="-d32 -server"
83
# JVM_SIZE="-d64 -XX:+UseCompressedOops -server"
84
85
# CMSInitiatingOccupancyFraction: Determine how frequently to do a full CMS
86
# Special: New "Most important". This primarily affects long-term growth
87
# of the heap. The percentage of used space before CMS starts a collection.
88
# 95 is sufficient for general stuff. 85 is useful for things that suddenly
89
# need a lot.
90
# Dynamic maps, in particular, no longer needs 125 MB of survivor -- it can
91
# get by with 60-75. It can go much lower, but then the garbage collections
92
# need to be started sooner, or else it will never have enough memory and
93
# always grow the heap.
94
#
95
# To clarify: This is obsolete -- completely -- in G1GC. 
96
# This needs to be low enough that sudden spurts of temporary memory trigger
97
# garbage collection first. 
98
# This should be re-worked as a "MB safety level" -- for example, if you have
99
# 300 MB of tenured, and want at least 30 MB free. But Java don't work that way.
100
# As tenured increases, this will also increase the "keep free" level.
101-
MAX=3000	
101+
102
103
# Memory tuning:
104
# Command line controls total heap, and "new". "Tenured" is the difference.
105
# Bigger "new": Less frequent collections.
106
107
# These numbers are in "Megabytes", the java "m" suffix.
108
109
# The rule of memory tuning:
110
# SurvivorSpace * (SurvivorRatio + 2) = New
111-
Tenured=450
111+
112
# SurvivorSpace * SurvivorRatio = Eden.
113
# Two additional survivor spaces are used to copy surviving objects across minor collections.
114
115
# MAX: Maximum heap space used.
116-
# Dynamic maps wants this at least 100, preferrably 125.
116+
117
# Does not include JVM overhead
118
MAX=1300
119
120
# Tenured: Desired long-term storage space
121
# Will vary based on mods, and "loaded chunks"
122
# -- how many parties of players close to each other.
123
#
124-
## SurvivorCopySize=26
124+
125-
## 26 is too small -- memory consumption keeps going up with dynamic maps
125+
126
#
127-
# To clarify: You can easily use 12 here if you do not use dynamic maps.
127+
128
Tenured=500
129-
# You can use less if you use lowres maps.
129+
130-
# The big memory consumer is low-angle maps; the 30 degree maps of high res,
130+
131-
# or the 20 degree maps I use on my server are the big memory consumers.
131+
132
# During startup, expect this to overflow frequently.
133
# Actual space allocated is 2 spaces, each one twice this size.
134
# "waste/overhead" will be two about to three times this number.
135
# *** Maximum of 1/6rd of "new" 
136
# Pay attention to the tenuring distribution logs. 
137
# *** This should be enough for generation 3 95%+ of the time. ***
138
# ** TOO SMALL WILL KILL YOUR GARBAGE COLLECTION **
139
# (worse, it will cause tenured space to grow.)
140
# ** TOO BIG WILL WASTE SPACE **
141
# (this is a minor issue overall.)
142
#
143-
# Even in Mystcraft, with massive amounts of decay everywhere, 95% of the time 1 meg suffices.
143+
# To clarify: You can easily use 12 here if you stay primarily in one place.
144
# You can even use less, but the memory savings are not significant.
145
# Biggest usage I've seen: dimension travel (see below) and dynamic maps with
146
# low angle (20 degree) maps.
147
#
148
# *** Recommendations
149-
# should be less than this 95% of the time.
149+
# If you do a lot of dimension travel -- including using a nether hub to get
150
# around, or lots of Mystcraft books (two links to go from A to B in the same
151
# world), then you want 65 MB if you can afford it, or 53 MB minimum. That's
152
# for one person; groups will be higher.
153
# (tested in a world with forests at the hub zone; may be higher for extreme hills,
154-
# and the speed at which they move. Adjust to your server, and your mods.
154+
# jungles, or jungle-hills (expected to be the worst case.))
155
#
156-
# Single player won't need that much. Really.
156+
# If you do a lot of high speed rail travel over large distances, a minimum of
157
# 40 MB. Ditto if you use a lot of flight mode movement. More if you have a
158
# speed boost.
159
#
160
# If you do very little travel -- if you are just building in one spot -- then
161-
# Must be at least 2x SurvivorCopySize. Java requires it to be
161+
# 15 MB is plenty.
162-
# an integer multiple of that value.
162+
163
# If you are testing a new mod pack, and have no clue what you need, start at 100.
164
# If you are working with a multiplayer server, I don't know what to advise --
165
# test and tell me.
166
167-
# plus 100, will be used by java to start the heap. Up to a max of MAX. 
167+
# SurvivorCopySize=65
168
SurvivorCopySize=100
169
170
# Survivor target ratio. Java defaults to 50%, which wastes a lot of space. If you know how much
171
# you need (see below), you can set this value higher; this gives less waste and "better performance".
172
173
TargetSurvivorRatio=90
174
175
## Notes on "SurvivorCopySize":
176
# Flying around in creative mode, in already generated chunks will want
177
#   at least 30-35, preferrably 40 meg.
178
# Standing around, single player, can be happy with less than 1.
179
# Even in Mystcraft, with massive amounts of decay everywhere (implies lots of block
180
# updates), 95% of the time 1 meg suffices.
181
# Moving around a little, doing basic building/digging, about 3.
182
#
183
# The rule: You want to see "new threshold 4 (max 4)" most of the time.
184
# The total value at age three -- 
185
# - age   3:      36712 bytes,    5897520 total
186
# should be less than this 98% of the time.
187
# 12 meg is more than enough for one person with EBXL, Mystcraft, Twilight Forest,
188
# and Custom Ore Gen. Even in EBXL's extreme jungle with Mystcraft's decay littering the ground.
189
#
190
# The single biggest factor is chunks loaded; that will depend more on parties than on players,
191
# and the speed at which they move. Adjust to your server, and your mods. Remember, teleportation
192
# will raise this value very rapidly.
193
#
194
# To clarify the problem: An overflow here forces memory into tenured space. If tenured is
195
# nearly full -- lets say Java will start a garbage collection soon -- then this can force
196
# more space to be allocated to tenured.
197
#
198
# Forcing more space into tenured is the disaster that this is all about preventing.
199
# Tenured space is more expensive to collect, and Java does not release this back to the
200
# operating system when it does not need it. This means, on systems that are doing more than
201
# just a minecraft server (say, a client, a browser, perhaps voice chat software, etc), this
202
# will impact your whole system's performance.
203
#
204
# A secondary effect of overflow is forcing earlier/more frequent full collections.
205
# For single/dual processors, this may be the more dominant factor.
206
# For 4/8 core processors, this is minor.
207
208
# Second most important tuning. Eden.
209
# Making this bigger means less frequent small collections.
210
# General rule: Make this as big as your memory can handle.
211
# Must be at least 2x Survivor. Java requires it to be
212
# an integer multiple of Survivor.
213
214
desiredEden=250
215
216
# Summary: Approximately desiredEden, plus 2 times Survivor,
217
# plus 130, will be used by java to start the heap. Up to a max of MAX. 
218
# Script will attempt to ensure at least Tenured space exist;
219
# should exit with a message if it cannot.
220
#
221
# In theory, Java will allocate extra space to new or tenured as needed.
222
# In practice, I've never seen it increase "new".
223
#
224
# See the bottom of the config section for more.
225
226
# If your shell cannot do math, replace these with an appropriate constant
227
228
MaxNew=$(($MAX - $Tenured))
229
230
## Survivor=$((2 * $SurvivorCopySize))
231
## Working with survivor target. "2" is for 50%. For 90%, it's much closer to 1.
232
## What we want is 100 / target percentage, as the ratio instead of 2.
233
## For integer only shell math, we re-write as (100 * survivor) / target, which gives us
234
## close integer to the desired result -- as close as we can get in the shell.
235
236
Survivor=$(( ($SurvivorCopySize * 100 ) / $TargetSurvivorRatio ))
237
238
## Equally, the "3" in sanity test is from 3 bins -- two survivors, one eden.
239
## But that does NOT change here -- it's still the sanity test lower limit.
240
241
sanityTest=$((3 * $Survivor))
242-
# New space is small -- specified eden.
242+
243
then
244
	echo Memory config error >& 2
245
	exit 1
246-
# Tenured is small -- specified tenured space.
246+
247-
## Should Not Be Needed -- "NewSize" and "MaxNewSize" specified separately.
247+
248-
# In theory, Java should now adjust new as neeed.
248+
249-
#NEW=$biggerNew
249+
250-
#RATIO=$MaxRatio
250+
251
# 1. Find the multiple of Survivor that is bigger than S and less than MN.
252
# 2. Determine survivor ratio from that. Subtract 2 (java.)
253
# 3. Specify -Xmn for new, and survivor ratio, to set eden and new.
254
255
# "New" will be Eden plus 2* Survivor.
256
257
# MaxRatio -- what the ratio is if we use all of maxnew.
258
MaxRatio=$(( ($MaxNew / $Survivor) - 2 ))
259
# DesiredRatio -- what the ratio is based on declared eden space
260
# There is no "-2" here -- this will allocate eden plus 2* survivor.
261
desiredRatio=$(( ($desiredEden / $Survivor)  ))
262
263
# SurvivorSpace * (SurvivorRatio + 2) = New
264
265
# Now check for "desired Eden". If survivor is not an exact multiple of DE,
266
# then we have just rounded down. Test for this, and if so, see if we can
267
# raise it up (watch out for maxnew)
268
269
## TODO! FIXME! This is a cheap approximation
270
if ( [ $(( $desiredRatio + 1 )) -le $MaxRatio ] )
271
then	desiredRatio=$(( $desiredRatio + 1 ))
272
fi
273
274
desiredNew=$(($Survivor * ($desiredRatio + 2) ))
275
biggerNew=$(($Survivor * ($MaxRatio + 2) ))
276
277
echo Debug: Max ratio $MaxRatio, desiredRatio $desiredRatio
278
echo Debug: biggerNew $biggerNew, should be less than MaxNew $MaxNew
279
echo Debug: desired eden $desiredEden, survivor $Survivor, actual new $desiredNew
280
281
# desiredNew: Gives an eden up to, not bigger, than desiredEden.
282
# biggerNew: Gives an eden at least as big as desiredEden.
283
# FIXME: DesiredNew / ratio should be smallest at least as big as desiredEden
284-
	-jar new_server.jar nogui 147test
284+
285
#
286-
# The last word of that exec statement -- '147test' -- is just something that shows up in
286+
287
288
# Q: Desired numbers? Bigger/Max numbers?
289
290
# Choose one of these pairs
291
292
# New space is sufficient -- specified eden.
293
NEW=$desiredNew
294
RATIO=$desiredRatio
295
296
# borderline case: Not quite the desired eden, we ran into maxnew.
297
# Disable these; we'd rather raise maxNew for Minecraft.
298
## NEW=$biggerNew
299
## RATIO=$MaxRatio
300
301
### 1.7 bug fix! Turns out that borderline cases could fail.
302
if ( [ $NEW -gt $MaxNew ] )
303
then
304
	MaxNew=$NEW
305
	## CAUTION! "Start" below may fail.
306
	## If you get here, then you need to watch out for tenured overflow
307
	echo "** Launcher adjusted MaxNew; make sure tenured space is big enough"
308
fi
309
310
START=$(($NEW + 130))
311
# Experimal behavior: Starting up, and loading one person into a world, takes
312
# 130 MB of tenured space to have Java "happy". Less just results in repeated
313
# full collections while Java raises the amount of allocated memory. 
314
315
## TESTME: Does "MaxNewSize" matter if we have adaptive GC boundary? Does it hurt?
316
317
# A few more notes ...
318
319
# -XX:+UseAdaptiveGCBoundary -- apparently, adjust the boundary between new and tenured as needed.
320
# Nice to see; did not know about it before. 
321
# Sadly, it seems to have no effect.
322
323
# -XX:+CMSIncrementalMode: Tells the garbage collector to break the job into many small parts.
324
# May result in better performance. Essential on systems with few cores.
325
326
# -XX:CompileThreshold: Tells Java when to convert bytecode into compiled code.
327
# Server defaults to something like 15,000, which is appropriate for something that will
328
# run for months, and horrible for something that will run for a few hours.
329
# Non-server defaults to ... 1000? 1500? Much lower.
330
331
exec $java \
332
	$JVM_SIZE \
333
	-Xms${START}m -Xmx${MAX}m \
334
	-XX:NewSize=${NEW}m -XX:MaxNewSize=${MaxNew}m \
335
	-XX:+UseAdaptiveGCBoundary \
336
	-XX:SurvivorRatio=$RATIO \
337
	-XX:TargetSurvivorRatio=$TargetSurvivorRatio \
338
	-XX:CompileThreshold=3000 \
339
	-XX:CMSInitiatingOccupancyFraction=$CMSInitiatingOccupancyFraction \
340
\
341
	-XX:SoftRefLRUPolicyMSPerMB=0 \
342
	-XX:MaxPermSize=150m \
343
	-XX:+UseConcMarkSweepGC -XX:+UseParNewGC \
344
	-XX:MaxHeapFreeRatio=20 \
345
	-XX:MinHeapFreeRatio=15 \
346
	-XX:+DisableExplicitGC \
347
	-XX:MaxTenuringThreshold=4 \
348
	-XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution \
349
	-XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log \
350
	-jar new_server.jar nogui 151myst10.3
351
352
# The last word of that exec statement -- '151myst10.3' -- is just something that shows up in
353
# the process list, so I can tell which process is which server (each copy of this script
354
# has a different name in that field).