View difference between Paste ID: duLuWGJJ and 9czuNyPP
SHOW: | | - or go back to the newest paste.
1
#!/bin/bash 
2
# This line is for starting from mac os icon double click
3
cd "$( dirname "$0" )" 
4
5
## V1.6. This time the 64 bit flag works.
6
# Again: This is throwing "too much" memory at eden/survivor.
7
# Pay attention to the GC log. Sections like
8
9
# 1552.959: [CMS-concurrent-reset-start]
10
# 1552.965: [CMS-concurrent-reset: 0.005/0.005 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
11
# {Heap before GC invocations=17 (full 1):
12
#  par new generation   total 180224K, used 146721K [0000000009000000, 0000000016c00000, 0000000025200000)
13
#   eden space 135168K, 100% used [0000000009000000, 0000000011400000, 0000000011400000)
14
#   from space 45056K,  25% used [0000000014000000, 0000000014b48750, 0000000016c00000)
15
#   to   space 45056K,   0% used [0000000011400000, 0000000011400000, 0000000014000000)
16
#  concurrent mark-sweep generation total 102400K, used 63696K [0000000025200000, 000000002b600000, 000000003b000000)
17
#  concurrent-mark-sweep perm gen total 24940K, used 21197K [000000003b000000, 000000003c85b000, 0000000044600000)
18
19
# The "CMS-concurrent-reset" means it has finished a full collection.
20
# The "concurrent mark-sweep generation" line has the tenured allocation
21
# In this case, 63 MB of long-term data.
22
23
# Sections like
24
25
# 56.089: [GC 56.089: [ParNew
26
# Desired survivor size 41523608 bytes, new threshold 4 (max 4)
27
# - age   1:   17033216 bytes,   17033216 total
28
# - age   2:   22458216 bytes,   39491432 total
29
# - age   3:      14552 bytes,   39505984 total
30
# - age   4:     136648 bytes,   39642632 total
31
32
# 17 MB of temporary space was recently allocated; just under 40 MB of temporaries are "surviving"
33
# and not immediately useless. (This comes from startup; almost all of that will be valid for the life
34
# of the program run.)
35
36
# Age 4 overflows (age 4's total larger than the max, or worse, "new threshold 2" or other small number)
37
# means the potential for tenured space to grow unnecessarily. This kills long-term performance.
38
39
## V1.5-corrected: 64 bit version, and ** LOTS OF MEMORY ALLOCATION **
40
# This is to give information for data collection.
41
# Determine how much your system uses, and then adjust numbers down to avoid waste
42
#
43
# Specifically: By making the memory pools large, we see how much is used.
44
# Then, we can determine what to cut them down to.
45
#
46
# This will probably be the last CMS version; G1GC next.
47
48
java=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Commands/java
49
50
## V1.4: Java is now customizable (see the above line), thank you oracle for a
51
## java (1.7) that does not support 32 bit servers for reduced memory consumption. 
52
53
## V1.3: The -XX:SoftRefLRUPolicyMSPerMB=0 flag got lost! Back in now.
54
## This flag defaults to 1000, and can cause memory leaks.
55
56
## V1.2: We now play with -XX:TargetSurvivorRatio=n to reduce waste in new, permitting more
57
## space to be used
58
59
# Configurables:
60
# -d32 is for heap size up to 2.5gb. (NB: apparently only 1.5 gb on microsoft windows?)
61
# Change to "-d64 XX:+UseCompressedOops" if you use more.
62
# ** Mention that flag specifically, do not rely on it being autoset.
63
# ** Known and documented JVM bug -- https://forums.oracle.com/forums/thread.jspa?messageID=10017916
64
65
#JVM_SIZE="-d32 -server"
66
JVM_SIZE="-d64 -XX:+UseCompressedOops -server"
67
68
# CMSInitiatingOccupancyFraction: Determine how frequently to do a full CMS
69
# Special: New "Most important". This primarily affects long-term growth
70
# of the heap. The percentage of used space before CMS starts a collection.
71
# 95 is sufficient for general stuff. 85 is useful for things that suddenly
72
# need a lot.
73
# Dynamic maps, in particular, no longer needs 125 MB of survivor -- it can
74
# get by with 60-75. It can go much lower, but then the garbage collections
75
# need to be started sooner, or else it will never have enough memory and
76
# always grow the heap.
77
#
78
# To clarify: This is obsolete -- completely -- in G1GC. 
79
# This needs to be low enough that sudden spurts of temporary memory trigger
80
# garbage collection first. 
81
# This should be re-worked as a "MB safety level" -- for example, if you have
82
# 300 MB of tenured, and want at least 30 MB free. But Java don't work that way.
83
# As tenured increases, this will also increase the "keep free" level.
84
CMSInitiatingOccupancyFraction=80
85
86
# Memory tuning:
87
# Command line controls total heap, and "new". "Tenured" is the difference.
88
# Bigger "new": Less frequent collections.
89
90
# These numbers are in "Megabytes", the java "m" suffix.
91
92
# The rule of memory tuning:
93
# SurvivorSpace * (SurvivorRatio + 2) = New
94
# ("SurvivorSpace" is twice the actual surviving threshold.)
95
# SurvivorSpace * SurvivorRatio = Eden.
96
# Two additional survivor spaces are used to copy surviving objects across minor collections.
97
98
# MAX: Maximum heap space used.
99
# Does not include permanent (byte/compiled code)
100
# Does not include JVM overhead
101
MAX=3000	
102
103
# Tenured: Desired long-term storage space
104
# Will vary based on mods, and "loaded chunks"
105
# -- how many parties of players close to each other.
106
#
107
# Starting assumption: 250 for 1 person, plus 50 per group
108
# of players near each other. 
109
#
110
# That is a guess. Please report what numbers work for your server.
111
Tenured=450
112
113
# Most important tuning number. Survivor.
114
# Making this higher: Fewer full collections, but more wasted space.
115
# During startup, expect this to overflow frequently.
116
# Dynamic maps wants this at least 100, preferrably 125.
117
# Actual space allocated is 2 spaces, each one twice this size.
118
# "waste/overhead" will be two about to three times this number.
119
# *** Maximum of 1/6rd of "new" 
120
# Pay attention to the tenuring distribution logs. 
121
# *** This should be enough for generation 3 95%+ of the time. ***
122
# ** TOO SMALL WILL KILL YOUR GARBAGE COLLECTION **
123
# ** TOO BIG WILL WASTE SPACE **
124
## SurvivorCopySize=26
125
## 26 is too small -- memory consumption keeps going up with dynamic maps
126
#
127
# To clarify: You can easily use 12 here if you do not use dynamic maps.
128
# You can even use less, but the memory savings are not significant.
129
# You can use less if you use lowres maps.
130
# The big memory consumer is low-angle maps; the 30 degree maps of high res,
131
# or the 20 degree maps I use on my server are the big memory consumers.
132
SurvivorCopySize=100
133
134
# Survivor target ratio. Java defaults to 50%, which wastes a lot of space. If you know how much
135
# you need (see below), you can set this value higher; this gives less waste and "better performance".
136
137
TargetSurvivorRatio=90
138
139
## Notes on "SurvivorCopySize":
140
# Flying around in creative mode, in already generated chunks will want
141
#   at least 30-35, preferrably 40 meg.
142
# Standing around, single player, can be happy with less than 1.
143
# Even in Mystcraft, with massive amounts of decay everywhere, 95% of the time 1 meg suffices.
144
# Moving around a little, doing basic building/digging, about 3.
145
#
146
# The rule: You want to see "new threshold 4 (max 4)" most of the time.
147
# The total value at age three -- 
148
# - age   3:      36712 bytes,    5897520 total
149
# should be less than this 95% of the time.
150
# 12 meg is more than enough for one person with EBXL, Mystcraft, Twilight Forest,
151
# and Custom Ore Gen. Even in EBXL's extreme jungle with Mystcraft's decay littering the ground.
152
#
153
# The single biggest factor is chunks loaded; that will depend more on parties than on players,
154
# and the speed at which they move. Adjust to your server, and your mods.
155
#
156
# Single player won't need that much. Really.
157
158
# Second most important tuning. Eden.
159
# Making this bigger means less frequent small collections.
160
# General rule: Make this as big as your memory can handle.
161
# Must be at least 2x SurvivorCopySize. Java requires it to be
162
# an integer multiple of that value.
163
164
desiredEden=250
165
166
# Summary: Approximately desiredEden, plus 2 times Survivor,
167
# plus 100, will be used by java to start the heap. Up to a max of MAX. 
168
# Script will attempt to ensure at least Tenured space exist;
169
# should exit with a message if it cannot.
170
#
171
# In theory, Java will allocate extra space to new or tenured as needed.
172
# In practice, I've never seen it increase "new".
173
#
174
# See the bottom of the config section for more.
175
176
# If your shell cannot do math, replace these with an appropriate constant
177
178
MaxNew=$(($MAX - $Tenured))
179
180
## Survivor=$((2 * $SurvivorCopySize))
181
## Working with survivor target. "2" is for 50%. For 90%, it's much closer to 1.
182
## What we want is 100 / target percentage, as the ratio instead of 2.
183
## For integer only shell math, we re-write as (100 * survivor) / target, which gives us
184
## close integer to the desired result -- as close as we can get in the shell.
185
186
Survivor=$(( ($SurvivorCopySize * 100 ) / $TargetSurvivorRatio ))
187
188
## Equally, the "3" in sanity test is from 3 bins -- two survivors, one eden.
189
## But that does NOT change here -- it's still the sanity test lower limit.
190
191
sanityTest=$((3 * $Survivor))
192
if [ $sanityTest -gt $MaxNew ]
193
then
194
	echo Memory config error >& 2
195
	exit 1
196
fi
197
198
# We cannot use more than MaxNew.
199
200
# The idea:
201
# 1. Find the multiple of Survivor that is bigger than S and less than MN.
202
# 2. Determine survivor ratio from that. Subtract 2 (java.)
203
# 3. Specify -Xmn for new, and survivor ratio, to set eden and new.
204
205
# "New" will be Eden plus 2* Survivor.
206
207
# MaxRatio -- what the ratio is if we use all of maxnew.
208
MaxRatio=$(( ($MaxNew / $Survivor) - 2 ))
209
# DesiredRatio -- what the ratio is based on declared eden space
210
# There is no "-2" here -- this will allocate eden plus 2* survivor.
211
desiredRatio=$(( ($desiredEden / $Survivor)  ))
212
213
# SurvivorSpace * (SurvivorRatio + 2) = New
214
215
# Now check for "desired Eden". If survivor is not an exact multiple of DE,
216
# then we have just rounded down. Test for this, and if so, see if we can
217
# raise it up (watch out for maxnew)
218
219
## TODO! FIXME! This is a cheap approximation
220
if ( [ $(( $desiredRatio + 1 )) -le $MaxRatio ] )
221
then	desiredRatio=$(( $desiredRatio + 1 ))
222
fi
223
224
desiredNew=$(($Survivor * ($desiredRatio + 2) ))
225
biggerNew=$(($Survivor * ($MaxRatio + 2) ))
226
227
echo Debug: Max ratio $MaxRatio, desiredRatio $desiredRatio
228
echo Debug: biggerNew $biggerNew, should be less than MaxNew $MaxNew
229
echo Debug: desired eden $desiredEden, survivor $Survivor, actual new $desiredNew
230
231
# desiredNew: Gives an eden up to, not bigger, than desiredEden.
232-
	-d32 -server \
232+
233
# FIXME: DesiredNew / ratio should be smallest at least as big as desiredEden
234
# This means, if less, then add 1 to ratio and add to new.
235
#
236
# "Bigger" assigns ALL non-tenured memory to new.
237
238
# Q: Desired numbers? Bigger/Max numbers?
239-
        -XX:CMSInitiatingOccupancyFraction=$CMSInitiatingOccupancyFraction \
239+
240
# Choose one of these pairs
241
242-
        -XX:MaxPermSize=150m \
242+
243
NEW=$desiredNew
244
RATIO=$desiredRatio
245
246
# Tenured is small -- specified tenured space.
247
## Should Not Be Needed -- "NewSize" and "MaxNewSize" specified separately.
248
# In theory, Java should now adjust new as neeed.
249-
	-jar new_server.jar nogui 147MystTest
249+
250
#RATIO=$MaxRatio
251
252
START=$(($NEW + 130))
253
254
## TESTME: Does "MaxNewSize" matter if we have adaptive GC boundary? Does it hurt?
255
256
# A few more notes ...
257
258
# -XX:+UseAdaptiveGCBoundary -- apparently, adjust the boundary between new and tenured as needed.
259
# Nice to see; did not know about it before. 
260
# Sadly, it seems to have no effect.
261
262
# -XX:+CMSIncrementalMode: Tells the garbage collector to break the job into many small parts.
263
# May result in better performance. Essential on systems with few cores.
264
265
exec $java \
266
	$JVM_SIZE \
267
	-Xms${START}m -Xmx${MAX}m \
268
	-XX:NewSize=${NEW}m -XX:MaxNewSize=${MaxNew}m \
269
	-XX:+UseAdaptiveGCBoundary \
270
	-XX:SurvivorRatio=$RATIO \
271
	-XX:TargetSurvivorRatio=$TargetSurvivorRatio \
272
	-XX:CompileThreshold=3000 \
273
	-XX:CMSInitiatingOccupancyFraction=$CMSInitiatingOccupancyFraction \
274
\
275
	-XX:SoftRefLRUPolicyMSPerMB=0 \
276
	-XX:MaxPermSize=150m \
277
	-XX:+UseConcMarkSweepGC -XX:+UseParNewGC \
278
	-XX:MaxHeapFreeRatio=20 \
279
	-XX:MinHeapFreeRatio=15 \
280
	-XX:+DisableExplicitGC \
281
	-XX:MaxTenuringThreshold=4 \
282
	-XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution \
283
	-XX:+PrintCommandLineFlags -XX:+PrintGCDetails -Xloggc:GC.log \
284
	-jar new_server.jar nogui 147test
285
286
# The last word of that exec statement -- '147test' -- is just something that shows up in
287
# the process list, so I can tell which process is which server (each copy of this script
288
# has a different name in that field).