API tools faq
paste
Advertisement
Guest User

Roentgen

a guest
Feb 1st, 2010
407
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 15.81 KB | None | 0 0
raw download clone embed print report
  1. #!/bin/bash
  2. VER=1.6.1
  3. #--[ Intro ]--------------------------------------#
  4. # #
  5. # Tur-DirlogClean. For a nice and updated search #
  6. # function in glftpd. Keep the dirlog up to date! #
  7. # #
  8. # Note: 'site dupe' is not affected by this and #
  9. # shouldnt be. 'site dupe' is for what WAS #
  10. # on site and 'site search' is for what IS! #
  11. # #
  12. # All this script really does, is #
  13. # 1: Run glupdate on every section you define, #
  14. # updating with what is there now. Works for #
  15. # dated dirs too. #
  16. # #
  17. # 2 Run olddirclean2, thus removing all releases #
  18. # that is not on the site anymore / sorts the #
  19. # file. #
  20. # #
  21. #--[ Installation ]-------------------------------#
  22. # #
  23. # Copy this script to where ever (/glftpd/bin). #
  24. # Make it executable (chmod 700 tur-dirclean.sh). #
  25. #- #
  26. # If you are running glftpd 1.32, you need to #
  27. # verify your version of 'glupdate'. If you have #
  28. # version 2.2, you'll need to compile the 2.2.1 #
  29. # version included. 2.2 has a bug which wont let #
  30. # you specify location of glftpd.conf. #
  31. # See changelog below for version 1.5 on how to #
  32. # compile it. #
  33. # #
  34. # If you are using glftpd 2.0, you need the fixed #
  35. # glupdate binary found in the forums at #
  36. # www.glftpd.com (sticky post). #
  37. # #
  38. # If you are running 2.01+, you do not need to #
  39. # worry about glupdate. #
  40. #- #
  41. # When it comes to olddirclean2, its fine too, #
  42. # however, if you have lots of dirs on your site #
  43. # like a mp3 or 0day archive, you will want to #
  44. # modify it or it will crap out. Might as well do #
  45. # it now which we're messing with it anyway. #
  46. # Edit /glftpd/bin/sources/olddirclean2.c and #
  47. # find: #define MAXDIRLOGSIZE 10000 #
  48. # Change that number to 150000 or something, save #
  49. # it, then recompile it: #
  50. # gcc olddirclean2.c -o /glftpd/bin/olddirclean2 #
  51. #- #
  52. # Edit the settings: #
  53. # #
  54. # glconf= Full path to your glftpd.conf file. #
  55. # This will be added automatically to #
  56. # cleanold and glupdate below so #
  57. # dont specify it on those settings. #
  58. # #
  59. # cleanold= What to run to clean out old folders #
  60. # that are not there anymore. #
  61. # #
  62. # cleanlog= Where to log to. Only one day will #
  63. # be logged at a time (no point to log #
  64. # more stuff). #
  65. # #
  66. # glupdate= Where is glupdate? As with #
  67. # olddirclean2, this comes with glftpd #
  68. # and should be working from shell. #
  69. # #
  70. # dirlog= "/glftpd/ftp-data/logs/dirlog" #
  71. # The path to the dirlog file itself. #
  72. # #
  73. # keepchmod= What to chmod 'dirlog' above with? #
  74. # olddirclean2 sets it to 644, so we #
  75. # might need to chmod it so that any #
  76. # pre script or whatever can write to #
  77. # it. "" = Disable. #
  78. # #
  79. # glroot= Specify where /site is. This is just #
  80. # so you can use $glroot/<section> in #
  81. # sections below. For lazy people. #
  82. # #
  83. # sections= Specify each section on your site #
  84. # that you want to search for releases #
  85. # in. #
  86. # Add :DEEP to the end to dive one #
  87. # dir into that (for dated dirs). #
  88. # #
  89. # Add :2xDEEP to the end to dive two #
  90. # dirs down. #
  91. # #
  92. # Finally, :3xDEEP is also accepted to #
  93. # jump 3 levels down the dir structure.#
  94. # #
  95. # Example: Say you have the following #
  96. # path: #
  97. # $glroot/Archive/0DAYS/0DAYS1/2004/0102 #
  98. # Now, instead of adding 0101 / 0102, #
  99. # etc, you can do: #
  100. # $glroot/Archive/0DAYS/0DAYS1/2004:DEEP #
  101. # Or: #
  102. # $glroot/Archive/0DAYS/0DAYS1:2xDEEP #
  103. # Or: #
  104. # $glroot/Archive/0DAYS:3xDEEP #
  105. # #
  106. # If you set this to "", it will only #
  107. # remove dir from dirlog that isnt #
  108. # there anymore ( from all sections ). #
  109. # #
  110. # If you have a space in your sections #
  111. # then use [:space:] #
  112. # #
  113. # exclude= This is only if you specify :DEEP #
  114. # 2xDEEP or 3xDEEP. #
  115. # What NOT to add to dirlog? Pre #
  116. # folders, !today symlinks, .message #
  117. # file, etc etc. Anything that can be #
  118. # in the dirs, but does not belong #
  119. # in the dirlog. #
  120. # #
  121. # Note, this is a simple egrep -vi #
  122. # line, so if you specify, for example #
  123. # PRE, it will skip any dir containing #
  124. # that word. To specify only the dir #
  125. # use ^PRE$ and it will only match on #
  126. # the dir thats actually named PRE. #
  127. # #
  128. #--[ Running it ]---------------------------------#
  129. # #
  130. # You may run this script with the argument #
  131. # 'debug' and it will show you what it is doing. #
  132. # #
  133. # Try it by adding a dir from shell in any of #
  134. # the defined sections. Then search for it with #
  135. # site search and make sure it does not find it. #
  136. # Run this script and try and search for it again.#
  137. # #
  138. # Second try: Remove a dir from site. Search for #
  139. # it. It should be found still. Now run this #
  140. # script again and it should be gone when #
  141. # searching. #
  142. # #
  143. # Crontab this script to run whenever you like. #
  144. # like: #
  145. # 1 1 * * * /glftpd/bin/tur-dirlogclean.sh #
  146. # to run it 01:01AM each day. #
  147. # #
  148. #-[ Contact ]-------------------------------------#
  149. # WEB: http://www.grandis.nu #
  150. # http://grandis.mine.nu #
  151. # #
  152. #-[ Changelog ]-----------------------------------####
  153. # #
  154. # 1.6.1: Chg: Changed the help text for how to #
  155. # glupdate and olddirclean2. This should #
  156. # help those new to this script. #
  157. # #
  158. # 1.6 : Add: You may now specify :2xDEEP in sections#
  159. # to dive 2 dirs into the specifed #
  160. # section (whereas :DEEP is only 1 dir) #
  161. # #
  162. # 1.5 : Chg: Reversed order. It will now first run #
  163. # glupdate and after that, olddirclean2. #
  164. # This was made because olddirlogclean2 #
  165. # also sorts the dirlog, making it #
  166. # faster to search in (I guess). #
  167. # This was done by advice from neewbiee. #
  168. # #
  169. # Add: Included a glupdate.c file. If you #
  170. # currently run glupdate v2.2, it wont #
  171. # work since you can specify path to #
  172. # glftpd.conf (cause of a bug). #
  173. # If you do, compile glupdate 2.2.1 #
  174. # instead: #
  175. # gcc -o /glftpd/bin/glupdate glupdate.c #
  176. # #
  177. # This is NOT an official update since I #
  178. # did it myself with the help from #
  179. # www.glftpd.com. #
  180. # #
  181. # 1.4 : Add: You can now use [:space:] in sections #
  182. # incase you have spaces in them. #
  183. # #
  184. # Add: For all you lazy people, added #
  185. # glroot option to specify where /site #
  186. # is, then you can use $glroot/section #
  187. # in sections. #
  188. # #
  189. # 1.3 : Fix: Some people reporting having problems #
  190. # with the '-r $glconf' usage. Changed #
  191. # it to -r${glconf} instead. #
  192. # Thanks |_SL_| #
  193. # #
  194. # 1.2 : Add: With 'debug' it now shows where it is. #
  195. # #
  196. # Add: Added two new settings. dirlog and #
  197. # keepchmod. When running olddirclean2, #
  198. # changed the chmod on the file to 644. #
  199. # With keepchmod set, it will chmod the #
  200. # file set in 'dirlog' to that value. #
  201. # Pre scripts might have had problems #
  202. # adding the rel to dirlog if it was 644.#
  203. # #
  204. # Add: Added checks to see if it can find/run #
  205. # the required binaries. #
  206. # #
  207. # Chg: Renamed from tur-dirclean.sh to #
  208. # tur-dirlogclean.sh. Dont forget to #
  209. # change your crontabs. #
  210. # Changed all 'awk' into 'cut'. #
  211. # #
  212. # 1.1 : Fix: Added -r options to proggs for those #
  213. # that do not have glftpd.conf in /etc #
  214. # #
  215. # 1.0. Initial release. ####
  216. # # ^
  217. #-[ Settings ]------------------------------------# |
  218. # Lazy
  219. glconf="/etc/glftpd.conf"
  220.  
  221. cleanold="/glftpd/bin/olddirclean2"
  222. cleanlog="/glftpd/ftp-data/logs/olddirclean.log"
  223.  
  224. glupdate="/glftpd/bin/glupdate"
  225.  
  226. dirlog="/glftpd/ftp-data/logs/dirlog"
  227. keepchmod=666
  228.  
  229. glroot="/glftpd/site"
  230.  
  231. sections="
  232. $glroot/pub/0day:DEEP
  233. "
  234.  
  235. exclude="^GROUPS$|^lost\+found$|^All$"
  236.  
  237.  
  238. #-[ Script Start ]--------------------------------#
  239.  
  240. if [ "$1" = "debug" ]; then
  241. echo "Verifying existance of required bins.."
  242. fi
  243.  
  244. if [ ! -x "$cleanold" ]; then
  245. echo "Error. Cant execute $cleanold. Check existance and perms."
  246. exit 1
  247. fi
  248. if [ ! -x "$glupdate" ]; then
  249. echo "Error. Cant execute $glupdate. Check existance and perms."
  250. exit 1
  251. fi
  252. if [ ! -r "$glconf" ]; then
  253. echo "Error. Cant read $glconf. Check existance and perms."
  254. exit 1
  255. fi
  256.  
  257. if [ "$1" = "debug" ]; then
  258. echo "Checking for new or moved files to add to db."
  259. fi
  260.  
  261. for section in $sections; do
  262. section="`echo "$section" | sed -e 's/\[\:space\:\]/ /g'`"
  263. dated="$( echo $section | cut -d ':' -f2 )"
  264. if [ "$dated" != "DEEP" ] && [ "$dated" != "2xDEEP" ] && [ "$dated" != "3xDEEP" ]; then
  265. if [ "$1" = "debug" ]; then
  266. echo "Entering \"$section\""
  267. fi
  268. $glupdate -r ${glconf} "$section" >> $cleanlog
  269. else
  270. section="$( echo $section | cut -d ':' -f1 )"
  271. if [ ! -d "$section" ]; then
  272. echo "Error. \"$section\" does not exist. Skipping."
  273. else
  274. cd "$section"
  275. LIST="$( ls -1 | egrep -vi "$exclude" )"
  276. for folder in $LIST; do
  277.  
  278. if [ "$dated" = "2xDEEP" ]; then
  279. LIST2="$( ls -1 $folder | egrep -vi "$exclude" )"
  280. for folder2 in $LIST2; do
  281. if [ ! -d "$section/$folder/$folder2" ]; then
  282. if [ "$1" = "debug" ]; then
  283. echo "Not a dir: $section/$folder/$folder2 - Skipping."
  284. fi
  285. else
  286. if [ "$1" = "debug" ]; then
  287. echo "Entering (2xDEEP) \"$section/$folder/$folder2\""
  288. fi
  289. $glupdate -r ${glconf} "$section/$folder/$folder2" >> $cleanlog
  290. fi
  291. done
  292.  
  293. elif [ "$dated" = "3xDEEP" ]; then
  294. elif [ "$dated" = "3xDEEP" ]; then
  295. LIST2="$( ls -1 $folder | egrep -vi "$exclude" )"
  296. for folder2 in $LIST2; do
  297. if [ ! -d "$section/$folder/$folder2" ]; then
  298. if [ "$1" = "debug" ]; then
  299. echo "Not a dir: $section/$folder/$folder2 - Skipping."
  300. fi
  301. else
  302. LIST3="$( ls -1 $folder/$folder2 | egrep -vi "$exclude" )"
  303. for folder3 in $LIST3; do
  304. if [ ! -d "$section/$folder/$folder2/$folder3" ]; then
  305. if [ "$1" = "debug" ]; then
  306. echo "Not a dir: $section/$folder/$folder2/$folder3 - Skipping."
  307. fi
  308. else
  309. if [ "$1" = "debug" ]; then
  310. echo "Entering (3xDEEP) \"$section/$folder/$folder2/$folder3\""
  311. fi
  312. $glupdate -r ${glconf} "$section/$folder/$folder2/$folder3" >> $cleanlog
  313. fi
  314. done
  315. fi
  316. done
  317.  
  318. else
  319. if [ ! -d "$section/$folder" ]; then
  320. if [ "$1" = "debug" ]; then
  321. echo "Not a dir: $section/$folder - Skipping."
  322. fi
  323. else
  324. if [ "$1" = "debug" ]; then
  325. echo "Entering (DEEP) \"$section/$folder\""
  326. fi
  327. $glupdate -r ${glconf} "$section/$folder" >> $cleanlog
  328. fi
  329. fi
  330.  
  331. done
  332. fi
  333. fi
  334. done
  335.  
  336. if [ "$keepchmod" ]; then
  337. if [ "$1" = "debug" ]; then
  338. echo "Running chmod $keepchmod $dirlog"
  339. fi
  340. chmod $keepchmod $dirlog
  341. fi
  342.  
  343. echo "" >> $cleanlog
  344.  
  345. ## Run olddircleaner.
  346. if [ "$1" = "debug" ]; then
  347. echo "Checking for files that does not exist anymore / sorting dirlog. This could take some time."
  348. fi
  349.  
  350. $cleanold -P -D -r${glconf} > $cleanlog
  351.  
  352. if [ "$keepchmod" ]; then
  353. if [ "$1" = "debug" ]; then
  354. echo "Running chmod $keepchmod $dirlog"
  355. fi
  356. chmod $keepchmod $dirlog
  357. fi
  358.  
  359. exit 0
  360.  
  361.  
  362.  
  363.  
  364.  
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!