Advertisement
Guest User

Untitled

a guest
Apr 17th, 2018
220
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.88 KB | None | 0 0
  1.  
  2. #------------------------------------------------------------------------------
  3. # NRT sif classes form the basis of customized DMSP data processing. Each
  4. # type of sif class my be inherited, configured, and specialized to allow any
  5. # manner of processing to be applied to DMSP data without concern for other
  6. # deails such as how the data triggers processing, etc. All that needs to be
  7. # done in a sif is to say how to process data and the NRT system takes care of
  8. # the rest. 9 times of of ten a sif file is just going to act as a
  9. # configuration file but, because it's an actual subclass, specialized
  10. # behaviour can be dynamically added and refined.
  11. #------------------------------------------------------------------------------
  12.  
  13.  
  14. #
  15. # we start out with a basic class definition
  16. #
  17.  
  18. class ExampleSif < NRT::Geotiffed
  19.  
  20. #------------------------------------------------------------------------------
  21. # valid subscription types, along with description, are:
  22. #
  23. # - NRT::OLSSubscription::Copied
  24. # any file is simply copied into the output space
  25. #
  26. # - NRT::OLSSubscription::CopiedIfCovered
  27. # any file is copied into the output space if, and only if, it covers
  28. # the configured roi
  29. #
  30. # - NRT::OLSSubscription::Subsetted
  31. # any file is cropped and copied into the output space if, and only if,
  32. # it covers the configured roi which is used as the crop specification.
  33. # any data over the roi is provided but the entire roi may or may not be
  34. # covered. if multiple non-continuous sections cover the roi then the
  35. # smallest crop that included all covering sections is used.
  36. #
  37. # - NRT::OLSSubscription::SubsettedMulitple
  38. # this is the same as Subsetted subscriptions with the exception that,
  39. # when multiple sections of data cover the roi they are delivered
  40. # individually rather than taking the minimum bounding rectangle which
  41. # contained all sections
  42. #
  43. # - NRT::OLSSubscription::Gridded
  44. # this is the same as Subsetted but, additionally, all sections will be
  45. # gridded according to the sif configuration. grids are delivered in
  46. # raw/envi format with appropriate header files
  47. #
  48. # - NRT::OLSSubscription::Geotiffed
  49. # the same as Gridded but each grid is also delivered in geotiff format
  50. #
  51. # - NRT::OLSSubscription::GriddedToRealData
  52. # the same as Gridded, but all output files have leading and trailing
  53. # artifical data removed
  54. #
  55. # - NRT::OLSSubscription::GriddedToRealDataAndGeotiffed
  56. # the same as Geotiffed, but all output files have leading and trailing
  57. # artifical data removed
  58. #------------------------------------------------------------------------------
  59.  
  60. #------------------------------------------------------------------------------
  61. # following are the most important and commonly used configuration options
  62. #------------------------------------------------------------------------------
  63.  
  64. #
  65. # mode: the only important mode is 'production', which allows a sif to
  66. # accept files for processing. any other mode, such as 'stopped',
  67. # effectively turns the sif off
  68. #
  69. mode 'production'
  70.  
  71. #
  72. # roi: a list/array of coordinates to specify the roi in UL/LR-lat/lon pairs
  73. #
  74. roi 10, -170, 5, -160
  75.  
  76. #
  77. # satellites: a list of strings denoting which satellites should be accepted
  78. # for processing
  79. #
  80. satellites %w( F14 F15 )
  81.  
  82. #
  83. # extensions: a list of strings denoting which extenstions should be
  84. # accepted for processing
  85. #
  86. extensions %w( OIS OIF )
  87.  
  88. #
  89. # orbital_start_direction: which type of input file (*.a.OIS | *.d.OIS) we
  90. # prefer, can be one of
  91. #
  92. # true : the file must *have* a direction specified
  93. # false : the file may *not* have a direction specified
  94. # descending : the file must be a *.d file
  95. # ascending : the file must be a *.a file
  96. #
  97. orbital_start_direction 'descending'
  98.  
  99. #
  100. # solarelevations: a list of minimum, maximum, and pecent of line in min-max
  101. # range for which data will be selected
  102. #
  103. solarelevations -90, -3, 10.0
  104.  
  105. #
  106. # hold: how old data, with respect to aquisition time, must be in hours
  107. # before any output will be allowed to be downloaded/viewed
  108. #
  109. hold 1
  110.  
  111. #
  112. # username: the login for web pickup - transparently added to .htpasswd file
  113. # in the web root
  114. #
  115. username 'foo'
  116.  
  117. #
  118. # password: the password for web pickup - transparently added to .htpasswd
  119. # file in the web root
  120. #
  121. password 'bar'
  122.  
  123. #
  124. # start_time: the wallclock time the subscription starts, no data will be
  125. # processed before this *local* time, an iso8601 timestamp
  126. #
  127. start_time '2008-01-01'
  128.  
  129. #
  130. # data_start_time: data aquired before this utc iso8601 will not be
  131. # processed
  132. #
  133. data_start_time_utc '2008-12-31'
  134.  
  135. #
  136. # end_time: the wallclock *local* time, an iso8601 timestamp, beyond which
  137. # no data will be processed
  138. #
  139. end_time '2009-01-01'
  140.  
  141. #
  142. # keep: a list of regular expressions all output files will be matched
  143. # against when deciding which files to keep. files not matched are not
  144. # kept. you may also use the alias "to_keep".
  145. #
  146. keep %r/.OIS.tif$/
  147.  
  148. #
  149. # reject: a list of regular expressions all output files will be matched
  150. # against when deciding which files to reject. files not matched are kept.
  151. # you may also use the alias "to_reject". note that this list is processed
  152. # *before* keep/to_keep - so files are pruned first by rejecting, then by
  153. # selecting. you may also use the alias "to_reject".
  154. #
  155. reject %r/.OIS.tif$/
  156.  
  157. #
  158. # tmpwatch: a string temporal expression describing how long to keep files
  159. # in the sif directory. files older than this will be deleted.
  160. #
  161. tmpwatch '8 days'
  162.  
  163. #
  164. # minsize: files with fewer lines that this (after processing) do not
  165. # produce any output - they are ignored
  166. #
  167. minsize 42
  168.  
  169. #
  170. # flagged: if this is set to true a flag file will be searched for and
  171. # included in processing of, for example, gridding and cropping. because
  172. # this is a reltively new feature it is *not* an error for the flag file to
  173. # be unavailable at the time of processing - instead a warning is simply
  174. # issued
  175. #
  176. flagged true
  177.  
  178. #------------------------------------------------------------------------------
  179. # the following configuration settings are available, but should rarely be
  180. # needed
  181. #------------------------------------------------------------------------------
  182.  
  183. #
  184. # roi_padding: an integer by which to pad the roi for certain operations
  185. # such as pre-crop done before gridding and before the final crop. it's
  186. # important this amount be large enough to allow smoothing along edges.
  187. #
  188. roi_padding 1
  189.  
  190. #
  191. # smooth_samples: the section/columns of the data considered for coverage
  192. # operations in smooth data. a list of left, right bounds.
  193. #
  194. smooth_samples [31, 1435]
  195.  
  196. #
  197. # smooth_stride: the step used during fast subsetting of smooth data to test
  198. # roi coverage. this means that every 'smooth_stride' pixels are sampled
  199. # for in-ness/out-ness
  200. #
  201. smooth_stride 20
  202.  
  203. #
  204. # fine_samples: the section/columns of the data considered for coverage
  205. # operations in fine data. a list of left, right bounds.
  206. #
  207. fine_samples [151, 7173]
  208.  
  209. #
  210. # fine_stride: the step used during fast subsetting of fine data to test
  211. # roi coverage. this means that every 'fine_stride' pixels are sampled
  212. # for in-ness/out-ness
  213. #
  214. fine_stride 100
  215.  
  216. #
  217. # nighttime: only night files (*.nt.*) will be accepted as input
  218. #
  219. nighttime true
  220.  
  221. #
  222. # dem: full path to a dem file usable by grid_ols
  223. #
  224. dem '/dmsp/local/srtm/srtm_gtopo30.dem'
  225.  
  226. #
  227. # data_end_time_utc: a time beyond aquisition time no data will be processed
  228. #
  229. data_end_time_utc '2008-12-31'
  230.  
  231. #
  232. # priority: the priority this sif will be submitted to the job queue (rq) at
  233. #
  234. priority 42
  235.  
  236. #
  237. # contacts: list of strings, email addresses, etc. to contact regarding this
  238. # sif
  239. #
  240. contacts 'ara.t.howard@gmail.com'
  241.  
  242. #
  243. # turd: if this is set to true tmp files are not cleaned up. useful for
  244. # debugging
  245. #
  246. turd true
  247.  
  248. #
  249. # dump_header: if this is true a txt header will be delived alongside any
  250. # other products
  251. #
  252. dump_header true
  253.  
  254. #
  255. # solar_time_range: time range of on-the-ground time data will be cropped to
  256. #
  257. solar_time_range '05:30:00', '06:30:00'
  258.  
  259. #------------------------------------------------------------------------------
  260. # many programs have default options which can be configured in a sif via the
  261. # configuration method. the key is not always the program name, but is
  262. # sometime a particular application of a program by the nrtlib
  263. # a partial list is:
  264. #
  265. # - subset
  266. # - fast_subset
  267. # - crop_to_solar_times
  268. # - confirmation_subset
  269. # - rename_ols
  270. # - change_scanner_offset
  271. # - grid_ols
  272. # - mk_geotiff
  273. # - crop_to_real_data
  274. #------------------------------------------------------------------------------
  275.  
  276. configure 'grid_ols', 'artificialscans' => 'process'
  277.  
  278. #------------------------------------------------------------------------------
  279. # the nrtlib provides many hooks which can save you quite a hassle trying to
  280. # override a particular method - although you are always free to override a
  281. # given method at any point these hooks are by default empty, called for you,
  282. # and a good place to inject special behaviour into a sif
  283. #------------------------------------------------------------------------------
  284.  
  285. #
  286. # pre_process_ols: called right before processing starts
  287. #
  288. def pre_process_ols
  289. abort unless @pathname =~ %r/foobar/
  290. end
  291.  
  292. #
  293. # post_process_ols: called right after processing has completed
  294. #
  295. def post_process_ols
  296. abort unless @pathname =~ %r/foobar/
  297. end
  298.  
  299. end
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement