Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- #------------------------------------------------------------------------------
- # NRT sif classes form the basis of customized DMSP data processing. Each
- # type of sif class my be inherited, configured, and specialized to allow any
- # manner of processing to be applied to DMSP data without concern for other
- # deails such as how the data triggers processing, etc. All that needs to be
- # done in a sif is to say how to process data and the NRT system takes care of
- # the rest. 9 times of of ten a sif file is just going to act as a
- # configuration file but, because it's an actual subclass, specialized
- # behaviour can be dynamically added and refined.
- #------------------------------------------------------------------------------
- #
- # we start out with a basic class definition
- #
- class ExampleSif < NRT::Geotiffed
- #------------------------------------------------------------------------------
- # valid subscription types, along with description, are:
- #
- # - NRT::OLSSubscription::Copied
- # any file is simply copied into the output space
- #
- # - NRT::OLSSubscription::CopiedIfCovered
- # any file is copied into the output space if, and only if, it covers
- # the configured roi
- #
- # - NRT::OLSSubscription::Subsetted
- # any file is cropped and copied into the output space if, and only if,
- # it covers the configured roi which is used as the crop specification.
- # any data over the roi is provided but the entire roi may or may not be
- # covered. if multiple non-continuous sections cover the roi then the
- # smallest crop that included all covering sections is used.
- #
- # - NRT::OLSSubscription::SubsettedMulitple
- # this is the same as Subsetted subscriptions with the exception that,
- # when multiple sections of data cover the roi they are delivered
- # individually rather than taking the minimum bounding rectangle which
- # contained all sections
- #
- # - NRT::OLSSubscription::Gridded
- # this is the same as Subsetted but, additionally, all sections will be
- # gridded according to the sif configuration. grids are delivered in
- # raw/envi format with appropriate header files
- #
- # - NRT::OLSSubscription::Geotiffed
- # the same as Gridded but each grid is also delivered in geotiff format
- #
- # - NRT::OLSSubscription::GriddedToRealData
- # the same as Gridded, but all output files have leading and trailing
- # artifical data removed
- #
- # - NRT::OLSSubscription::GriddedToRealDataAndGeotiffed
- # the same as Geotiffed, but all output files have leading and trailing
- # artifical data removed
- #------------------------------------------------------------------------------
- #------------------------------------------------------------------------------
- # following are the most important and commonly used configuration options
- #------------------------------------------------------------------------------
- #
- # mode: the only important mode is 'production', which allows a sif to
- # accept files for processing. any other mode, such as 'stopped',
- # effectively turns the sif off
- #
- mode 'production'
- #
- # roi: a list/array of coordinates to specify the roi in UL/LR-lat/lon pairs
- #
- roi 10, -170, 5, -160
- #
- # satellites: a list of strings denoting which satellites should be accepted
- # for processing
- #
- satellites %w( F14 F15 )
- #
- # extensions: a list of strings denoting which extenstions should be
- # accepted for processing
- #
- extensions %w( OIS OIF )
- #
- # orbital_start_direction: which type of input file (*.a.OIS | *.d.OIS) we
- # prefer, can be one of
- #
- # true : the file must *have* a direction specified
- # false : the file may *not* have a direction specified
- # descending : the file must be a *.d file
- # ascending : the file must be a *.a file
- #
- orbital_start_direction 'descending'
- #
- # solarelevations: a list of minimum, maximum, and pecent of line in min-max
- # range for which data will be selected
- #
- solarelevations -90, -3, 10.0
- #
- # hold: how old data, with respect to aquisition time, must be in hours
- # before any output will be allowed to be downloaded/viewed
- #
- hold 1
- #
- # username: the login for web pickup - transparently added to .htpasswd file
- # in the web root
- #
- username 'foo'
- #
- # password: the password for web pickup - transparently added to .htpasswd
- # file in the web root
- #
- password 'bar'
- #
- # start_time: the wallclock time the subscription starts, no data will be
- # processed before this *local* time, an iso8601 timestamp
- #
- start_time '2008-01-01'
- #
- # data_start_time: data aquired before this utc iso8601 will not be
- # processed
- #
- data_start_time_utc '2008-12-31'
- #
- # end_time: the wallclock *local* time, an iso8601 timestamp, beyond which
- # no data will be processed
- #
- end_time '2009-01-01'
- #
- # keep: a list of regular expressions all output files will be matched
- # against when deciding which files to keep. files not matched are not
- # kept. you may also use the alias "to_keep".
- #
- keep %r/.OIS.tif$/
- #
- # reject: a list of regular expressions all output files will be matched
- # against when deciding which files to reject. files not matched are kept.
- # you may also use the alias "to_reject". note that this list is processed
- # *before* keep/to_keep - so files are pruned first by rejecting, then by
- # selecting. you may also use the alias "to_reject".
- #
- reject %r/.OIS.tif$/
- #
- # tmpwatch: a string temporal expression describing how long to keep files
- # in the sif directory. files older than this will be deleted.
- #
- tmpwatch '8 days'
- #
- # minsize: files with fewer lines that this (after processing) do not
- # produce any output - they are ignored
- #
- minsize 42
- #
- # flagged: if this is set to true a flag file will be searched for and
- # included in processing of, for example, gridding and cropping. because
- # this is a reltively new feature it is *not* an error for the flag file to
- # be unavailable at the time of processing - instead a warning is simply
- # issued
- #
- flagged true
- #------------------------------------------------------------------------------
- # the following configuration settings are available, but should rarely be
- # needed
- #------------------------------------------------------------------------------
- #
- # roi_padding: an integer by which to pad the roi for certain operations
- # such as pre-crop done before gridding and before the final crop. it's
- # important this amount be large enough to allow smoothing along edges.
- #
- roi_padding 1
- #
- # smooth_samples: the section/columns of the data considered for coverage
- # operations in smooth data. a list of left, right bounds.
- #
- smooth_samples [31, 1435]
- #
- # smooth_stride: the step used during fast subsetting of smooth data to test
- # roi coverage. this means that every 'smooth_stride' pixels are sampled
- # for in-ness/out-ness
- #
- smooth_stride 20
- #
- # fine_samples: the section/columns of the data considered for coverage
- # operations in fine data. a list of left, right bounds.
- #
- fine_samples [151, 7173]
- #
- # fine_stride: the step used during fast subsetting of fine data to test
- # roi coverage. this means that every 'fine_stride' pixels are sampled
- # for in-ness/out-ness
- #
- fine_stride 100
- #
- # nighttime: only night files (*.nt.*) will be accepted as input
- #
- nighttime true
- #
- # dem: full path to a dem file usable by grid_ols
- #
- dem '/dmsp/local/srtm/srtm_gtopo30.dem'
- #
- # data_end_time_utc: a time beyond aquisition time no data will be processed
- #
- data_end_time_utc '2008-12-31'
- #
- # priority: the priority this sif will be submitted to the job queue (rq) at
- #
- priority 42
- #
- # contacts: list of strings, email addresses, etc. to contact regarding this
- # sif
- #
- contacts 'ara.t.howard@gmail.com'
- #
- # turd: if this is set to true tmp files are not cleaned up. useful for
- # debugging
- #
- turd true
- #
- # dump_header: if this is true a txt header will be delived alongside any
- # other products
- #
- dump_header true
- #
- # solar_time_range: time range of on-the-ground time data will be cropped to
- #
- solar_time_range '05:30:00', '06:30:00'
- #------------------------------------------------------------------------------
- # many programs have default options which can be configured in a sif via the
- # configuration method. the key is not always the program name, but is
- # sometime a particular application of a program by the nrtlib
- # a partial list is:
- #
- # - subset
- # - fast_subset
- # - crop_to_solar_times
- # - confirmation_subset
- # - rename_ols
- # - change_scanner_offset
- # - grid_ols
- # - mk_geotiff
- # - crop_to_real_data
- #------------------------------------------------------------------------------
- configure 'grid_ols', 'artificialscans' => 'process'
- #------------------------------------------------------------------------------
- # the nrtlib provides many hooks which can save you quite a hassle trying to
- # override a particular method - although you are always free to override a
- # given method at any point these hooks are by default empty, called for you,
- # and a good place to inject special behaviour into a sif
- #------------------------------------------------------------------------------
- #
- # pre_process_ols: called right before processing starts
- #
- def pre_process_ols
- abort unless @pathname =~ %r/foobar/
- end
- #
- # post_process_ols: called right after processing has completed
- #
- def post_process_ols
- abort unless @pathname =~ %r/foobar/
- end
- end
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement