3.3.2 - Download Remote Files

      Action Option -DR (-DownloadRemote) (Aliass: -DownloadRemoteFile) :

      an action to download or copy server files onto local working disk, or process data locally to generate new data files. These data files that are downloaded/copied/generated are called remote files and they are further validated/converted to local files via Action -BL (-BuildLocal).

      dsupdt[[-(DS|dataset)] dsnnn.n] -(DR|DownloadRemote) [Mode Options]
        [-(CI|ControlIndex) UpdateControlIndex]
        [-(LI|LocalIndex) LocalFileIndices]
        [-(LF|LocalFile) LocalFileNames]
        [-(RF|RemoteFile) RemoteFileNames]
        [-(SF|ServerFile) ServerFileNames]
        [-(DO|DownloadOrder) DownloadOrderIndices]
        [-(XO|ExecOrder) ExecOrderIndex]
        [-(DC|DownloadCommand) DownloadCommand]
        [-(PR|ProcessRemote) AdditionalRemoteFilePrecess]
        [-(ED|EndDate) NextDataEndDate]
        [-(EH|EndHour) NextDataEndHour]
        [-(GP|GenericPattern) GenericPatterns]
        [-(WD|WorkDir) WorkingDirectory]
        [-(MR|MissRemote) AllowMissRemoteFile]
        [-(FQ|Frequency) UpdateFrequency]
        [-(CD|CurrentDate) CurrentDate]
        [-(CH|CurrentHour) CurrentHour]
        [-(VS|ValidSize) MinSizeForValidFile]
        [-(PL|ProcessLimit) MaxNumberOfChildProcesses]
        [-(HO|HourOffset) TimeZoneHourOffset]
        [-(SB|SbatchOptions) SlurmBatchOptions]
        [-(QS|QsubOptions) PBSBatchOptions]
        [-(BP|BatchProcess) [BatchProcessHosts]]
        [-(CC|CarbonCopy) Cc'dEmailAddresses]
        [-(DB|Debug) DebugModeInfo]

      Mode option that can be specified for downloading file Action:

      If a remote file is on the local disk already, the download action for this remote file is skipped; the existing remote file must removed manually or Mode option -RD (-RetryDownload) is present if download/copy is needed again.

      A server file is the original data file on remote server. It is specified only when it is named different from the remote file name.



      Back to Top
      -(BG|BackGround)background process to turn off screen display for both
      -(CN|CheckNewer)for server file is locally, checks if the file is changed
      -(CP|CurrentPeriod)allows action of the end date/hour beyond the current
      -(EE-ErrorEmail)sends email only whne error happens during retrieving
      -(FU|ForceUpdate)if present, force download at lease for one end data/time,
      -(GZ|GMTZone)uses GMT dates/times as controlling times
      -(HU|HourlyUpdate)forces update of time to hours after successful remote
      -(IE|IgnoreError)works with Mode option -MU (-MultipleUpdate) to skip
      -(KS|KeepServer)keep server file on local disk by copying it to remove
      -(LO|LogOn)turn detail logging on if option -PL is present
      -(MO|MissedOnly)download remote file only if it is not archived yet
      -(MU|MultipleUpdate)allows multiple downloads/copies if multiple data update
      -(NE|NoEmail)does not send email to the specialist after download
      -(NY|NoLeapYear)skips February 29 for leap years
      -(QE|QuitError)quits downloading remote files of a dataset when an
      -(RD|RetryDownload)retry download the remote file if it is local already
      -(SE|SummaryEmail)send a summary email to the specialist after update
      -(UB|UseBeginTime)when present, use the beginning time of update period
      -(UB|UseBeginTime)when present, use the beginning time of update period for replace temporal pattern of end time in file names and download commands