3.8 Retaining Retrieved Files

Availability: All operators Short options: ‘-R’ Long options: ‘--rtn’, ‘--retain’

In order to conserve local file system space, files retrieved from remote locations are automatically deleted from the local file system once they have been processed. Many NCO operators were constructed to work with numerous large (e.g., 200 MB) files. Retrieval of multiple files from remote locations is done serially. Each file is retrieved, processed, then deleted before the cycle repeats. In cases where it is useful to keep the remotely-retrieved files on the local file system after processing, the automatic removal feature may be disabled by specifying ‘-R’ on the command line.

Invoking -R disables the default printing behavior of ncks. This allows ncks to retrieve remote files without automatically trying to print them. See ncks netCDF Kitchen Sink, for more details.

Note that the remote retrieval features of NCO can always be used to retrieve any file, including non-netCDF files, via SSH, anonymous FTP, or msrcp. Often this method is quicker than using a browser, or running an FTP session from a shell window yourself. For example, say you want to obtain a JPEG file from a weather server.

     ncks -R -p ftp://weather.edu/pub/pix/jpeg -l . storm.jpg

In this example, ncks automatically performs an anonymous FTP login to the remote machine and retrieves the specified file. When ncks attempts to read the local copy of storm.jpg as a netCDF file, it fails and exits, leaving storm.jpg in the current directory.

If your NCO is DAP-enabled (see OPeNDAP), then you may use NCO to retrieve any files (including netCDF, HDF, etc.) served by an OPeNDAP server to your local machine. For example,

     ncks -R -l . -p \
     http://www.cdc.noaa.gov/cgi-bin/nph-nc/Datasets/ncep.reanalysis.dailyavgs/surface \
       pres.sfc.1969.nc

Note that NCO is never the preffered way to transport files from remote machines. For large jobs, that is best handled by FTP, SSH, or wget. It may occasionally be useful to use NCO to transfer files when your other preferred methods are not available locally.