CDDIS File Upload FAQ
- What is the new CDDIS file upload procedure?
The CDDIS is transitioning the method used to supply files to its archive from an ftp-based to an HTTP-based procedure. The new system also uses a NASA single sign-on system for user authentication.
- Why is CDDIS making this change?
The major reason for changing these procedures at CDDIS is security; ftp with its clear text username and password is just not acceptable in this day and age. In fact, some organizations have deprecated ftp or outright banned its usage. ftp also has the disadvantage of being a two-port protocol along with both an active and passive operation mode. This means that EVERY firewall/switch in the network path between CDDIS and the data provider must be configured correctly to function reliably. Unfortunately we have found this too often to not be the case and in most instances outside the control of CDDIS or the data provider to fix.
- Why not use sftp or some other ftp variant?
CDDIS looked into using some other variants to ftp for file upload but there were disadvantages (some major) to all of these approaches. The http(s) protocol offered more advantages and had performance equal to ftp.
- What is Earthdata Login?
Earthdata login (https://urs.earthdata.nasa.gov/) is the single mechanism for user registration and profile management for all Earth Observing System Data and Information System (EOSDIS) components. CDDIS, as one of the 12 EOSDIS Distributed Active Archive Centers (DAACs), is moving to be more integrated with its larger parent organization.
- When will this new file upload system go operational?
Current plans are for the new file upload procedure to go operational in Fall 2016 when CDDIS moves to its new computer facilities.
- Will my old scripts stop working?
When CDDIS transitions to its new computer facilities in Fall 2016, all legacy ftp upload systems will be turned off. Any application or scripts using ftp for file upload will cease to work at that time.
- Do I need to only use cURL?
No, you can use any tool that allows you to submit an HTTP POST request and that follows an HTTP redirect (302 request). CDDIS recommends that you use cURL as it’s what we support but you are free to use other tools or write your own. CDDIS has provided a Java example to showcase how this can be done.
- Is there a list of file upload parameters and examples?
Yes. A list of upload parameters and example filenames is available.
- When I upload a batch of files there seems to be a delay before anything happens; is this normal?
Yes, the new upload system allows the user to upload either a single file or batch of files in a single call. If a large number of files are being uploaded in a single call there can be a short time lag as the HTTP POST request is generated. NOTE: the HTTP POST generation takes place on the user’s machine and therefore is not related to CDDIS performance. You are always free to write your scripts so that single files or a small number of files are uploaded sequentially instead of in batch mode.
- What is the largest file size that CDDIS can accept?
500MB; for larger files please contact email@example.com for assistance.
- What browsers are supported for the web-based interface?
The latest versions of Firefox, Chrome, Internet Explorer (IE), and Safari are officially supported. The process may work in other browsers, but is not guaranteed.
- Who do I contact for questions?
Any questions related to the new upload system should be directed to firstname.lastname@example.org.