On 31 October 2020, anonymous ftp access will officially be discontinued at CDDIS. Beginning October 1, 2020, CDDIS will bandwidth limit ALL IPs using anonymous ftp to just 50KB/s and will limit simultaneous downloads to only 3 per IP address. CDDIS is STRONGLY recommending all users begin to transition to using alternate method of access to the CDDIS archive. Examples of alternate methods can be found at: https://cddis.nasa.gov/Data_and_Derived_Products/CDDIS_Archive_Access.html.

close window
CDDIS https Archive Access/File Download FAQ
  1. How do I download data after anonyous ftp is discontinued on October 31, 2020?
  2. What is the new CDDIS file download procedure?
    The CDDIS is adding an https-based method to retrieve files from its archive; users can continue to use anonymous ftp to access the CDDIS archive and retrieve files until October 31, 2020. The new system also uses a NASA single sign-on system for user authentication.
  3. Why is CDDIS making this change?
    The major reason for changing these procedures at CDDIS is security; ftp with its clear text username and password is just not acceptable in this day and age. In fact, some organizations have deprecated ftp or outright banned its usage. ftp protocol also has the disadvantage of being a two-port protocol along with both an active and passive operation mode. This means that EVERY firewall/switch in the network path between CDDIS and the data provider must be configured correctly to function reliably. Unfortunately we have found this too often to not be the case and in most instances outside the control of CDDIS or the data provider to fix.
  4. What are the advantages of https over anonymous ftp?
    The https protocol is as efficient as ftp transfer without the firewall/router issues of ftp. Since ftp is a two-port protocol, users can have connectivity problems (e.g., with firewall, DNS, etc.); https is a one-port protocol with fewer issues with downloads. The Earthdata Login system will be used for access through https. Earthdata Login allows users to easily search and access the full breadth of all 12 EOSDIS DAAC archives. Earthdata Login will also allow CDDIS to know our users better which will then allow us to improve CDDIS capabilities.
  5. Why not use sftp or some other ftp variant?
    CDDIS looked into using some other variants to ftp for file upload but there were disadvantages (some major) to all of these approaches. The http(s) protocol offered more advantages and had performance equal to ftp.
  6. What is Earthdata Login?
    Earthdata login (https://urs.earthdata.nasa.gov/) is the single mechanism for user registration and profile management for all Earth Observing System Data and Information System (EOSDIS) components. CDDIS, as one of the 12 EOSDIS Distributed Active Archive Centers (DAACs), is moving to be more integrated with its larger parent organization.
  7. When will this new file download system become operational?
    The new file download procedure is now fully operational.
  8. Will my old scripts stop working?
    Yes, scripts based on anonymous ftp will stop working after October 31, 2020. The https-based system currently exists in parallel with the legacy ftp system.
  9. Do I need to only use cURL?
    No, you can use any tool that allows you to submit an https request and that follows an http redirect (302 request). CDDIS recommends that you use cURL as it's what we support but you are free to use other tools or write your own.
  10. What browsers are supported for the web-based interface?
    The latest versions of Firefox, Chrome, Internet Explorer (IE), and Safari are officially supported. The process may work in other browsers, but is not guaranteed.
  11. What is the index.html file found in the archive directories?
    An index.html file is required in order for web browsers to be able to display the contents of a directory in a proper format. This file is used when accessing the CDDIS archive through https, not ftp. There is no harm in downloading this file; however, if you prefer to not retrieve the index.html file, you will need to modify your scripts to exclude any file named 'index.html' when downloading all files from a particular directory.
  12. Who do I contact for questions?
    Any questions related to the new upload system should be directed to support-cddis@earthdata.nasa.gov.
  13. How do I download files from the https archive using the command line or a script?
    Instructions and several examples of downloading files from the https archive using cURL, Wget, ftp-ssl, and Python are provided on our CDDIS Archive Access page.
  14. Why do I only get HTML code when trying to download a file via HTTPS?
    If you do not pass your credentials properly you are automatically redirected to the Earthdata Login web page. Since you are making an HTTPS request, the server is assuming you are using a web browser and returns the login page. This normally occurs when your .netrc file is missing or in the wrong place, or you have the wrong credentials in your .netrc file.
  15. Why do I get strange errors when I try to change my HTTPS or plain ftp request into one using ftp-ssl?
    The URL for using ftp-ssl is different from the ones used by our old ftp server and that of the HTTPS server. Please change your requests to reflect the new URL -- ftp://gdc.cddis.eosdis.nasa.gov/.
  16. Why am I getting an error stating that the maximum number of redirects has been followed?
    You are missing the required cookie file for an HTTPS download. If your login information is not saved in a cookie file, the client is stuck in a loop of trying to login and never saves that information. Check your syntax and make sure you are setting a cookie file.
  17. Why am I getting a cURL (35) error when I attempt to download?
    The most common reason for this is that you are using an older version of cURL that does not support the Earthdata Login requirement of TLS 1.2. You will need to update your cURL client.

    Also, some very recent versions of cURL and the most recent Debian-based linux distributions have this same issue. We are working on a change on the server to fix this problem. In the meantime, please try this workaround for this issue - add this to the command line:

    --ciphers DEFAULT@SECLEVEL=1
  18. Why does the filename on my system have a .gz extension but is no longer compressed after I download it?
    Certain download clients (including Chrome, Safari and some python libraries) automatically decompress a .gz file during the download process. However the file is not renamed, which leads to a confusing situation.

    Possible solutions:

    1. Use a different client if possible; cURL and Firefox do not appear to have this issue.
    2. Rename the decompressed file by dropping the .gz extension
    The CDDIS is aware of this issue and is working on a possible server-side solution.

Related Links: