- GNSS
- SLR
- VLBI
-
Product holdings
- VLBI TRF products
- VLBI Daily SINEX Independent Solution product (DSNX)
- VLBI Daily Intensive Solution Product (DSNI)
- Global VLBI Celestial Reference Frame solution
- Global VLBI Terrestrial Reference Frame solution product
- Intensive Earth Orientation Parameter Series (EOP-I) Product
- Session Earth Orientation Parameter Series (EOP-S) Product
- DORIS
- Other products
- Archive search information
- Archive access information
- Reports
The Crustal Dynamics Data Information System (CDDIS) supports data archiving and distribution activities for a global user community. Since its inception, users have relied on anonymous ftp for accessing and downloading files from the system. Although this protocol allows users to easily automate file downloads, many organizations, data systems, and users have already migrated from ftp or are actively pursuing a move away from the protocol due to problems from a system and security standpoint. U.S. Government agencies have become increasingly concerned about this legacy protocol and ensuring data integrity for the user community and have disallowed the use of the ftp protocol. The CDDIS, located at NASA GSFC, has therefore addressed these concerns and provided alternative methods for access to its archive for continued easy and automated download of its contents.
Therefore, the CDDIS discontinued anonymous ftp access to its archive in October 2020. Users must now utilize more secure access protocols such as https or ftp-ssl.
In order to access the CDDIS archive, users will need to use a client that supports Transport Security Layer (TLS). When using the https protocol (for example, cURL, Wget, or Python commands/scripts) users will need to have an Earthdata Login account. In addition, when utilizing the cURL and Wget, users will need to create a .netrc file; instructions for creating this file are available.
Examples of accessing the archive via TLS using various methods can be found below. Click on the name of a method to view the instructions; click on a link within a method to see expected results of that command. In the examples, <email address> means type in your actual email address without the < and > symbols.
Click on an example in the table below; the example details will appear below the table.
| Client | HTTPS Access | ftp-ssl Access |
|---|---|---|
| https://cddis.nasa.gov/archive/ | ftp://gdc.cddis.eosdis.nasa.gov/ | |
| cURL |
|
|
| Wget (version ≥ 1.18) |
|
|
| lftp |
|
|
| Python 3 |
|
|
| Java |
|
|
If you are having issues with any of these examples, please check our Archive Download FAQ Page
Provide a list of files in the archive directory /gps/data/daily/2019/045/19o/
- -c = save cookies to a file
- [file] = file to hold cookies
- -n = use .netrc file
- -L = follow redirects
> curl -c [file] -n -L "https://cddis.nasa.gov/archive/gps/data/daily/2019/045/19o/*?list"
Download the file 'cs2rx17001.001.Z' from archive directory /doris/data/cs2/2017/ to your local file system
- -c = save cookies to a file
- [file] = file to hold cookies
- -n = use .netrc file
- -L = follow redirects
- -O = write to local system using remote filename
> curl -c [file] -n -L -O "https://cddis.nasa.gov/archive/doris/data/cs2/2017/cs2rx17001.001.Z"
Download all files from archive directory /doris/data/cs2/2017/ matching the pattern 'cs2rx1700*.001.Z' to [files.tar] on your local file system.
- -c = save cookies to a file
- [file] = file to hold cookies
- -n = use .netrc file
- -L = follow redirects
- -o = write to specified output filename
- -O = write to local system using remote filename(s)
> curl -c [file] -n -L -o [files.tar] "https://cddis.nasa.gov/archive/doris/data/cs2/2017/cs2rx1700*.001.Z"
Download three files ('19JAN12XK_N003.gz', '19JAN19XK_N003.gz', '19JAN20XK_N003.gz') from archive directory vlbi/ivsdata/ngs/2019/ to your local file system.
> curl -c [file] -n -L -O "https://cddis.nasa.gov/archive/vlbi/ivsdata/ngs/2019/19JAN{12,19,20}XK_N003.gz"
Download a range of files to your local file system:
> curl -c [file] -n -L -O "https://cddis.nasa.gov/archive/doris/data/cs2/2017/cs2rx1700[1-3].001.Z"
Get a list of files in archive directory '/gnss/products/2048/'
- -u = specify user
- anonymous:<your_email_address> = anonymous is the user name; your email address is the password
- -ftp-ssl = use ftp-ssl protocol
> curl -u anonymous:<your_email_address> --ftp-ssl ftp://gdc.cddis.eosdis.nasa.gov/gnss/products/2048/
Download file 'gfz20483.sp3.Z' from archive directory '/gnss/products/2048/' to your local file system
- -u = specify user
- anonymous:<your_email_address> = anonymous is the user name; your email address is the password
- -ftp-ssl = use ftp-ssl protocol
- -O = write to local system using remote filename
> curl -u anonymous:<your_email_address> -O --ftp-ssl ftp://gdc.cddis.eosdis.nasa.gov/gnss/products/2048/gfz20483.sp3.Z
Get a list of files from archive directory /doris/data/cs2/2017/ with names containing 'cs2rx1700', '.001', and '.Z' in that order.
- --auth-no-challenge = Needed for servers such as Earthdata Login which do not send HTTP authentication challenges, but accept unsolicited authentication information
- -q = turn off Wget's output
- -O = write to output file
> wget --auth-no-challenge -qO- "https://cddis.nasa.gov/archive/doris/data/cs2/2017/cs2rx1700*.001.Z*?list"
Download the file 'cs2rx17001.001.Z' from archive directory '/doris/data/cs2/2017/' to your local file system.
- --auth-no-challenge = Needed for servers such as Earthdata Login which do not send HTTP authentication challenges, but accept unsolicited authentication information
> wget --auth-no-challenge "https://cddis.nasa.gov/archive/doris/data/cs2/2017/cs2rx17001.001.Z"
Download the file 'cs2rx17001.001.Z' from archive directory '/doris/data/cs2/2017/' to your local file system.
- --ftp-user anonymous = specifies the user name is "anonymous"
- --ftp-password <email address> = specifies that the password is your email address
> wget --ftp-user anonymous --ftp-password <email address>
ftps://gdc.cddis.eosdis.nasa.gov/doris/data/cs2/2017/cs2rx17001.001.Z
Download all files in archive directory '/vlbi/ivsdata/ngs/2019/' to your local file system.
- --ftp-user anonymous = specifies the user name is "anonymous"
- --ftp-password <email address> = specifies that the password is your email address
- -r = Turn on recursive retrieving.
> wget --ftp-user anonymous --ftp-password <email address> -r ftps://gdc.cddis.eosdis.nasa.gov/vlbi/ivsdata/ngs/2019/
Log into the archive enabling the use of any FTP command.
- -d = turn on debugging mode
- -u anonymous,<email address> = specifies the username is "anonymous" and the password is your email address
- -e = execute the lftp command following the -e and don't exit
- set ftp:ssl-force true = encrypt all communication between you and the server
> lftp -d -u anonymous,<email address> -e 'set ftp:ssl-force true' gdc.cddis.eosdis.nasa.gov
Download the file 'cs2rx17001.001.Z' from archive directory '/doris/data/cs2/2017'/ to the current directory on your local file system.
- -d = turn on debugging mode
- -u anonymous,<email address> = specifies the username is "anonymous" and the password is your email address
- -e = execute the lftp command following the -e and don't exit
- set ftp:ssl-force true = encrypts all communication between you and the server
- exit = closes the connection
> lftp -d -u anonymous,<email address> -e 'set ftp:ssl-force true' -e 'get doris/data/cs2/2017/cs2rx17001.001.Z;exit' gdc.cddis.eosdis.nasa.gov
Download all files in the '/vlbi/ivscontrol/' directory to the current directory on your local file system.
- -d = turn on debugging mode
- -u anonymous,<email address> = specifies the username is "anonymous" and the password is your email address
- -e = execute the lftp command following the -e and don't exit
- set ftp:ssl-force true = encrypts all communication between you and the server
- exit = closes the connection
> lftp -d -u anonymous,<email address> -e 'set ftp:ssl-force true' -e 'mget vlbi/ivscontrol/*;exit' gdc.cddis.eosdis.nasa.gov
Download the file 'cs2rx17001.001.Z' from archive directory '/doris/data/cs2/2017/' to your local file system
Place the code below into a file. The lines that begin with "#" are comments and explain what the next line of code does.
import sys
import pycurl
# Set the archiveLocation to the first command line argument
archiveLocation = sys.argv[1]
# Set the fileName to the second command line argument
fileName = sys.argv[2]
# Initialize the cURL connection object
curl = pycurl.Curl()
# Define the url to use
curl.setopt(curl.URL, archiveLocation + fileName)
# Set curl to follow redirects, needed to allow user login
curl.setopt(curl.FOLLOWLOCATION, True)
# Set the requirement that cURL use a netrc file found in users home directory
curl.setopt(curl.NETRC,2)
# Set the file used to store cookie
curl.setopt(curl.COOKIEJAR, '.cddis_cookies')
# Writes the remote file to a new file with the same name
with open(fileName, 'w') as f:
curl.setopt(curl.WRITEFUNCTION, f.write)
curl.perform()
# Clean up and close the cURL object
curl.close()
Enter this command on the command line:
> python [code_filename] https://cddis.nasa.gov/archive/doris/data/cs2/2017/ cs2rx17001.001.Z
Download all files from archive directory /doris/data/cs2/2017/ to your local file system
Place the code below into a file. The lines that begin with "#" are comments and explain what the next line of code does.
import cStringIO
import sys
#Set the archive directory to the command line argument
archiveDirectory = sys.argv[1]
#Initialize the response object for the directory listing
response = cStringIO.StringIO()
#Initialize the cURL python object
curl = pycurl.Curl()
#Set the cURL URL to the directory passed in with '*?list' added to get a directory listing
curl.setopt(curl.URL, archiveDirectory + '*?list')
#Tell cURL to follow redirects, needed to allow user login
curl.setopt(curl.FOLLOWLOCATION, True)
#Set the requirement that cURL use a .netrc file found in the users home directory
curl.setopt(curl.NETRC,2)
#Set the file used to store the login cookie
curl.setopt(curl.COOKIEJAR, 'cookie.txt')
#Tell cURL to write the response from the cURL call to the response object
curl.setopt(curl.WRITEFUNCTION, response.write)
#Execute the cURL call
curl.perform()
#Get the text response and store it in a variable
output = response.getvalue()
#Split the output into lines, based on a new line character
lines = output.split('\n')
#Iterate over the lines of the listing
for line in lines:
if ((('index.html') not in line) and (('#') not in line)):
filename = line.split(' ')[0].strip()
#Drop some blank lines that can occur
if (len(filename) > 0):
print archiveDirectory + filename
#Set the cURL URL to a new URL, defined by the original directory + the filename from the listing
curl.setopt(curl.URL, archiveDirectory + filename)
#Open a new file on the local host, with the filename calculated above
fpointer = open(filename, 'wb')
#Tell cURL to write this response to a file
curl.setopt(curl.WRITEFUNCTION, fpointer.write)
#Execute this cURL command
curl.perform()
curl.close()
Enter this command on the command line:
> python [code_filename] https://cddis.nasa.gov/archive/doris/data/cs2/2017/
Download the file passed as a command line argument from archive directory to your local file system
Place the code below into a file. The lines that begin with "#" are comments and explain what the next line of code does.
Note: this code example requires a .netrc file set up as described here
import requests
import sys
# Reads the URL from the command line argument
url = sys.argv[1]
# Assigns the local file name to the last part of the URL
filename = url.split('/')[-1]
# Makes request of URL, stores response in variable r
r = requests.get(url)
# Opens a local file of same name as remote file for writing to
with open(filename, 'wb') as fd:
# Closes local file
fd.close()
Enter the following on the command line:
> python3 <source code file> <URL to download>
List all files in a directory.
Place the code below into a file. The lines that begin with "#" are comments and explain what the next line of code does.
Note: this code example requires a .netrc file set up as described here
import requests
import sys
url = sys.argv[1]
#Adds '*?list' to the end of URL if not included already
if not url.endswith("*?list"):
#Makes request of URL, stores response in variable r
r = requests.get(url)
#Prints the results of the directory listing
print(r.text)
Enter the following on the command line:
> python3 <source code file> <URL of directory to list>
Download single file via ftp-ssl
Place the code below into a file.
from ftplib import FTP_TLS
import sys
email = sys.argv[1]
directory = sys.argv[2]
filename = sys.argv[3]
ftps = FTP_TLS(host = 'gdc.cddis.eosdis.nasa.gov')
ftps.login(user='anonymous', passwd=email)
ftps.prot_p()
ftps.cwd(directory)
ftps.retrbinary("RETR " + filename, open(filename, 'wb').write)
Enter the following on the command line:
> python3 <source code file> <Your email address> <directory> <filename>
List all files in a directory.
Place the code below into a file.
from ftplib import FTP_TLS
import sys
email = sys.argv[1]
directory = sys.argv[2]
ftps = FTP_TLS(host = 'gdc.cddis.eosdis.nasa.gov')
ftps.login(user='anonymous', passwd=email)
ftps.prot_p()
ftps.cwd(directory)
print(ftps.dir())
Enter the following on the command line:
> python3 <source code file> <Your email address> <directory to list>
Download the file 'ids17d01.snx.Z' from archive directory '/doris/products/sinex_global/ids/' to your local file system.
There are 3 steps to complete before receiving the file:
- Save the code below to a file.
- Compile the code.
- Run the code.
Step 1 is below and Steps 2 and 3 are shown after the code.
Place the code below into a file named CddisHttpsFileDownload.java (or name you select). The lines that begin with "//" are comments and explain what the next line of code does. A comment block begins with "/*" and ends with "*/".
import java.io.*;
import java.net.*;
import java.util.*;
import javax.net.ssl.*;
public class CddisHttpsFileDownload {
static String URS = "https://urs.earthdata.nasa.gov";
public static void main(String args[]){
if (args.length < 2){
System.exit(0);
String resource = args[0].trim();
String username = null;
String password = null;
/* Save the filename from the resource requested as the local file name; this is one method but it can be done in many ways. */
String localFilename = resource.split("/")[resource.split("/").length-1];
/* Read netrc file and extract username and password; this is one method but it can be done in many ways. */
try {
while((line=br.readLine())!=null) {
password = line.split(" ")[5];
catch (Exception e){
try {
CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));
/* Retrieve a stream for the resource */
InputStream in = getResource(resource, username, password);
/* Read from the URL and write to a local file */
try {
FileOutputStream fileOS = new FileOutputStream(localFilename);
byte data[] = new byte[1024];
int byteContent;
while ((byteContent = inputStream.read(data, 0, 1024)) != -1) {
catch (IOException e) {
catch( Exception t) {
System.out.println(t.getMessage());
t.printStackTrace();
/* Return an input stream for a designated resource on a URS-authenticated remote server.*/
public static InputStream getResource (
/* Place an upper limit on the number of redirects we will follow */
while( redirects < 10 ) {
/* Configure a connection to the resource server and submit the request for our resource. */
URL url = new URL(resource);
HttpsURLConnection connection = null;
if (url.getProtocol().equalsIgnoreCase("https")) {
/* Handle any redirect that goes to http - set it back to an https request */
else {
connection.setRequestMethod("GET");
connection.setInstanceFollowRedirects(false);
connection.setUseCaches(false);
connection.setDoInput(true);
/* If this is the URS server, add in the authentication header. */
if( resource.startsWith(URS) ) {
connection.setRequestProperty (
"Basic " + Base64.getEncoder().encodeToString (
/*Execute the request and get the response status code. A return status code of 200 is good - it means that we have our resource. We can return the input stream so it can be read (may also want to return additional header information such as the mime type or size). */
int status = connection.getResponseCode();
if( status == 200 ) {
/* Any returned status code other than 302 (a redirect) will need custom handling. A 401 code means that the credentials aren't valid. A 403 code means that the user hasn't authorized the application. */
if( status != 302 ) {
/* Get the redirection location and continue. This should really have a null check, just in case. */
resource = connection.getHeaderField("Location");
}
throw new Exception("Redirection limit exceeded");
}
Compile the Java code:
> javac CddisHttpsFileDownload.java
Run the code
> java CddisHttpsFileDownload https://cddis.nasa.gov/archive/doris/products/sinex_global/ids/ids17d01.snx.Z
Download the file 'ign02wd02.eop.Z' from archive directory 'doris/products/eop/' to your local file system
There are 3 steps to complete before receiving the file:
- Download the apache commons net jar file (commons-net-3.6.jar) from: https://commons.apache.org/proper/commons-net/download_net.cgi.
- Save the code below to a file.
- Compile and run the code.
Step 2 is below and Step 3 is shown after the code.
Place the code below into a file named FileDownload.java (or name you select) . The lines that begin with "//" are comments and explain what the next line of code does. A comment block begins with "/*" and ends with "*/".
import java.io.FileOutputStream;
import java.io.PrintWriter;
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.*;
public class FileDownload {
Takes the following arguments:
- args[0] = FTP SSL Archive
- args[1] = Full path to remote file, including filename
- args[2] = Full path to write local file
- args[3] = User's email address
*/
FileDownload fd = new FileDownload();
fd.downloadSingleFile(args[0].trim(), args[1].trim(), args[2].trim(), args[3].trim());
FTPSClient client = new FTPSClient();
//Optional, print out full command responses during exectute. Helpful for debugging.
client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
try{
client.connect(ftpServer, 21);
//Login using required credentials
client.login("anonymous", emailAddress);
//Set the Protection Buffer Size
client.execPBSZ(0);
//Set the Data Channel Protection Level to P-Private
client.execPROT("P");
//Set the current data connection mode to PASSIVE_LOCAL_DATA_CONNECTION_MODE
client.enterLocalPassiveMode();
//Create new FileOutputStream for writing local file
FileOutputStream fos = new FileOutputStream(localFilename);
//Retrieve file from server
client.retrieveFile(remoteFilename, fos);
//Flush the FileOutputStream, to get all data to the file
fos.flush();
//Close the FileOutputStream
fos.close();
//Logout from ftp-ssl server
client.logout();
}
}
Compile the Java code:
> javac -classpath .:commons-net-3.6.jar FileDownload.java
Run the code
> java -cp .:commons-net-3.6.jar FileDownload gdc.cddis.eosdis.nasa.gov doris/products/eop/ign02wd02.eop.Z <INSERT_EMAIL_ADDRESS>
Download all files that contain 'cod20961' from archive directory 'gnss/products/2096/' to your local file system
There are 3 steps to complete before receiving the file:
- Download the apache commons net jar file (commons-net-3.6.jar) from: https://commons.apache.org/proper/commons-net/download_net.cgi.
- Save the code below to a file.
- Compile and run the code.
Step 2 is below and Step 3 is shown after the code.
Place the code below into a file named FilesDownload.java (or name you select). The lines that begin with "//" are comments and explain what the next line of code does. A comment block begins with "/*" and ends with "*/".
import java.io.File;
import java.io.FileOutputStream;
import java.io.PrintWriter;
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.*;
public class FilesDownload {
Takes the following arguments:
- args[0] = FTP SSL Archive
- args[1] = Full path to remote files
- args[2] = Filename pattern to match
- args[3] = Full path to write local file
- args[4] = User's email address
*/
FilesDownload fd = new FilesDownload();
fd.downloadMultipleFiles(args[0].trim(), args[1].trim(), args[2].trim(), args[3].trim(), args[4].trim());
FTPSClient client = new FTPSClient();
//Optional, print out full command responses during exectute. Helpful for debugging.
client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
try{
client.connect(ftpServer, 21);
//Login using required credentials
client.login("anonymous", emailAddress);
//Set the Protection Buffer Size
client.execPBSZ(0);
//Set the Data Channel Protection Level to P-Private
client.execPROT("P");
//Set the current data connection mode to PASSIVE_LOCAL_DATA_CONNECTION_MODE
client.enterLocalPassiveMode();
//Get a list of all files in the desired directory
FTPFile[] files = client.listFiles(directory);
//Iterate over all files
for (FTPFile file : files){
if (file.getName().contains(filePattern)){
FileOutputStream fos = new FileOutputStream(localDir + File.separator + file.getName());
//Retrieve file from server
client.retrieveFile(directory + "/" + file.getName(), fos);
//Flush the FileOutputStream, to get all data to the file
fos.flush();
//Close the FileOutputStream
fos.close();
}
//Logout from ftp-ssl server
client.logout();
}
}
Compile the Java code:
> javac -classpath .:commons-net-3.6.jar FilesDownload.java
Run the code
> java -cp .:commons-net-3.6.jar FilesDownload gdc.cddis.eosdis.nasa.gov /gnss/products/2096/ cod20961 <INSERT LOCAL DIRECTORY> <INSERT_EMAIL_ADDRESS>
Web Curator: Lori J. Tyahla
FAQ
Earthdata Forum
Contact Us
Last updated: May 27, 2021