ERDDAP   Easier access to scientific data

Brought to you by NOAA NMFS SFSC ERD

ERDDAP > Computer Programs

Accessing ERDDAP as a Web Service

• A web application – a web site that humans with browsers can use (in this case, to get data, graphs, and information about datasets).
• A web service – a web site that computer programs can use (in this case, to get data, graphs, and information about datasets).

Requests
Requests for interface information from ERDDAP (for example, search results) use the web's universal standard for requests: HTTP GET.
This is the same mechanism that your browser uses when you fill out a form on a web page and click on Submit.
To use HTTP GET, you generate a specially formed URL (perhaps with a query) and send it to the web with HTTP GET.
You can form these URLs by hand and enter them in the address textfield of your browser
(for example, http://coastwatch.pfeg.noaa.gov/erddap/search/index.json?searchFor=temperature),
or a computer program can be written to create the URLs, submit them, and get the responses.
Often, the computer program will need to percent encode the URL
(for example, replace " " in a query value with "%20", for example, "searchFor=temperature%20wind%20speed").
(Browsers do this for you automatically when you fill out a form on a web page and click on Submit.)
Programming languages have tools to do this (for example, see java.net.URLEncoder).
HTTP GET was chosen because it is simple, it works,
it is universally supported (in browsers, computer languages, operating system tools, etc),
and it is a foundation of REST and ROA.
These URLs are great because they completely define a given request,
and you can bookmark them in your browser, write them in your notes, email them to a friend, ...

Requesting Compressed Files
ERDDAP doesn't offer results stored in compressed (e.g., .zip or .gzip) files.
Instead, ERDDAP looks for accept-encoding in the HTTP GET request header sent by the client.
If a supported compression type ("gzip", "x-gzip", or "deflate") is found in the accept-encoding list,
ERDDAP includes "content-encoding" in the HTTP response header and compresses the data as it transmits it.
It is up to the client program to look for "content-encoding" and decompress the data.
Browsers and OPeNDAP clients do this by default. They request compressed data and decompress the returned data automatically.
Other clients (e.g., Java programs) have to do this explicitly.

Responses
Although humans using browsers want to receive interface results (for example, search results) as HTML documents,
computer programs often prefer to get results in simple, easily parsed, less verbose documents.
ERDDAP can return interface results as a table of data in these common, computer-program friendly, file types:

• .csv - Download the data as comma-separated ASCII text table (missing value = 'NaN'; times are ISO 8601 strings). (more info)
• .htmlTable - View an HTML file with the data in a table (times are ISO 8601 strings). (more info)
• .json - Download the data as a JSON table (missing value = 'null'; times are ISO 8601 strings). (more info)
• .mat - Download a MATLAB binary file. (more info)
• .tsv - Download the data as tab-separated ASCII text table (missing value = 'NaN'; times are ISO 8601 strings). (more info)
• .xhtml - View an XHTML file with the data in a table (times are ISO 8601 strings). (more info)

.csv and .tsv issues:

• If a datum in a .csv file has internal double quotes or commas, ERDDAP follows the .csv specification strictly: it puts double quotes around the datum and doubles the internal double quotes.
• If a datum in a .csv or .tsv file has internal newline characters, ERDDAP converts the newline characters to character #166 (¦). This is non-standard.

Requests for .json files may now include an optional jsonp request
by adding "&.jsonp=functionName" to the end of the query.
Basically, this just tells ERDDAP to add "functionName(" to the beginning of the response
and ")" to the end of the response.
If originally there was no query, leave off the "&" in your query.

Remember that these are the file types ERDDAP can use to respond to user-interface types of requests (for example, search requests).
ERDDAP supports a different set of file types for the actual data (for example, satellite and buoy data) requests (see griddap and tabledap).

Access URLs
ERDDAP has these URL access points for computer programs:

• The list of the main resource access URLs can be returned as a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• The current list of all datasets can be returned as a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• Info about a specific data set (the list of variables and their attributes) is returned by a call to
  http://coastwatch.pfeg.noaa.gov/erddap/info/<datasetID>/index<fileType> ,
  for example, a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• The results of full text searches for datasets can be returned as a (for example) .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
  (You probably need to percent-encode the values in the query (e.g., "searchFor=temperature%20wind%20speed").
• The list of categoryAttributes can be returned as a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• The list of categories for a specific categoryAttribute can be returned as a (for example) .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• The list of datasets in a specific category can be returned as a (for example) .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• The current list of datasets which can be accessed by the various protocols can be returned as
  • For griddap: a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
  • For tabledap: a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
  • For WMS: a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• The Data Access Forms are just simple web pages to generate URLs which request data (e.g., satellite and buoy data).
  The Make A Graph pages are just simple web pages to generate URLs which request graphs of a subset of the data.
  Your program can generate these URLs directly. For more information, read about griddap, tabledap, and WMS.
  griddap and tabledap can return data in many common file formats. See their documentation for details.
  Note that griddap and tabledap can return .dds files with the dataset's structure, including variable names.
  Note that griddap and tabledap can return .das files with all of the dataset's metadata.
  Or, you can get variable names and metadata via a call to
  http://coastwatch.pfeg.noaa.gov/erddap/info/<datasetID>/index<fileType> ,
  for example, a .csv, .htmlTable, .json, .mat, .tsv, or .xhtml document.
• ERDDAP offers email/URL and RSS subscriptions, so your computer program can be notified whenever a dataset changes.
• Need other types of information?
  The links above are just the first batch of links to provide access to information from ERDDAP.
  If you have suggestions for additional links, contact bob dot simons at noaa dot gov.

Using ERDDAP as a Data Source within Your Java Program

Or, since ERDDAP is an all-open source program, you can also set up your own copy of ERDDAP
on your own server (internal or public) to serve your own data.
Any Java programs that you write can get data from that copy of ERDDAP.

Or, since ERDDAP is an all-open source, all-Java program,
you can also set up your own copy of ERDDAP on your own computer within your Java program.
Then you can access ERDDAP's services from within your program
(for example, doing full-text searches for datasources or creating data files from remote data sources) .
See Set Up Your Own ERDDAP, especially the JavaDocs for the gov.noaa.pfel.erddap.Erddap class.
 
ERDDAP Version 1.22