Easier access to scientific data
Brought to you by NOAA NMFS SWFSC ERD    

The EDDTableFromEML and EDDTableFromEMLBatch Options in GenerateDatasetsXml

[This web page will only be of interest to ERDDAP administrators who work with EML files.]

ERDDAP is a data server that gives users a simple, consistent way to download subsets of gridded and tabular scientific datasets in common file formats and make graphs and maps. ERDDAP works with a given dataset as either a group of multidimensional gridded variables (e.g., satellite or model data) or as a database-like table (with a column for each type of information and a row for each observation). ERDDAP is Free and Open Source Software, so anyone can download and install ERDDAP to serve their data.

To add a dataset to an ERDDAP installation, the ERDDAP administrator must add a chunk of XML describing the dataset to a file called datasets.xml. (There is thorough documentation for datasets.xml.) Although it is possible to generate the chunk of XML for datasets.xml entirely by hand, ERDDAP comes with a tool called GenerateDatasetsXml which can generate the rough draft of the chunk of XML needed for a given dataset based on some source of information about the dataset.

The first thing GenerateDatasetsXml asks is what type of dataset you want to create. GenerateDatasetsXml has a special option, EDDTableFromEML, which uses the information in an Ecological Metadata Language (EML) (external link) XML file to generate the chunk of XML for datasets.xml to create an EDDTableFromAsciiFiles dataset from each data table in an EML file. This works very well for most EML files, mostly because EML files do an excellent job of storing all of the needed metadata for a dataset in an easy-to-work-with format. The information that GenerateDatasetsXml needs to create the datasets is in the EML file, including the URL for the data file, which GenerateDatasetsXml downloads, parses, and compares to the description in the EML file. (Many groups would do well to switch to EML, which is a great system for documenting any tabular scientific dataset, not just ecological data. And many groups that create XML schemas would do well to use EML as a case study for XML schema that are clear, to the point, not excessively deep (i.e., too many levels), and easy for humans and computers to work with.)


Here are all the questions GenerateDatasetsXml will ask, with comments about how you should answer if you want to process just one EML file or a batch of EML files:

EML plus ERDDAP is a great combination, since ERDDAP can give users more direct access to the wealth of Knowledge Network for Biocomplexity (KNB) (external link) and Long Term Ecological Research (LTER) (external link) data and help those projects meet the US government's Public Access to Research Results (PARR) requirements by making the data available via a web service. Also, EML plus ERDDAP seems like a great bridge between scientists in the academic / NSF-funded realm and scientists in the federal agency (NOAA, NASA, USGS) realm.

If you have questions, comments, suggestions, or need help, please send an email to bob dot simons at noaa dot gov .

Design Details

Here are the design details of the EDDTableFromEML option in GenerateDatasetsXml.
Some are related to differences in how EML and ERDDAP do things and how the converter deals with these problems.
If one of these was a problem, it is now mostly dealt with/solved.

Issues with the EML Files

Here are some issues/problems with the EML files that cause problems when a software client (such as the EDDTableFromEML option in GenerateDatasetsXML) tries to interpret/process the EML files. Here are the issues:


Questions, comments, suggestions? Please send an email to bob dot simons at noaa dot gov .

ERDDAP, Version 1.82
Disclaimers | Privacy Policy