Easier access to scientific data
Brought to you by NOAA NMFS SWFSC ERD
ERDDAP is a great example of User-Driven Innovation, where product innovation often comes from consumers (ERDDAP users), not just the producers (ERDDAP developers). Over the years, most of the ideas for new features and changes in ERDDAP have come from users. Those users are credited below for their great ideas. Thank you! Please keep those great suggestions coming!
Here are the changes associated with each ERDDAP release.
Not all text or all web pages were translated. There were time constraints on this project that prevented Qi and Bob from getting to 100%.
The obvious question is: why did we put so much effort into this when Chrome will translate web pages on-the-fly? The answer is: this way, we get much more control over how the translation is done. Notably, there are lots of words that shouldn't be translated on the web pages, e.g., the titles and summaries of datasets, the names of variables, parameters, units, and organizations. Much of the translation effort was identifying words and phrases that shouldn't be translated. Also, the machine translations tended to mangle certain types of HTML markup. Managing the translation allowed us to minimize this problem.
The translation project was done by Qi Zeng (a Google Summer of Code intern) and Bob Simons using Google's Translation web service. It was a huge project. Thanks, Qi!
<theShortDescriptionHtml><![CDATA[ <h1>ERDDAP</h1> &erddapIs; &thisParticularErddap; [standardShortDescriptionHtml] ]]></theShortDescriptionHtml>The last 3 lines of content are things that will be substituted with translated text. If you convert any of them (notably &thisParticularErddap;) or all of them to explicit text in datasets.xml (which has priority, if present) or messages.xml, that text will appear no matter what language the user selects. This is not perfect, but I figured that few administrators would want to edit <theShortDescriptionHtml> in 35 different files to provide 35 different translated versions of that tag.
<ipAddressMaxRequests></ipAddressMaxRequests> <!-- current default=7 --> <ipAddressMaxRequestsActive></ipAddressMaxRequestsActive> <!-- current default=2 --> <ipAddressUnlimited></ipAddressUnlimited> <!-- default=empty -->For further information, see ipAddressMaxRequests. ERDDAP also now prints the "Number of unique users (since startup)" on the status.html page.
It is possible that this tag will lead to other, similar diagnostic information being emailed to
erd.data at noaa.gov in the future.
The email's content will always be minimal and related to bugs, and not, for example, usage information.
Thanks to Marco Alba.
RECOMMENDED (because there are users who want this):
If you want to make all EDD...FromFiles datasets accessible via the files system, then
The latest version of Java 8 (not Java 9, 10, 11, ...) from AdoptOpenJDK remains the recommended version of Java for ERDDAP. Java 8 has Long Term Support from AdoptOpenJDK so it remains safe to use, but remember to get the latest version of it periodically for security reasons.
WARNING: For datasets that are affected by this change, it is possible that you will
see problems with the dataset because the data that ERDDAP reads from the source may be
different (e.g., variables previously read as signed integers may now be read as unsigned integers).
The resulting problems include: new files not being added to the dataset,
and/or errors when you try to access the data.
If a dataset has problems, the first thing to try is to
set a hardFlag for the dataset.
If that doesn't resolve the problem, then you have to look at log.txt to see the
error messages, delve into the datasets.xml for the dataset,
and/or perhaps rerun generateDatasets.xml for the dataset.
Thanks to netcdf-java 5.x (which forced the issue) and the upcoming CF 1.9.
WARNING: It is possible this will cause problems for some existing datasets (e.g., cause new files to be labeled "bad"). If so, set a hardFlag for the dataset so that all of the source files will be reread with the new system.
This tool is especially useful for datasets that you created some time ago and now want to bring up-to-date with your current metadata preferences. For example, early versions of GenerateDatasetsXml didn't put any effort into creating global creator_name, creator_email, creator_type, or creator_url metadata. You could use ERDDAP-lint to identify the datasets that lack those metadata attributes.
Thanks to Rob and Adam for creating this tool and making it available to the ERDDAP community.
There are also new statistics named "Large Request, IP address" which show the IP addresses of the users who made large requests (currently, gridded .nc files > 1GB).
Also, the time series table on the status.html page now includes a "memFail"
column showing the number of requests that failed with "OutOfMemory (Too Big)"
errors since the last major Load Datasets. Any number other than 0 here is
at least some cause for concern.
Thanks to Bob Simons.
I explored a different solution (storing strings in StringArray as UTF-8-encoded byte arrays). That reduces memory usage another ~33%, but at the cost of ~33% slowdown. Compared to the system that is now being used, that seemed like a bad trade off. It's easier to give a computer more memory (buy more memory for ~$200) than to make it faster (buy a whole new computer).
If it is convenient, it's still always a good idea to split huge tabular data files into several smaller files based on some criteria like stationID and/or time. ERDDAP will often only have to open one of the small files in response to a user's request, and thus be able to respond much faster.
I didn't clearly state before: you should just use standardizeWhat features when you actually need them (e.g., when different source files store time values in different ways), because some requests to datasets that use standardizeWhat will be processed a little slower.
<!-- The tags below are described in setupDatasetsXml.html. The defaults listed below are as of ERDDAP v2.00. --> <cacheMinutes></cacheMinutes> <!-- default=60 --> <decompressedCacheMaxGB></decompressedCacheMaxGB> <!-- default=10 --> <decompressedCacheMaxMinutesOld></decompressedCacheMaxMinutesOld> <!-- default=15 --> <drawLandMask></drawLandMask> <!-- "over" or "under" (default) --> <graphBackgroundColor></graphBackgroundColor> <!-- 0xAARRGGBB, default is 0xffccccff --> <loadDatasetsMinMinutes></loadDatasetsMinMinutes> <!-- usually=default=15 --> <loadDatasetsMaxMinutes></loadDatasetsMaxMinutes> <!-- default=60 --> <logLevel></logLevel> <!-- "warning" (fewest messages), "info" (default), or "all" (most messages) --> <nGridThreads></nGridThreads> <!-- default=1 --> <nTableThreads></nTableThreads> <!-- default=1 --> <partialRequestMaxBytes></partialRequestMaxBytes> <!-- default=490000000 --> <partialRequestMaxCells></partialRequestMaxCells> <!-- default=10000000 --> <slowDownTroubleMillis></slowDownTroubleMillis> <!-- default=1000 --> <unusualActivity></unusualActivity> <!-- default=10000 --> <!-- The defaults for the following tags are in messages.xml. --> <startHeadHtml5></startHeadHtml5> <startBodyHtml5></startBodyHtml5> <!-- This is often customized. --> <theShortDescriptionHtml></theShortDescriptionHtml> <!-- This is often customized. --> <endBodyHtml5></endBodyHtml5> <standardLicense></standardLicense> <standardContact></standardContact> <standardDataLicenses></standardDataLicenses> <standardDisclaimerOfEndorsement></standardDisclaimerOfEndorsement> <standardDisclaimerOfExternalLinks></standardDisclaimerOfExternalLinks> <standardGeneralDisclaimer></standardGeneralDisclaimer> <standardPrivacyPolicy></standardPrivacyPolicy>
If your value is different from the new suggested default (other than for <startBodyHtml5> and <theShortDescriptionHtml>, which are useful for customizing your ERDDAP installation), please consider switching to the new default values. This is particularly true of <partialRequestMaxBytes> and <partialRequestMaxCells>, where the default/suggested value has changed significantly over the years.
After you copy each value, delete the tag and its description from setup.xml. It is better to have these tags in datasets.xml. And there are now better descriptions in setupDatasetsXml.html.
A quirk of the new system is that the very first web page when you start up ERDDAP will be the default ERDDAP web page. Every subsequent webpage will use the ...Html content you specify in datasets.xml.
This works great!
In most cases, the slowdown related to decompressing the files is minor. We strongly encourage you to try this, notably for datasets and/or data files that are infrequently used.
This may save you $30,000 or more!
This is one of the few ERDDAP features that can save you lots of money -- if you compress a lot of data files, you will need far fewer RAIDs/hard drives to store the data, or conversely, you can serve far more data (up to 10x) with the RAIDs you already have. If this feature saves you from buying another RAID, then it has saved you about $30,000.
See the Externally Compressed Files documentation. Thanks to Benoit Perrimond and Paloma de la Vallee.
We recommend using the default <startHeadHtml5> and <endBodyHtml5>.
We recommend: If you made changes to the original <startBodyHtml> and/or want to customize your ERDDAP now, please copy the new <startBodyHtml5> tag (from below) into your setup.xml and modify it to customize your ERDDAP so that ERDDAP's web pages reflect your organization, not NOAA ERD. Notably, please change the "Brought to you by" to your organization(s). If you need help, please email bob.simons at noaa.gov. (If you don't want to customize your ERDDAP now, use the default <startBodyHtml5>.)
Then delete the 3 old tags in your setup.xml which are no longer used.
<startBodyHtml5><![CDATA[ <body> <table class="compact nowrap" style="width:100%; background-color:#128CB5;"> <tr> <td style="text-align:center; width:80px;"><a rel="bookmark" href="https://www.noaa.gov/"><img title="National Oceanic and Atmospheric Administration" src="&erddapUrl;/images/noaab.png" alt="NOAA" style="vertical-align:middle;"></a></td> <td style="text-align:left; font-size:x-large; color:#FFFFFF; "> <strong>ERDDAP</strong> <br><small><small><small>Easier access to scientific data</small></small></small> </td> <td style="text-align:right; font-size:small;"> &loginInfo; <br>Brought to you by <a title="National Oceanic and Atmospheric Administration" rel="bookmark" href="https://www.noaa.gov">NOAA</a> <a title="National Marine Fisheries Service" rel="bookmark" href="https://www.fisheries.noaa.gov">NMFS</a> <a title="Southwest Fisheries Science Center" rel="bookmark" href="https://www.fisheries.noaa.gov/about/southwest-fisheries-science-center">SWFSC</a> <a title="Environmental Research Division" rel="bookmark" href="https://www.fisheries.noaa.gov/about/environmental-research-division-southwest-fisheries-science-center">ERD</a> </td> </tr> </table> ]]></startBodyHtml5>
There are additional ways you can customize ERDDAP so ERDDAP's web pages reflect your organization instead of NOAA ERD.
If you didn't customize those tags, please delete them from your setup.xml file.
Now they all have defaults in messages.xml that refer to datasets
in Bob's ERDDAP at https://coastwatch.pfeg.noaa.gov/erddap/index.html .
So you no longer need to have specific datasets in your ERDDAP.
If you want to override the defaults, copy some or all of those tags
into your setup.xml and change their values.
If you want the examples to point to your ERDDAP, the easiest method is:
<dataset type="EDDGridFromErddap" datasetID="jplMURSST41" active="true"> <sourceUrl>https://coastwatch.pfeg.noaa.gov/erddap/griddap/jplMURSST41</sourceUrl> </dataset> <dataset type="EDDTableFromErddap" datasetID="pmelTaoDySst" active="true"> <sourceUrl>https://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTaoDySst</sourceUrl> </dataset>
If you did customize those tags, leave them as is and please add these 2 new tags to your setup.xml to specify the ERDDAP URL for these datasets, but change the URL to your ERDDAP's (https?) URL:
So, please check your datasets: if a variable's values are packed and
if actual_range is specified as packed data values,
please add an <addAttributes> actual_range value to specify the unpacked values.
Otherwise, the dataset will not load in ERDDAP.
A simple and almost perfect way to do this is to search your datasets.xml for
sourceAttributes that have
<att name="actual_range" type="shortList">
or <att name="actual_range" type="intList">
and a scale_factor other than 1.0. Those are the actual_range attributes that you might have to fix.
For axis variables in EDDGrid datasets, ERDDAP always sets the actual_range attribute to be the actual range of the values since it knows those values.
For axis variables with descending values (e.g., some latitude variables), ERDDAP created actual_range with the ...[last] values, which were high...low. Now it always uses low...high values to make the new CF definition.
The correctness of the actual_range values is particularly important for EDDTable datasets, because ERDDAP will quickly reject user requests for data values which are less than the actual_range minimum value or which are greater than the actual_range maximum value.
Related: the actual_min, actual_max, data_min and data_max attributes are now deprecated. Please convert your datasets to use actual_range instead.
The netcdf-java developers maintain that these vulnerabilities are not relevant because of the way that netcdf code uses these libraries and in any case would only be relevant when accessing Amazon S3. See https://github.com/Unidata/thredds/issues/866 . I believe them. If you still have concerns about this, please contact the netcdf-java developers. (Note that if you don't believe the netcdf-java developers and are contemplating not using ERDDAP because of this, you shouldn't use THREDDS either, because THREDDS uses netcdf-java more fundamentally and more extensively than ERDDAP.)
Details: The troublesome code and the vulnerability warnings are:
See https://nvd.nist.gov/vuln/detail/CVE-2016-7051 -- High
See https://nvd.nist.gov/vuln/detail/CVE-2016-7051 -- High
See https://nvd.nist.gov/vuln/detail/CVE-2016-7051 -- High
See https://nvd.nist.gov/vuln/detail/CVE-2016-3720 -- Critical
See https://nvd.nist.gov/vuln/detail/CVE-2016-7051 -- High
See https://nvd.nist.gov/vuln/detail/CVE-2016-3720 -- Critical
"For version 4.6.10, aws-java-sdk-core pulls in version 2.6.6 of jackson-* artifacts." (email from netcdf-java people).
Thanks to Charles Carleton and NCEI.
<att name="colorBarMaximum" type="double">8000.0</att> <att name="colorBarMinimum" type="double">-8000.0</att> <att name="colorBarPalette">TopographyDepth</att>
<!-- If you want to restrict access to some datasets, you need to specify the method used for logging on (authentication). See the info at https://coastwatch.pfeg.noaa.gov/erddap/download/setup.html#security Currently, the options are: "" (logins not supported, the default), "custom", "email", and "google" (recommended). [No longer supported: "basic", "openid"] -->
In your setup.xml, please change the description for <baseUrl> to be
<!-- baseUrl is the start of the public URL, to which "/erddap" is appended. For example: For running/testing on your personal computer: <baseUrl>http://localhost:8080</baseUrl> (127.0.0.1 doesn't work with authentication=google). If you want to encourage all users to use https (not http), make the baseUrl the same as the baseHttpsUrl (see below). For ERD releases, we used to use <baseUrl>http://coastwatch.pfeg.noaa.gov</baseUrl> For ERD releases, we now use <baseUrl>https://coastwatch.pfeg.noaa.gov</baseUrl> -->
<!-- For "custom" authentication, this specifies how you have stored passwords in the roles tags in datasets.xml. If you aren't storing any passwords, this is irrelevant. The options (in order of increasing security) are: "MD5", "UEPMD5" (MD5(UserName:ERDDAP:Password)), "SHA256", "UEPSHA256" (SHA256(UserName:ERDDAP:Password), the default). You should only use "MD5" or "SHA256" if you need to match values stored that way in an external password database. See the info at https://coastwatch.pfeg.noaa.gov/erddap/download/setup.html#security -->
<!-- This is a variant of <baseUrl> which is used when authentication is active and the user is logged in. In general, you take the <baseUrl>, change "http" to "https", and change/add ":8443". This must begin with "https://". If you make a proxy so that ":8443" isn't needed, then don't use ":8443" here. This is relevant even if <authentication> is "". See the instructions at https://coastwatch.pfeg.noaa.gov/erddap/download/setup.html#security For example: For running/testing on your personal computer: <baseHttpsUrl>https://localhost:8443</baseHttpsUrl> For releases at ERD, we use: <baseHttpsUrl>https://coastwatch.pfeg.noaa.gov</baseHttpsUrl> If you want to encourage all users to use https (not http), make the baseUrl (see above) the same as the baseHttpsUrl. -->
<!-- Normally, if you have a EDDGridFromErddap or EDDTableFromErddap dataset in your datasets.xml, it will try to subscribe to the remote ERDDAP dataset so that the local dataset is kept perfectly up-to-date. If this ERDDAP is not publicly accessible (http://localhost), or its IP address will change soon, or you have some other reason, you can tell this ERDDAP to not try to subscribe to the remote ERDDAP datasets by setting this to false. (default=true) This is the overall setting for this ERDDAP. It can be overridden by the same tag (with a different value) in the datasets.xml chunk for a given EDD...FromErddap dataset. For each fromErddap dataset that doesn't subscribe to the remote ERDDAP dataset, you should set <reloadEveryNMinutes> to a smaller number so that the local dataset stays reasonably up-to-date. --> <subscribeToRemoteErddapDataset>true</subscribeToRemoteErddapDataset>
We still recommend: If a dataset has a large number of files (e.g., >1,000), the operating system (and thus EDDGridFromFiles and EDDTableFromFiles) will operate much more efficiently if you store the files in a series of subdirectories (one per year, or one per month for datasets with very frequent files), so that there are never a huge number of files in a given directory.
<!-- If true, when you start up ERDDAP, some types of datasets (e.g., EDDGridFromDap) will used cached information (.dds, .das, etc.) to reload very quickly, without contacting the remote server. The dataset's age will be based on when the dataset was reloaded last. Normally this should be true (the default), but set it to false if you want to bypass the cached information. <quickRestart>true</quickRestart>
<!-- ERDDAP lets you choose between two search engines for full text searches: * original (the default) -- is the best choice if your ERDDAP has fewer than about 10,000 datasets. It is very robust and trouble free. * lucene -- is the best choice for more than about 10,000 datasets. The advantages are that with any number of datasets it works fast and uses very little memory. But there are many things that might go wrong with individual queries and with the whole system. And although its behaviour (the datasets it finds and the order that it ranks them) is almost identical to the original search engine, it has a few quirky, subtle, small differences. --> <searchEngine>original</searchEngine>
<!-- Tabledap Examples This group of settings is used to make examples for the tabledap documentation that appears at [baseUrl]/erddap/tabledap/documentation.html and elsewhere. If you include the erdGlobecBottle dataset in your datasets.xml (recommended), you don't need to change these. If you don't, you MUST change these before you make your ERDDAP public; otherwise, none of the examples will work! The new settings should be very similar to the defaults. If your ERDDAP won't serve any tabular datasets, use "NOT_APPLICABLE" for all of the entities. In .xml files like this, ampersand, lessThan, and greaterThan have to be HTML encoded as "&", "<", ">". --> <!-- This is the datasetID for an EDDTable dataset that is served by your ERDDAP. This dataset is used as the basis for all of the EDDGrid examples below. Ideally, it is a dataset that has longitude, latitude, and time variables (among others). ('time' allows for making a time series graph. 'latitude' and 'longitude' allow for making a map.) The dataset can have longitude values -180 to 180, or 0 to 360. --> <EDDTableIdExample>erdGlobecBottle</EDDTableIdExample> <!-- This is a comma-separated list of variables from the dataset. It is useful if it is "longitude,latitude,time," plus a data variable name. --> <EDDTableVariablesExample>longitude,latitude,time,bottle_posn,temperature1</EDDTableVariablesExample> <!-- This is the constraints example which is appended to EDDTableVariablesExample. --> <EDDTableConstraintsExample>&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z</EDDTableConstraintsExample> <!-- This is an example data query using an ISO-formatted time. You could generate your example via your dataset's Data Access Form in ERDDAP. --> <EDDTableDataTimeExample>longitude,latitude,time,bottle_posn,temperature1&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z</EDDTableDataTimeExample> <!-- This is an equivalent example data query, but which specifies time as seconds-since-1970-01-01. If you need to convert a date/time to "seconds since 1970-01-01", use https://coastwatch.pfeg.noaa.gov/erddap/convert/time.html --> <EDDTableDataValueExample>longitude,latitude,time,bottle_posn,temperature1&time>=1029542400&time<=1029788280</EDDTableDataValueExample> <!-- This is an example query which generates a graph. You could generate your example via your dataset's Make A Graph form in ERDDAP. --> <EDDTableGraphExample>bottle_posn,temperature1&time=2002-08-19T10:06:00Z&.draw=lines</EDDTableGraphExample> <!-- This is an example query which generates a map. In the default mapExample, temperature1, time, bottle_posn are useful because they appear in GoogleEarth with the .kml example and are ignored by the other image file types. --> <EDDTableMapExample>longitude,latitude,temperature1,time,bottle_posn&time>=2002-08-13T00:00:00Z&time<=2002-08-20T00:00:00Z&bottle_posn=1&.draw=markers&.marker=5|5</EDDTableMapExample> <!-- This is a Matlab example which uses data from the EDDTableGraphExample. Note the Matlab notation datasetName.variableName. --> <EDDTableMatlabPlotExample>plot(erdGlobecBottle.bottle_posn, erdGlobecBottle.temperature1)</EDDTableMatlabPlotExample>
Note that actual_range is unchanged: it may have low,high values or high,low values, since it is intended to indicate the range and the order of storage.
<!-- These default accessConstraints, fees, and keywords are used by the SOS, WCS, and WMS services. They can be overridden by "accessConstraints", "fees", "keywords" attributes in a dataset's global metadata. If a dataset that has an "accessibleTo" tag doesn't override "accessConstraints", then the default for "accessConstraints" is the "accessRequiresAuthorization" value. --> <accessConstraints>NONE</accessConstraints> <accessRequiresAuthorization>only accessible to authorized users</accessRequiresAuthorization> <fees>NONE</fees> <keywords>Earth science, oceans</keywords> <!-- This appears on the erddap/legal.html web page after the General Disclaimer. You can replace any of the [standardParts] with your own HTML. --> <legal><![CDATA[ [standardDisclaimerOfEndorsement] [standardDisclaimerOfExternalLinks] [standardPrivacyPolicy] [standardDataLicenses] ]]></legal> <!-- Specify the default units standard (e.g., "UDUNITS" (the default) or "UCUM") that you (the ERDDAP admin) are using to specify units. The value is case-sensitive. This is used by ERDDAP's SOS server to determine if the units need to be converted to UCUM units for WMS and SOS GetCapabilities responses. --> <units_standard>UDUNITS</units_standard> <!-- For the wms examples, pick one of your grid datasets that has longitude and latitude axes. The sample variable must be a variable in the sample grid dataset. The bounding box values are minx,miny,maxx,maxy. --> <wmsSampleDatasetID>erdBAssta5day</wmsSampleDatasetID> <wmsSampleVariable>sst</wmsSampleVariable> <!-- The bounding box values are minLongitude,minLatitude,maxLongitude,maxLatitude. Longitude values within -180 to 180, or 0 to 360, are now okay. --> <wmsSampleBBox>0,-75,360,75</wmsSampleBBox>
<!-- startHeadHtml has the start of the HTML document and the 'head' tags (starting at "<!DOCTYPE>", but not including "</head>") for all HTML web pages. This may include &erddapUrl;, which is expanded to be [baseUrl]/erddap (or [baseUttpsUrl]/erddap if the user is logged in). If your ERDDAP allows users to log in, all referenced image files, css files, etc. must be in [tomcat]/content/erddap/images or a subdirectory and must be referenced here with &erddapUrl;/images/[fileName]. favicon.ico is the image that browsers associate with your website. For more information, see https://en.wikipedia.org/wiki/Favicon . You can use your own favicon.ico file by putting it in [tomcat]/content/erddap/images. *** Optional: you can change the appearance of all of your ERDDAP's HTML pages by changing the CSS <style> settings below. For an example of a very different style, change the import reference to <tomcat>/content/erddap/images/erddapAlt.css *** If your CSS style includes links to files (e.g., images), that style information must be inline in the style tag below, after the 'import' line, not in the .css file. Put all of the (e.g., image) files in the [tomcat]/content/erddap/images directory (or a subdirectory) and reference them below starting with &erddapUrl;. Why? On ERDDAP https: web pages, *all* links should use "https:" (not "http:"); otherwise, most browsers consider the web page not fully secure. Because ERDDAP would use the same .css file for http: and https: web pages, the links within the .css file wouldn't switch between http: and https:. There doesn't seem to be a way around this other than using inline style information. --> <startHeadHtml><![CDATA[ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <title>ERDDAP</title> <link rel="shortcut icon" href="&erddapUrl;/images/favicon.ico"> <style type="text/css"> <!-- @import "&erddapUrl;/images/erddap.css"; --> </style> ]]></startHeadHtml> <!-- The tableCommonBGColor MUST be the same color as the table.commonBGColor in erddap.css above. Suggested is #f1ecd8. But if you use erddapAlt.css, change this to #e7dec5. --> <tableCommonBGColor>#f1ecd8</tableCommonBGColor> <!-- This is used, e.g., for the type=variable rows on the metadata info tables. --> <tableHighlightBGColor>#cceecc</tableHighlightBGColor>Thanks to POST, Hans Vedo, and Rick Blair.
<theShortDescriptionHtml><![CDATA[ <h1>ERDDAP</h1> This website (the Environmental Research Division's Data Access Program) aggregates scientific data from diverse local and remote sources and offers you a simple, consistent way to download subsets of the data in common file formats and make graphs and maps. This particular ERDDAP installation has oceanographic data (for example, data from satellites and buoys). [standardShortDescriptionHtml] ]]></theShortDescriptionHtml>Feel free to change this, particularly the last sentence in the first paragraph.
<!-- If you want to restrict access to some datasets, you need to specify the method used for logging on (authentication). See the info at https://coastwatch.pfeg.noaa.gov/erddap/download/setup.html#security Currently, the options are: "" (logins not supported, the default), "custom", "openid". Note that openid login doesn't work when testing with localhost (https://127.0.0.1:8443). --> <authentication></authentication> <!-- This specifies how you have stored passwords in the roles tags in datasets.xml. If you aren't storing any passwords this is irrelevant. The options (in order of increasing security) are: "plaintext", "MD5", or "UEPMD5" (MD5(UserName:ERDDAP:Password), the default). You should only use "plaintext" or "MD5" if you need to match values stored that way in an external password database. See the info at https://coastwatch.pfeg.noaa.gov/erddap/download/setup.html#security --> <passwordEncoding>UEPMD5</passwordEncoding> <!-- This determines whether datasets that the user doesn't currently have access to (because he isn't logged in or because his roles don't allow access) should be shown on lists of data sets (e.g., from full text search, categorize, view all datasets, ...). The options are: "true", or "false" (the default). If false, no information about the dataset (even its existence) is shown to users who don't have access to it. If true, some information about the dataset (title, summary, etc) is shown to users who don't have access to it. If the user clicks on a link to a dataset he doesn't have access to, he will get an error message and be prompted to log in. --> <listPrivateDatasets>false</listPrivateDatasets> <!-- If the number of requests between two runs of LoadDatasets exceeds unusualActivity, an email is sent to emailEverythingTo. The default is 10000. --> <unusualActivity>10000</unusualActivity>
There is nothing new about making a local copy of a dataset. What is new here is that these classes make it *easy* to create and *maintain* a local copy of data from a *variety* of types of remote data sources and *add metadata* while copying the data.
These dataset types are part of a complete set of features that simplify the creation of grids/clusters/federations of ERDDAPs to handle very heavy loads (e.g., in a data center).
<!-- ERDDAP has a service that lets remote users set a flag to notify ERDDAP to try to reload a dataset. These requests use a key which is generated based on baseUrl/warName, a datasetID, and flagKeyKey. *** Change this once, to any text (a favorite quote? random text? It doesn't matter). Normally, you won't ever change this again. But if you think someone is abusing the flag system, change this text again, restart ERDDAP, and send all of the users of the flag system the relevant new flagKeys (see the list in the Daily Report). --> <flagKeyKey>A stitch in time saves nine. CHANGE THIS!!!</flagKeyKey> <!-- ERDDAP has an email/URL subscription system which sends a user an email or pings a url whenever a dataset of interest changes. (This is different from the RSS system, which is always active.) The system relies on the server being able to send out emails to people to validate their subscription requests. The emails appear to come from the emailFromAddress below. So if your server can't send out emails, don't make this system active. You may choose (for whatever reason) to make this system active or not, so valid values below are "true" (the default) and "false". Note that if you change this and restart ERDDAP, the list of subscriptions (in [bigParentDirectory]/subscriptionsV1.txt) isn't affected. See also the subscriptionEmailBlacklist in datasets.xml. --> <subscriptionSystemActive>true</subscriptionSystemActive>
<!-- The bounding box values are minLongitude,minLatitude,maxLongitude,maxLatitude. Longitude values within -180 to 180, or 0 to 360, are now okay. --> <wmsSampleBBox>0,-75,360,75</wmsSampleBBox>
<!-- drawLand specifies the default Make A Graph setting for whether the landmask should be drawn "over" (the default) or "under" surface data on maps. "over" is recommended for primarily oceanographic data (so that grid data over land is obscured by the landmask). "under" is recommended for all other data. --> <drawLand>over</drawLand> <!-- Information about the ERDDAP administrator is used for the SOS and WMS servers. You MUST CHANGE these to describe your installation. --> <adminInstitution>NOAA Environmental Research Division</adminInstitution> <adminIndividualName>Your Name</adminIndividualName> <adminPosition>Webmaster</adminPosition> <adminPhone>your-phone-number</adminPhone> <adminAddress>99 Pacific St, Suite 255A</adminAddress> <adminCity>Monterey</adminCity> <adminStateOrProvince>CA</adminStateOrProvince> <adminPostalCode>93940</adminPostalCode> <adminCountry>USA</adminCountry> <adminEmail>yourName@yourSite</adminEmail> <!-- Information about the ERDDAP administrator is used for ERDDAP's SOS server. You MUST CHANGE these to describe your installation. --> <sosTitle>NOAA Environmental Research Division SOS</sosTitle> <sosAbstract>NOAA Environmental Research Division's ERDDAP makes data from multiple sources available via the SOS protocol.</sosAbstract> <sosKeywords>Weather, Ocean Currents, Temperature, Salinity</sosKeywords> <sosAccessConstraints>NONE</sosAccessConstraints> <sosFees>NONE</sosFees> <!-- Information about the ERDDAP administrator is used for ERDDAP's WMS server. You MUST CHANGE these to describe your installation. --> <wmsTitle>NOAA Environmental Research Division WMS</wmsTitle> <wmsAbstract>NOAA Environmental Research Division's ERDDAP makes data from multiple sources available via the WMS protocol.</wmsAbstract> <wmsKeywords>Weather, Ocean Currents, Temperature, Salinity</wmsKeywords> <wmsAccessConstraints>NONE</wmsAccessConstraints> <wmsFees>NONE</wmsFees> <!-- For the wms examples, pick one of your grid datasets that has longitude and latitude axes. The sample variable must be a variable in the sample grid dataset. The bounding box values are minx,miny,maxx,maxy. --> <wmsSampleDatasetID>erdBAssta5day</wmsSampleDatasetID> <wmsSampleVariable>sst</wmsSampleVariable> <wmsSampleBBox>0,-75,180,75</wmsSampleBBox>
you can join the ERDDAP Google Group / Mailing List by visiting
and clicking on "Apply for membership".
Once you are a member, you can post your question there
or search to see if the question has already been asked and answered.