CASSIOPE SPACECRAFT (Swarm-E)
PROCESSED DATA HANDBOOK
eDEx API Documentation *BETA*
* eDEx API is currently in Beta*
Based on simple REST principles, the eDEx Web API endpoints accept and return JSON data based on the given filters and constraints from the eDEx database. The user provides the names of the products desired with appropriate filters and is returned a list of filenames. The user can then confirm these are the files desired and use them in their next API request to receive a prepackaged zip file, which is emailed to the user in a temporary download link. We recognize that not all data product requests are large enough that they warrant this pre-packaging and email process, therefore we will be making direct download available in the future.
The base address of the eDEx Web API is https://edex.phys.ucalgary.ca/api. The API provides a set of endpoints, each with its own unique path.
Requests
The eDEx Web API is based on REST principles. Data resources are accessed via standard HTTPS requests in UTF-8 format to an API endpoint passing data in JSON format. eDEx Web API uses the appropriate HTTP verbs for each action:
| METHOD | ACTION |
| GET | Retrieves resources (obtaining filename list) |
| POST | Creates resources (create zip file to be downloaded) |
Response
eDEx Web API responses will also be a JSON string. See the reference documentation below to find descriptions of common responses from each endpoint.
End Points
| Endpoint | Description |
| /api_fetch_data | Takes a JSON string of product names and constraints, then returns a JSON containing a list of the individual product filenames filtered on the parameters given |
| /api_prepare_report | Takes a JSON string of product names, product filenames, and constraints, then retrieves the files from the database, zips them, and creates a temporary download link, which is emailed to the user |
| /get_instrument_products | Returns a JSON string list of all e-POP data products |
| /get_experiment_tags | Returns a JSON string list of all e-POP experiments |
| /get_spacecraft_constraint_fields | Returns a JSON string list of all e-POP constraints |
| /get_geophysical_constraint_fields | Returns a JSON string list of all geophysical constraints |
Python implementation
An example Python program that should make things easy. It makes a call to /api_fetch_data with all the JSON parameters in a dictionary. A list of files is then returned to the command prompt for you to confirm those are the files you wish to download. If you wish to proceed with these files you may by pressing enter. A call to /api_prepare_report is then made. The dictionary is sent again with the returned results from /api_fetch_data updated and you should then receive a success message, advising you will be receiving your download link email shortly.
An example shell script program is also available and included in the zip file.
How to download data with eDEx Web API
The common eDEx Web API process for obtaining product files:
- Send data product names and constraint filters in JSON format to the “/api_fetch_data” endpoint and receive the returned JSON list of filenames in “result_filenames”
- Confirm the individual data product filenames returned are the files you wish to download
- Send the data product filenames from “result_filenames”, your email and once again include the product and constraint filters (that were used in step 1) in JSON format to “/api_prepare_report” to have the files zipped and a temporary download link created.
- The temporary download link will be automatically emailed to you once your files are ready.
1: GET request data product filenames
A simple curl request:
curl https://edex.phys.ucalgary.ca/api_fetch_data -X GET -d '{"product": ["cassiope-orbit-geometry"], "date": [["2020-06-01", "2020-06-05"]], "query-logic": " 1 "}' -H 'Content-Type:application/json'Parameters:
- Curl – Command line tool for transferring data using various network protocols
- URL – edex url with API endpoint
- -X – Specifies a custom request method when communicating with the HTTP server
- -d – Sends the specified data in the request to the HTTP server
- -H – Specifies the HTTP header to be sent along with the request. Here we specify the data we’re sending as JSON data.
JSON string: All parameters available to be used in the -d JSON string are listed below in the “API Products and Constraints” section.
Response: The response will be a JSON string with a “result_filenames” property containing a list of all the returned filenames, or a JSON string with an “error” property whose value is a dictionary of the error details.
e.g. A successful response:
{"result_filenames": ["CAS_Orbit_GEO_20200601T000000_20200601T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200602T000000_20200602T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200603T000000_20200603T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200604T000000_20200604T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200605T000000_20200605T235959_1.1.0.sp3.zip"]}e.g. A failed response:
{"error":{"endpoint":"/api_fetch_data","message":"Parameter "cassiope-orbit-eometry" not found.","status_code":404}}2. Confirm data product file names
The GET request will return a list of product filenames in the “result_filenames” property. Confirm this list is what you are looking for as the list may be large. This list can be passed to the next POST request.
3. Post request to have the data files zipped and a download link created
A simple curl request:
curl https://edex.phys.ucalgary.ca/api_prepare_report -X POST -d '{"product": ["cassiope-orbit-geometry"], "date": [["2020-06-01", "2020-06-05"]], "email": "myemail@phys.ucalgary.ca", "result_filenames": ["CAS_Orbit_GEO_20200601T000000_20200601T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200602T000000_20200602T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200603T000000_20200603T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200604T000000_20200604T235959_1.1.0.sp3.zip", "CAS_Orbit_GEO_20200605T000000_20200605T235959_1.1.0.sp3.zip"], "query-logic": " 1 "}' -H 'Content-Type:application/json'Parameters:
- Curl – Command line tool for transferring data using various network protocols
- URL – edex url with API endpoint
- -X – Specifies a custom request method when communicating with the HTTP server
- -d – Sends the specified data in the request to the HTTP server
- -H – Specifies the HTTP header to be sent along with the request. Here we specify the data we’re sending as JSON data.
JSON string: All parameters available to be used in the -d JSON string are listed below in the “API Products and Constraints” section.
Response: If all goes well you will receive a message “Preparing report… You will receive an email from esoc@phys.ucalgary.ca shortly”. If all goes poorly you will receive an error response just like in step 1.
4. Download link emailed
After the post request is sent all the filenames specified are zipped in a file, which will then be emailed to you from esoc@phys.ucalgary.ca. The email will contain a brief summary of what your query was and a temporary download link. In the zip file, you will also find a text file of the summary database query and the list of files included. If any files are not found, there will be a “missing_files.txt” file detailing those files.
API Products and Constraints
The JSON string is made up of the products desired, and a number of constraints to filter those products. A “product” property, at least one constraint to filter on (e.g. “date”) and the “query-logic” property must be included when communicating with both /api_fetch_data and /api_prepare_report endpoints. The /api_prepare_report endpoint also requires an “email” property, and a “result_filenames” property.
Query Logic property
The query-logic property is the logic of how you would like the API to process the constraints. It is a string of a series of “AND”s and “OR”s, which can be used to union or intersect the given parameter constraints. e.g. ( 1 AND ( 2 OR 3 ) ). The numbers coincide to the order of constraints given, products not included.
Example:
{"product": ["cassiope-orbit-geometry", "cassiope-attitude-quaternions"], "date": [["2020-06-01", "2020-08-01"], ["2020-11-01", "2021-01-01"]], "swarm": ["swarm-a", "swarm-b", "swarm-c"], "query-logic": " ( 1 OR 2 ) AND ( 3 OR 4 OR 5 ) "}This JSON is requesting data products “cassiope-orbit-geometry” and “cassiope-attitude-quaternions”, which do not count towards the query-logic string. It is filtering on date ranges 2020-06-01 to 2020-08-01 unioned with 2020-11-01 to 2021-01-01, “( 1 OR 2 )” in the query-logic property. It intersects either of those dates with the union of “swarm-a”, “swarm-b”, or “swarm-c”, “AND ( 3 OR 4 OR 5)” in the query-logic property. So it will return all “cassiope-orbit-geometry” and “cassiope-attitude-quaternions” files in between 2020-06-01 to 2020-08-01 and 2020-11-01 to 2021-01-01 that also have any of “swarm-a”, “swarm-b”, or “swarm-c” in it.
Products
| “product”: [ | “cassiope-orbit-geometry”, |
| “cassiope-attitude-quaternions”, | |
| “cassiope-bus-telemetry”, | |
| “epop-quicklook”, | |
| “epop-ephemeris”, | |
| “cer-quicklook”, | |
| “cer-tec”, | |
| “cer-zip”, | |
| “fai-quicklook”, | |
| “fai-summary”, | |
| “fai-png”, | |
| “fai-hdf5”, | |
| “gap-quicklook”, | |
| “gap-lv1”, | |
| “gap-observation”, | |
| “irm-quicklook”, | |
| “irm-summary”, | |
| “irm-lv1-skin-current”, | |
| “irm-surface-sensor-current”, | |
| “irm-lv0b”, | |
| “mgf-quicklook”, | |
| “mgf-summary”, | |
| “mgf-residual”, | |
| “mgf-low-resolution-cdf”, | |
| “mgf-high-resolution-cdf”, | |
| “nms-lv0b”, | |
| “nms-quicklook”, | |
| “nms-quicklook-velocity”, | |
| “nms-quicklook-position”, | |
| “rri-quicklook”, | |
| “rri-hdf5”, | |
| “sei-quicklook”, | |
| “sei-summary”, | |
| “sei-lv0b”] |
Time Constraints
| “date”: [ | [“2013-09-30′, ‘2023-01-05”]] |
Geophysical Constraints
| ‘geophysical-parameters’: [ | [‘ae’, ‘0’, ‘2000’], |
| [‘ap’, ‘0’, ’35’], | |
| [‘dst’, ‘-600’, ‘100’], | |
| [‘f10.7’, ‘0’, ‘300’], | |
| [‘kp10’, ‘0’, ’90’]] | |
| ‘imf-parameters’: [ | [‘absolute-b’, ‘0’, ’50’], |
| [‘by-gsm’, ‘-50′, ’50’], | |
| [‘bz-gsm’, ‘-50′, ’50’], | |
| [‘flow-pressure’, ‘0’, ’20’], | |
| [‘flow-speed’, ‘0’, ‘900’], | |
| [‘proton-density’, ‘0’, ‘100’]] |
e-POP Constraints
| ‘e-pop-quicklook’: [ | ‘has-data’] |
| ‘cer’: [ | ‘has-data’, |
| [“receiver-site”, “arecibo”], | |
| [‘receiver-site’, ‘cuzco’], | |
| [‘receiver-site’, ‘eureka’], | |
| [‘receiver-site’, ‘jicamara’], | |
| [‘receiver-site’, ‘siefring’], | |
| [‘receiver-site’, ‘pmaldonad’], | |
| [‘receiver-site’, ‘delta-junct’], | |
| [‘receiver-site’, ‘gakon-ak’], | |
| [‘receiver-site’, ‘valdez-ak’]] | |
| ‘fai’: [ | ‘has-data’, |
| [‘camera-mode’, ‘high-res’], | |
| [‘camera-mode’, ‘med-res’], | |
| [‘camera-mode’, ‘low-res’], | |
| [‘camera-source’, ‘infrared’], | |
| [‘camera-source’, ‘visible’]] | |
| ‘gap’: [ | ‘has-data’, |
| [‘observation-rate’, ‘100hz’], | |
| [‘observation-rate’, ’50hz’], | |
| [‘observation-rate’, ’20hz’], | |
| [‘observation-rate’, ‘1hz’], | |
| [‘receiver-number’, ‘0’], | |
| [‘receiver-number’, ‘1’], | |
| [‘receiver-number’, ‘2’], | |
| [‘receiver-number’, ‘3’], | |
| [‘receiver-number’, ‘4’], | |
| [‘receiver-number’, ‘gap-a’], | |
| [‘receiver-number’, ‘gap-o’]] | |
| ‘irm’: [ | ‘has-data’] |
| ‘mgf’: [ | ‘has-data’] |
| ‘nms’: [ | ‘has-data’] |
| ‘rri’: [ | ‘has-data’, |
| [‘antenna’, ‘monopole’], | |
| [‘antenna’, ‘dipole’], | |
| [‘bandwidth’, ’30khz’], | |
| [‘bandwidth’, ‘5 KHz’], | |
| [‘dwell-a’, ‘0’, ’60’], | |
| [‘dwell-b’, ‘0’, ’60’], | |
| [“frequency”,”0″,”18.5″], | |
| [‘gain’, ‘high’], | |
| [‘gain’, ‘medium’], | |
| [‘gain’, ‘low’], | |
| [‘output-channels’, ‘i1-q1-i3-q3’], | |
| [‘output-channels’, ‘i1-i2-i3-i4’], | |
| [‘sweep’, ‘fixed’], | |
| [‘sweep’, ‘linear’], | |
| [‘sweep’, ‘logarithmic’]] | |
| ‘sei’: [ | ‘has-data’] |
| ‘spacecraft-position’: [ | [‘altitude’, ‘300’, ‘1500’], |
| [‘geographic-latitude’, ‘-81′,’81’], | |
| [‘geographic-longitude’, ‘-180’, ‘180’], | |
| [‘magnetic-local-time’, ‘0’, ’24’]] | |
| ‘spacecraft-attitude’: [ | [‘nadir-pointing’, ‘true’], |
| [‘nadir-pointing’, ‘false’], | |
| [‘pitch’, ‘-90′, ’90’], | |
| [‘roll’, ‘-180’, ‘-180’], | |
| [‘yaw’, ‘-180’, ‘-180’]] |
Planned Experiment Constraints
| ‘amateur-radio’: [ | ‘arrl-field-day’, |
| ‘ham-radio’] | |
| ‘auroral-region’: [ | ‘auroral-tilt’, |
| ‘auroral-zone’] | |
| ‘cusp-region’: [ | ‘cusp-region’] |
| ‘earth-limb-viewing’: [ | ‘earth-limb’] |
| ‘gap-orbit-determination’: [ | ‘gap-od’] |
| ‘miscellaneous’: [ | ‘rri-noise-survey’, |
| ‘rri-special-event’, | |
| ‘rri-vlf’, | |
| ‘churchill-vlf’, | |
| ‘lucky-lake-vlf’, | |
| ‘whistler-detection’, | |
| ‘flash’, | |
| ‘kilauea-eruption’, | |
| ‘lucky-lake-vlf-dmsp-satellite’, | |
| ‘rri-noise-survey-dmsp-satellite’, | |
| ‘he+-detection’, | |
| ‘lucky-lake-vlf-clyde-river-superdarn’, | |
| ‘gap-uncompressed’, | |
| ‘lucky-lake-vlf-dsx-conjunction’, | |
| ‘rri-nw-hemisphere’, | |
| ‘bbc-documentary’, | |
| ‘science-campaign’] | |
| ‘point-of-interest’: [ | ‘san-francisco’, |
| ‘revelstoke’, | |
| ‘portsmouth’, | |
| ‘hurricane-dorian’, | |
| ‘fort-mcmurray-fire’, | |
| ‘calgary’, | |
| ‘boston’, | |
| ‘hurricane-sandra’] | |
| ‘polar-cap-region’: [ | ‘polar-cap’] |
| ‘rocket-satellite-conjunctions’: [ | ‘dmsp-satellite’, |
| ‘cygnus-spacecraft’, | |
| ‘cluster-satellite’, | |
| ‘dsx-conjunction’, | |
| ‘ici-4-rocket’, | |
| ‘ici-5-rocket’, | |
| ‘renu2-rocket’, | |
| ‘techdemosat-satellite’, | |
| ‘zacube-1-satellite’] | |
| ‘saa-region’: [ | ‘saa-zone’, |
| ‘sei-saa’] | |
| ‘slew-targets’: [ | ‘poker-flat’, |
| ‘fort-smith’, | |
| ‘ice-bear-radar’] | |
| ‘solar-eclipse’: [ | ‘solar Eclipse’, |
| ‘solar-eclipse-buenos-aires’] | |
| ‘swarm’: [ | ‘swarm-a’, |
| ‘swarm-b’, | |
| ‘swarm-c’] | |
| ‘terrestrial-receiver’: [ | ‘autumn-x-receiver’, |
| ‘lsbb-receiver’, | |
| ‘carisma-receiver’, | |
| ‘above-receiver’] | |
| ‘terrestrial-transmitter’: [ | ‘tiger-superdarn’, |
| ‘rothr’, | |
| ‘wwv-radio-station’, | |
| ‘wake-island-transmitter’, | |
| ‘toolik-lake-transmitter’, | |
| ‘dho38-radio-station’, | |
| ‘bpm-time-signal-transmitter’, | |
| ‘kharkiv-isr’, | |
| ‘cyprus-transmitter’, | |
| ‘codar’, | |
| ‘clyde-river-superdarn’, | |
| ‘risr-transmitter’, | |
| ‘clyde-river-superdarn-saa-zone’, | |
| ‘chu-radio-station’, | |
| ‘cambridge-bay-cadi’, | |
| ‘arecibo-transmitter’, | |
| ‘surface-wave-radar’, | |
| ‘sura-heater’, | |
| ‘sondrestrom-radar’, | |
| ‘saskatoon-superdarn’, | |
| ‘isr-world-day’, | |
| ‘new-mexico-transmitter’, | |
| ‘nlk-radio-station’, | |
| ‘nwc-radio-station’, | |
| ‘ottawa-nrcan-transmitter’, | |
| ‘millstone-hill-transmitter’, | |
| ‘kodiak-superdarn’, | |
| ‘polardarn’, | |
| ‘prince-george-superdarn’, | |
| ‘rankin-inlet-superdarn’, | |
| ‘naa-transmitter’, | |
| ‘resolute-bay-transmitter’, | |
| ‘istok’, | |
| ‘hawaii-transmitter’, | |
| ‘hankasalmi-superdarn’, | |
| ‘haarp’, | |
| ‘eiscat-tromso’, | |
| ‘eiscat-svalbard’, | |
| ‘drdc’] |
Valid JSON Examples
Example 1
{"product": ["cassiope-orbit-geometry", "cassiope-attitude-quaternions", "gap-observation"],
"date": [["2020-06-01", "2020-08-01"], ["2020-11-01", "2021-01-01"]],
"swarm": ["swarm-a", "swarm-b", "swarm-c"],
"email": "myemail@phys.ucalgary.ca",
"result_filenames": [],
"query-logic": " ( 1 OR 2 ) AND ( 3 OR 4 OR 5 ) "
}
# To be used with /api_fetch_data to obtain "result_filenames".Example 2
{"product": ["cer-quicklook","fai-quicklook"],
"date": [["2018-05-01", "2018-05-10"]],
"fai": [["camera-mode", "med-res"],["camera-mode", "high-res"]],
"spacecraft-position": [["geographic-latitude", "-81", "-60"],["geographic-latitude", "60", "81"]],
"email": "myemail@phys.ucalgary.ca",
"result_filenames": ["FAIql_20180501_014343_015145.mp4", "FAIql_20180502_180943_181845.mp4", "FAIql_20180502_195143_195945.mp4", "FAIql_20180503_174443_175414.mp4", "FAIql_20180503_192643_193540.mp4", "FAIql_20180504_190443_191015.mp4", "FAIql_20180504_204643_205215.mp4", "FAIql_20180504_222913_223345.mp4", "FAIql_20180505_165913_170345.mp4", "FAIql_20180505_184013_184515.mp4", "FAIql_20180506_163443_163945.mp4", "FAIql_20180506_232217_232619.mp4", "FAIql_20180507_211513_212045.mp4", "FAIql_20180508_140513_141145.mp4", "FAIql_20180510_200043_200515.mp4", "FAIql_20180510_214243_214715.mp4"],
"query-logic": " ( 1 AND ( 2 OR 3 ) AND ( 4 OR 5 ) ) "
} # To be used with /api_prepare_report to create a zip file and temporary download linkReminder: “email” and “result_filenames” are only required for the /api_prepare_report endpoint, but won’t hurt if they are included in the /api_fetch_data endpoint, though “result_filenames” would be null as that information is not available until returned from /api_fetch_data.