Heliophysics Events Registry web API
Introduction
http://www.lmsal.com/helio-informatics/hpkb/images/HER_arch.jpg
Both Solarsoft and the [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html iSolSearch] web application lets users query the Heliophysics Events Registry (HER) in an interactive manner (much like Google Maps). iSolSearch interacts with HER by means of our web application programming interface (API). The API enables developers to create new third party clients for HER as well as scientists to write software code which interact with HER in an automated manner.
Usage of an API function amounts to using HTTP to fetch a web document. The base url for all API calls is (including the question mark)
http://www.lmsal.com/her/dev/search-hpkb/hek?
Following the url base, one can append a list of keywords to specify the type of command and options for that specific command.
Simple Search for Features/Events in HER
To do a simple search for Features and Events in HER, one must append to the base url:
cmd=search&type=column
In addition to the command (cmd), one must specify which event type to search for. For all event types, use **
cmd=search&type=column&event_type=**
For some but not all event types, use a comma delimited list of two-letter abbreviations (see http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html for full list of event types). E.g. the following specifies that one wishes to search for active regions, flares and emerging flux regions:
cmd=search&type=column&event_type=ar,fl,ef
One must also provide a time window using event_starttime and event_endtime
cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00
Only entries with duration fully contained within this time will be returned.
The spatial region must also be specified. At present, we support searches by helioprojective (from Earth's perspective, in arcseconds from disk center), Stonyhurst (longitude measured from central meridian and latitude in degrees) and Carrington (longitude and latitude in degrees). Let (x1,y1) and (x2,y2) be the lower-left and upper-right coordinates of a bounding box respectively. To get everything within the disk, it suffices to specify
cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200
To specify a bounding box in Stonyhurst coordinates, give the bounding longitude and latitude in degrees. For the full sphere, do
cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&event_coordsys=stonyhurst&x1=-180&x2=180&y1=-90&y2=90
To get query results in XML form, set cosec=1. To get JSON, set cosec=2.
The query for this example with XML output would then be
http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&cosec=1&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200
To show the search results in iSolSearch, simply point your browser to the iSolSearch link with and passing the full query as a hek_query parameter. For instance, click [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html?hek_query=http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&cosec=1&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200 here].
Customizing Searches with Filters
Searches can be customized with filters based on keywords. For a list of keywords/parameters for the various event and feature types, see http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html. A filter is a triplet consisting of the Parameter name (param), the comparison operation (op) and a value (value).
Filters on attributes attributes of type string
Suppose you wish to search for all events submitted by a particular Feature Recognition Method (FRM; could be person or a software code). The API allows you to specify the FRM_Name as a string. At present the API supports seven comparison operations for strings, namely =, != >, <, >=, <= and like.
Greater than or less than operations on string values refers to alphabetical ordering.
To specify the exact FRM_Name, simply append the following string to the query url
¶m0=FRM_Name&op0==&value0=Karel%20Schrijver
The example above tells the API to limit results to those with FRM_Name matching Karel Schrijver. The strange looking string %20 is simply the url-escaped code for a whitespace.
Instead of specify the exact string, one can also use a wildcard. For instance, if you wish to search for entries in the registry submitted by FRM's with first name equal to Karel, use the like operator with a '%' sign as a wildcard after Karel
¶m0=FRM_Name&op0=like&value0=Karel%25
The string %25 is simply the url-escaped code for a percent sign.
To add a second filter, simply define another triplet. This time, the number after param, op and value is 1.
¶m0=FRM_Name&op0==&value0=Karel%20Schrijver¶m1=OBS_Observatory&op1==&value1=TRACE
These two filters limit search results to those events/features reported by Karel Schrijver using data from the TRACE observatory.
The full query url for a search with these two filters for the time period October 2003 is then
http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2003-10-01T00:00:00&event_endtime=2003-11-01T00:00:00&cosec=1&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200¶m0=FRM_Name&op0==&value0=Karel%20Schrijver¶m1=OBS_Observatory&op1==&value1=TRACE
To see the results in iSolSearch, click on this [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html?hek_query=http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2003-10-01T00:00:00&event_endtime=2003-11-01T00:00:00&cosec=1&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200¶m0=FRM_Name&op0==&value0=Karel%20Schrijver¶m1=OBS_Observatory&op1==&value1=TRACE link]
To find Flares with GOES class X and above, use this filter
¶m0=FL_GOESCls&op0=%3E&value0=X
The string %3E is the url-escaped code for the > character.
Filters on attributes of type boolean
The API supports the equality (=) and inequality (!=) logical operations on attributes of type boolean.
E.g., for attributes such as FRM_HUMANFLAG, one should set op0== and value0=true. Note that value0=t will not work if you want to get HER entries with true values for a particular boolean attribute.
Requested behaviour: t=true=1. f=false=0
Filters on attributes of type integer/float
The API supports six comparison operations for such attributes, namely =, !=, >, <, >= and <=. For example, to get Active Regions with NOAA number between 10930 and 10934 (inclusive), use the following filter:
¶m0=AR_NOAANum&op0=%3E=&value0=10930¶m1=AR_NOAANum&op1=%3C=&value1=10934
One can learn to add filters to queries in [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html iSolSearch] by click on the Filters tab.
Maximum number of returned results
By default the maximum number of results given a query is 100. To change this limit, set
result_limit=Num
where Num is the maximum number of results to return. There is a currently a hard limit of 200 results. In other words, specifying a value of 500 will return 200 results maximum. The results are returned sorted by reverse chronological order (Based on event_starttime). To avoid unreasonably time-consuming requests from hoarding API resources, a search will timeout after 60s. In that case, please breakup you query into small chunks and perform searches incrementally.
Output format of queries
To get query results in XML form, include the string &cosec=1 in the query url. To get JSON, use &cosec=2.
The xml output is looks like this
http://www.lmsal.com/helio-informatics/hpkb/images/cosec1output.jpg
The JSON output looks like this
http://www.lmsal.com/helio-informatics/hpkb/images/cosec2output.jpg
If JSON output is requested, an optional parameter can be specified in the url to wrap the JSON with a call to a Javascript callback function. For instance, suppose the web client you are developing requests JSON from HER. If you want to handle the JSON with some function, say it's named processJSON, then simply provide the optional parameter 'callback=processJSON' in the query url. With this option, the output will have the string processJSON( ); wrapped around the JSON.
Returning a limited subset of parameters
By default, search results contain information about all the parameters that are either required or optional. If only most basic (required) parameters are needed, one can add the following string to the url
&return=required
Specialized Searches
It is also possible to run any specialized searches through the web interface. At present, there is only one: searching for certain events (call them type A) inside other events (call them type B). This means that events of type B need to have some kind of bounding chaincode. To run this search, a start time, end time, and two event types are required. The follow search looks for active regions inside coronal holes between the 23rd and 29th of October 2003.
Note: the geometry operation is done in Carrington coordinates, so if you were to plot the events in helioprojective coordinates, it may look as if certain events do not overlap.
https://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=ar&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00
Summary Page of a HER Event/Feature
Each event/feature in HER has a unique id (look for event_id parameter in the search results output). Let says, an entry has an event_id of 1011428. To view the summary page of the event, point your browser to
http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=view-voevent&id=1011428
Clickable link is [http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=view-voevent&id=1011428 here].
VOEvent XML of a HER Event/Feature
To get the VOEvent XML of an entry, use the [http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=export-voevent&cosec=1&id=1011428 link]
http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=export-voevent&cosec=1&id=1011428
where the value of id is the parameter given as 'event_id' (look for it in the search results output).
Recently Reported Events Page
To see the most recent submissions of events and featuresd detected by humans (i.e. not by software), go to
http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=home
Clickable link is [http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=home here]. Note that only those with frm_humanflag = true will be shown.
Authentication
All services that permanently edit the knowledgebase require authentication via your Heliophysics Knowledgebase (HPKB) account. Login is done via an HTTPS POST request, and session is maintained via an HTTP cookie. The login can be accomplished via:
curl -k -c /tmp/cookiejar.txt -d username=YourUsername -d password=YourPassword -d cmd=login -d cosec=1 https://www.lmsal.com/her/dev/search-hpkb/heks
It will then return a "login failure" or "login success" message. Once you are logged in, you can use any of the services below as long as you send the cookie received after login. All of them operate on HTTPS POST. For example, a comment can be added via:
curl -k -b /tmp/cookiejar.txt -d cmd=comment -d id=TARGET_EVENT_ID -d comment=YOUR_COMMENT -d cosec=2 https://www.lmsal.com/her/dev/search-hpkb/heks
Rating Events
To rate an event, use
https://www.lmsal.com/her/dev/search-hpkb/heks?cmd=rate&id=event_id&rating=XXX&cosec=2
where event_id is the id of the event you wish to rate, and XXX is the rating, a number between 0 and 1. The output is returned in JSON format, with the "status" parameter indicating "success", "permission denied", or "error" (such as when the user tries to rate the same event twice.) The user must be logged in to use this command.
Commenting on Events
To comment on an event, use
https://www.lmsal.com/her/dev/search-hpkb/heks?cmd=comment&id=event_id&comment=XXX&cosec=2
where event_id is the id of the event you wish to rate, and XXX is the comment. The output is returned in JSON format, with the "status" parameter indicating "success", "permission denied", or "error". The user must be logged in to use this command. Also, one can retrieve the existing comments by using the same url, without the comment parameter. The user does not need to be logged in to retrieve comments.
Adding a reference to an event
To add a reference to an event, use
https://www.lmsal.com/her/dev/search-hpkb/heks?cmd=add-reference&id=event_id&name=XXX&type=YYY&url=ZZZ&cosec=2
where event_id is the id of the event you wish to rate, XXX is the reference name, YYY is the reference type, and ZZZ is the reference URL. The output is returned in JSON format, with the "status" parameter indicating "success", "permission denied", or "error". The user must be logged in to use this command.
Uploading an event to the registry Uploading via the web interface is also done via a multipart HTTPS POST to https://www.lmsal.com/her/dev/search-hpkb/voeupload with the uploadedFile parameter set to the contents of the file.