TableOfContents(3)

Heliophysics Events Registry web API

Introduction

http://www.lmsal.com/helio-informatics/hpkb/images/HER_arch.jpg

Both Solarsoft and the [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html iSolSearch] web application lets users query the Heliophysics Events Registry (HER) in an interactive manner (much like Google Maps). iSolSearch interacts with HER by means of our web application programming interface (API). The API enables developers to create new third party clients for HER as well as scientists to write software code which interact with HER in an automated manner.

Querying the Heliophysics Events Registry

Usage of an API function amounts to using HTTP to fetch a web document. The base url for all API calls is (including the question mark)

http://www.lmsal.com/her/dev/search-hpkb/hek?

Following the url base, one can append a list of keywords to specify the type of command and options for that specific command.

Simple Search for Features/Events in HER

To do a simple search for Features and Events in HER, one must append to the base url:

cmd=search&type=column

In addition to the command (cmd), one must specify which event type to search for. For all event types, use **

cmd=search&type=column&event_type=**

For some but not all event types, use a comma delimited list of two-letter abbreviations (see http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html for full list of event types). E.g. the following specifies that one wishes to search for active regions, flares and emerging flux regions:

cmd=search&type=column&event_type=ar,fl,ef

One must also provide a time window using event_starttime and event_endtime

cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00

Only entries with duration fully contained within this time will be returned.

The spatial region must also be specified. At present, we support searches by helioprojective (from Earth's perspective, in arcseconds from disk center), Stonyhurst (longitude measured from central meridian and latitude in degrees) and Carrington (longitude and latitude in degrees). Let (x1,y1) and (x2,y2) be the lower-left and upper-right coordinates of a bounding box respectively. To get everything within the disk, it suffices to specify

cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200

To specify a bounding box in Stonyhurst coordinates, give the bounding longitude and latitude in degrees. For the full sphere, do

cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&event_coordsys=stonyhurst&x1=-180&x2=180&y1=-90&y2=90

To get query results in XML form, set cosec=1. To get JSON, set cosec=2 (this setting is needed for iSolSearch to render the results properly).

The query for this example with JSON output would then be

http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200

To show the search results in iSolSearch, simply point your browser to the iSolSearch link with and passing the full query as a hek_query parameter. For instance, click [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html?hek_query=http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2007-04-29T00:00:00&event_endtime=2007-05-07T00:00:00&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200 here]. Please note that iSolSearch requires cosec=2 type (JSON) format. Useful tools for inspecting JSON documents on the web are the Firefox Add-Ons JSONovich [https://addons.mozilla.org/en-US/firefox/addon/10122] and JSONView [https://addons.mozilla.org/en-US/firefox/addon/10869] (this one is recommended).

Customizing Searches with Filters

Searches can be customized with filters based on keywords. For a list of keywords/parameters for the various event and feature types, see http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html. A filter is a triplet consisting of the Parameter name (param), the comparison operation (op) and a value (value).

Filters on attributes of type string

Suppose you wish to search for all events submitted by a particular Feature Recognition Method (FRM; could be person or a software code). The API allows you to specify the FRM_Name as a string. At present the API supports seven comparison operations for strings, namely =, != >, <, >=, <= and like.

Greater than or less than operations on string values refers to alphabetical ordering.

To specify the exact FRM_Name, simply append the following string to the query url

&param0=FRM_Name&op0==&value0=Karel%20Schrijver

The example above tells the API to limit results to those with FRM_Name matching Karel Schrijver. The strange looking string %20 is simply the url-escaped code for a whitespace.

Instead of specify the exact string, one can also use a wildcard. For instance, if you wish to search for entries in the registry submitted by FRM's with first name equal to Karel, use the like operator with a '%' sign as a wildcard after Karel

&param0=FRM_Name&op0=like&value0=Karel%25

The string %25 is simply the url-escaped code for a percent sign.

To add a second filter, simply define another triplet. This time, the number after param, op and value is 1.

&param0=FRM_Name&op0==&value0=Karel%20Schrijver&param1=OBS_Observatory&op1==&value1=TRACE

These two filters limit search results to those events/features reported by Karel Schrijver using data from the TRACE observatory.

The full query url for a search with these two filters for the time period October 2003 is then

http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2003-10-01T00:00:00&event_endtime=2003-11-01T00:00:00&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200&param0=FRM_Name&op0==&value0=Karel%20Schrijver&param1=OBS_Observatory&op1==&value1=TRACE

To see the results in iSolSearch, click on this [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html?hek_query=http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=column&event_type=ar,fl,ef&event_starttime=2003-10-01T00:00:00&event_endtime=2003-11-01T00:00:00&event_coordsys=helioprojective&x1=-1200&x2=1200&y1=-1200&y2=1200&param0=FRM_Name&op0==&value0=Karel%20Schrijver&param1=OBS_Observatory&op1==&value1=TRACE link]

To find Flares with GOES class X and above, use this filter

&param0=FL_GOESCls&op0=%3E&value0=X

The string %3E is the url-escaped code for the > character.

Filters on attributes of type boolean

The API supports the equality (=) and inequality (!=) logical operations on attributes of type boolean.

E.g., for attributes such as FRM_HUMANFLAG, one should set op0== and value0=true. Note that value0=t will not work if you want to get HER entries with true values for a particular boolean attribute.

Filters on attributes of type integer/float

The API supports six comparison operations for such attributes, namely =, !=, >, <, >= and <=. For example, to get Active Regions with NOAA number between 10930 and 10934 (inclusive), use the following filter:

&param0=AR_NOAANum&op0=%3E=&value0=10930&param1=AR_NOAANum&op1=%3C=&value1=10934

One can learn to add filters to queries in [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html iSolSearch] by click on the Filters tab.

Maximum number of returned results

By default the maximum number of results given a query is 100. To change this limit, set

result_limit=Num

where Num is the maximum number of results to return. There is a currently a hard limit of 200 results. In other words, specifying a value of 500 will return 200 results maximum. The results are returned sorted by reverse chronological order (Based on event_starttime). To avoid unreasonably time-consuming requests from hoarding API resources, a search will timeout after 60s. In that case, please breakup you query into small chunks and perform searches incrementally.

Output format of queries

To get query results in XML form, include the string &cosec=1 in the query url. To get JSON, use &cosec=2.

The xml output is looks like this

http://www.lmsal.com/helio-informatics/hpkb/images/cosec1output.jpg

The JSON output looks like this

http://www.lmsal.com/helio-informatics/hpkb/images/cosec2output.jpg

If JSON output is requested, an optional parameter can be specified in the url to wrap the JSON with a call to a Javascript callback function. For instance, suppose the web client you are developing requests JSON from HER. If you want to handle the JSON with some function, say it's named processJSON, then simply provide the optional parameter 'callback=processJSON' in the query url. With this option, the output will have the string processJSON( ); wrapped around the JSON.

Viewing Test Events

For internal testing purposes, fake test events are regularly generated. To view these, one can add the following string to the url, where value is one of the following:

&showtests=value

Returning a limited subset of parameters

By default, search results contain information about all the parameters that are either required or optional. If only most basic (required) parameters are needed, one can add the following string to the url

&return=required

Containment search

This search function allows the user to find events which a located within another event (both in time and space). Suppose you want to find Active Regions inside Coronal Holes. In order to do this, set

cmd=search&type=contains

in the query url. This tells the API that you want to do a containment search instead of a 'column' type search (examples in previous sections). In the url, set

event_type1=ar&event_type2=ch

and as usual, specify the event_starttime and event_endtime parameters. An example of an AR-in-CH search is

http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=ar&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00

Clickable link for the JSON is [https://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=ar&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00 here]. To render these results in iSolSearch, click [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html?hek_query=https://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=ar&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00 here].

An example of an FL-in-CH search (for the same time period) is

http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=fl&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00

Clickable link for the JSON is [https://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=fl&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00 here]. To render these results in iSolSearch, click [http://www.lmsal.com/helio-informatics/hpkb/hek_isolsearch.html?hek_query=https://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=search&type=contains&event_type1=fl&event_type2=ch&event_starttime=2003-10-23T00:00:00&event_endtime=2003-10-29T00:00:00 here].

The exact mechanics of the containment search are as follows:

1. Find events of event_type1 and event_type2 that are completely contained within the time period between event_starttime to event_endtime. A default tolerance of 8 hours is used so that if event1 has an event_starttime within 8 hours of the event_starttime of event2, they are considered to be cotemporal.

2. If event_type2 has a bounding polygon (e.g. Coronal Holes have CH boundaries), then find events of event_type1 whose central coordinate position (in Carrington coordinates) lies within the bounded area of instances of event_type2. Although temporal overlap is not strictly enforced, the instances of event_type1 and event_type2 must occur sufficiently close in time.

3. If event_type2 does not have a bounding chaincode, use the bounding box coordinates (Carrington) to define a polygon that acts as a proxy for its bounding chaincode.

The JSON (for cosec=2) output returned provides instances of event_type1 and event_type2 that satisfy the search criteria in an array called 'results' (same format as output for simple queries). Furthermore, the relations between the returned events are given as an array called 'associations'. Each element in the 'associations' array contains information about the event types as well as their VOEvent IVORNs (unique labels for VOEvents). The 'results' and 'associations' arrays can be respectively thought of as nodes and edges of a graph.

Summary Page of a HER Event/Feature

Each event/feature in HER has a unique label, which is also known as its VOEvent IVORN (look for kb_archivid parameter in the search results output). Let says, an entry has an kb_archivid of ivo://helio-informatics.org/FA1550_YingnaSu_20090415_154655. To view the summary page of the event, set

cmd=view-voevent&ivorn=ivo://example-ivorn

Following is an actual example:

http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=view-voevent&ivorn=ivo://helio-informatics.org/FA1550_YingnaSu_20090415_154655

Clickable link is [http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=view-voevent&ivorn=ivo://helio-informatics.org/FA1550_YingnaSu_20090415_154655 here].

VOEvent XML of a HER Event/Feature

To get the VOEvent XML of an entry, use the [http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=export-voevent&cosec=1&ivorn=ivo://helio-informatics.org/FA1550_YingnaSu_20090415_154655 link]

http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=export-voevent&cosec=1&ivorn=ivo://helio-informatics.org/FA1550_YingnaSu_20090415_154655

where the value of ivorn is the parameter given as 'kb_archivid (look for it in the search results output).

Recently Reported Events Page

To see the most recent submissions of events and featuresd detected by humans (i.e. not by software), go to

http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=home

Clickable link is [http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=home here]. Note that only those with frm_humanflag = true will be shown.

List of existing values of HER attributes

Suppose one wishes to see what values of the attribute FRM_Name are already present in the database. Use the following web api function

http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=view-attributes

Clickable link is [http://www.lmsal.com/her/dev/search-hpkb/hek?cosec=2&cmd=view-attributes here] This returns a list of attributes, their values and the frequency of occurrence for each value. To constrain the list to attributes for a specific event type (e.g. ARs), append to the url &event_type=ar.

Authentication and Post-authentication Methods

All services that permanently edit the knowledgebase require authentication via your Heliophysics Knowledgebase (HPKB) account. To register for an account, go to http://www.lmsal.com/hpkb/Nav and click on Registration.

Login

Login is done via an HTTPS POST request, and session is maintained via an HTTP cookie. Make sure the command line tool curl (http://curl.haxx.se/) is installed. The login can be accomplished via:

curl -k -c /Users/home/cookiemonster/cookiejar.txt -d username=YourUsername -d password=YourPassword -d cmd=login -d cosec=1 https://www.lmsal.com/her/dev/search-hpkb/heks

In order for a cookie to be retrieved, be sure to use cosec=1.

The previous call will return either a "login failure" or a "login success" message. Once you are logged in, you can use any of the services below as long as you send the cookie received after login (in this example, /Users/home/cookiemonster/cookiejar.txt).

All of them operate on HTTPS POST. For example, a comment can be added via:

curl -k -b /Users/home/cookiemonster/cookiejar.txt -d cmd=comment -d id=TARGET_EVENT_ID -d comment=YOUR_COMMENT -d cosec=2 https://www.lmsal.com/her/dev/search-hpkb/heks

Rating Events

To rate an event, use

https://www.lmsal.com/her/dev/search-hpkb/heks?cmd=rate&id=event_id&rating=XXX&cosec=2

where event_id is the id of the event you wish to rate, and XXX is the rating, a number between 0 and 1. The output is returned in JSON format, with the "status" parameter indicating "success", "permission denied", or "error" (such as when the user tries to rate the same event twice.) The user must be logged in to use this command.

Commenting on Events

To comment on an event, use

https://www.lmsal.com/her/dev/search-hpkb/heks?cmd=comment&id=event_id&comment=XXX&cosec=2

where event_id is the id of the event you wish to rate, and XXX is the comment. The output is returned in JSON format, with the "status" parameter indicating "success", "permission denied", or "error". The user must be logged in to use this command. Also, one can retrieve the existing comments by using the same url, without the comment parameter. The user does not need to be logged in to retrieve comments.

Adding a reference to an event

To add a reference to an event, use

https://www.lmsal.com/her/dev/search-hpkb/heks?cmd=add-reference&id=event_id&name=XXX&type=YYY&url=ZZZ&cosec=2

where event_id is the id of the event you wish to rate, XXX is the reference name, YYY is the reference type, and ZZZ is the reference URL. The output is returned in JSON format, with the "status" parameter indicating "success", "permission denied", or "error". The user must be logged in to use this command.

Uploading an event to the registry

Uploading via the web interface is also done via a multipart HTTPS POST to https://www.lmsal.com/her/dev/search-hpkb/voeupload with the uploadedFile parameter set to the contents of the file.

Example

The following is an example of a Flare VOEvent XML http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=export-voevent&cosec=1&ivorn=ivo://helio-informatics.org/FL_SSWLatestEvents_20090704_053406_gev_20090704_0246_V1p0

Download it and modify some of the fields (e.g. FRM_Name, FRM_Contact, FRM_Institute etc). Suppose the file is saved as Test.xml. Now try to upload it with the following curl command

curl -k -b /Users/home/cookiemonster/cookiejar.txt -F uploadedFile=@Test.xml https://www.lmsal.com/her/dev/search-hpkb/voeupload

Upon successful import, you will get the message "Successfully imported XML". If you now go to http://www.lmsal.com/her/dev/search-hpkb/hek?cmd=home, you should see your new submission.

For details about the VOEvent XML specification for the Heliophysics Events Knowledgebase, go to http://www.lmsal.com/helio-informatics/hpkb/voevent.html. The full list of VOEvent types and attributes can be found at http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html.

Posting a revised event to the registry

Revising an event via the web interface is also done via a multipart HTTPS POST to https://www.lmsal.com/her/dev/search-hpkb/voerevise with the uploadedFile parameter set to the contents of the file.

Make sure that Event_IVORN of the revising XML matches the Event_IVORN of the entry to be revised.

Reporting relations between events/features

This section is under development Relations between existing entries in HER can be reported. All relations are of the form

A v B

where A and B are events/features (e.g. ARs, FLs etc) and v is a verb indicating the relation between A and B. The following verbs are allowed: