Technology

This application is coded in XQuery and runs on eXist-db version 1.4.2 on a BitFolks Server. Graphviz is used to generate the interactive graphs. Multiple APIs are used to transform data, and are listed when used. The core is an XML database containing descriptions of types and properties.

Base Properties

Base properties are the literal-valued properties such as address, latitude and longitude. Base properties have local names in a circe namespace. Base properties may themselves have properties. These include aliases, sameas URIs and relationships to other resources such as definitions and lists of values. The browser interface form allows the user to enter values of base properties.

Simple derived property

Derived properties have values computed from base property values. The property value is a template with parameters.

        <property name="map by lat/long" type="text/html"
                  value="http://maps.google.com/maps?z=16&amp;q={latitude},{longitude}">
                <param name="latitude"/>
                <param name="longitude"/>
       </property>
 

The parameters of this template are enclosed in { and } , and also declared explicitly (until I improve the parsing). If Circe has values for the literal properties latitude and longitude, these values will be substituted in the template and the derived property added to the set of defined properties.

Computed Parameters

Parameters often need to be re-formated before inclusion in a template:

           <property name="photos" value="confluence.php?lat={lat}&amp;lon={long}">
                <param name="lat">
                    <compute>round({latitude})</compute>
                    <param name="latitude"/>
                </param>
                <param name="long" >
                    <compute>round({longitude})</compute>
                    <param name="longitude"/>
                </param>
            </property>
   

The URI of the Degree Confluence page for a latitude/longitude needs the full latitude and longitude to be rounded to integers. If Circe has these base properties, then the derived parameter values can be computed (using util:eval in eXist) and substituted into the URI template and the derived property added to the set of defined properties.

Functions

Functions are properties whose URI returns xml (or data which can be converted to XML by Circe), from which new properties can be extracted and added to the set of defined properties.

         <property name="convert latlong to OS" type="application/xml"
                value="?geo/latlong2os.xq?lat={latitude}&amp;long={longitude}">
                <param name="latitude"/>
                <param name="longitude"/>
                <pre>wfn:between({latitude},49.7,60.8) and wfn:between({longitude},-8,2.1)</pre>
                <return name="easting"/>
                <return name="northing"/>
         </property>
    
   

The local web service latlong2os.xq accepts a latitude and longitude and returns a plain XML file containing 6 figure easting and northing. Provided that the latitude and longitude properties are defined and they are within a bounding box covering the UK (just a rough estimate here), the function is invoked, returning an XML document. By default the return variables are matched to elements of the same name anywhere in this document. The matched properties are added to the set of defined properties.

SPARQL Queries

SPARQL queries are also supported as functional properties. The following query uses the version of the data from OurAirports converted into RDF by Ian Davis and stored in the Talis Data Incubator. This query looks up an airport's IATA codes and returns its lat,long and name.

      <property name="convert via Talis Data Incubator" type="application/sparql-result+xml"
                value="http://api.talis.com/stores/airports/services/sparql?query={query}">

        <param name="query> 
                <compute eval="no">
                (as CDATA)                 
PREFIX p: <http://dbpedia.org/property/>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
select * where {
 ?airporturi p:iata '{IATA}';
     geo:lat ?latitude;
     geo:long ?longitude;
     rdfs:label ?label;
}
                <param name="IATA"/>
                </compute>
           <return name="latitude"/>
           <return name="longitude"/>
           <return name="label"/>
    </property>
 

The property value attribute defines the SPARQL endpoint. Variables in the SPARQL query parameter are replaced by their current values before sending to the SPARQL service. The returned SPARQL XML result format is converted to a vanilla XML format, and the return properties extracted.

Functions with computed returned properties

Often the xml returned does not support the default matching because names are different or values need to be manipulated. Compute expressions are supported on return parameters:

  <property name="convert lat/long to postcode" type="application/xml" 
            value="http://www.uk-postcodes.com/latlng/{latitude},{longitude}.xml">
                <param name="latitude" />
                <param name="longitude" />
                <pre>wfn:between({latitude},49.7,60.8) and wfn:between({longitude},-8,2.1)</pre>
                <return name="postcode"/>
                <return name="ward">
                    <compute>tokenize(.//ward/uri,'/')[last()]</compute>
                </return>
                <return name="local-authority-district">
                    <compute>tokenize(.//district/uri,'/')[last()]</compute>
                </return>
                <return name="ward-name">
                    <compute>.//ward/title</compute>
                </return>
                <return name="local-authority-district-name">
                    <compute>.//district/title</compute>
                </return>
  </property>
 

This function uses Stuart Harrison's API to convert lat/longs to post code and ONS areas. The returned XML needs some manipulation to get the data in the required form for Circe. For example we want the SNAC (or census) code for a ward but the API returns a full URI so we have to extract the last part of the URI. Relative XPath expressions are used here and are evaluated (with exist util:eval-in-context() ) in the context of the returned XML document.

Function network

The full network of functions currently defined is seen in the graph. To propagate values through this network, Circe finds candidate functions for which all the parameters are defined, the pre-condition is true and one or more of the returned properties is unknown. One of the candidate functions is chosen and executed. The process continues until there are no further candidate functions. This push algorithm is inefficient if only specific values are needed, when a pull algorithm is preferable, but harder to code.