A little rundown on how I’ve used Kimono and a little Json transformation to automatically retrieve and display various metrics from across the web on a Geckoboard dashboard.

Geckoboard

Geckoboard provide a nice way to display data on a dashboard and have a significant number of widgets that tie into various APIs but they also have a simple ability to display a number, a simple string of text or a chart of some kind when provided with a suitably structured JSON or XML payload. See more about the custom widgets here.

I’ve been a Geckoboard user for a number of years now, mostly just as a place to put up various metrics of things I’m somewhat vaguely keeping track of. Examples include my FitBit steps, share prices and the current temperature.

Kimono

Kimono have only been around for a short while but they’ve certainly exploded onto the scene with a rather impressive offering. Check out their intro videos and their interactive demo and see how easy it is to create an API from a website, you may never need to write a custom, hacky web-scraper ever again :)

I’ve created several APIs with Kimono to extract data on recurring basis, in this case it’s the recent atmostpheric Carbon Dioxide as measured at Cape Grim by CSIRO. Every week Kimono will scrape the site and update it’s cached local copy of the value. The Cape Grimm data is monthly data and is updated monthly (in theory, but this seems to vary), so checking weekly with Kimono sounds best and it costs nothing either way.

There are various transports available to retrieve the data, including JSON, CSV and RSS/XML. I’ve done a lot of XML and CSV over the years but not so much JSON so figured it was time to have a play with that. Rather than hacky string chomping I’ve found a simple library to do the work for me in much the same way I would use XPATH if I was consuming XML.

JsonPath

JsonPath is a little library from Ben Hale that has implemented the concepts discussed in an article JsonPath - XPATH for JSON by Stefan Goessner. Invoking the library is incredibly easy and has followed along the lines of using the Java regex Pattern class.

Both Kimono and Geckoboard have simple data structures for their services, but they’re not exactly compatible, so I’ve used JsonPath to ease my translation between them.

The output from my Kimono Cape Grim CO2 service is something like this:

{
  "name": "metric_Cape_Grim_CO2_Monthly",
  "count": 1,
  "frequency": "weekly",
  "version": 2,
  "newdata": false,
  "lastrunstatus": "success",
  "lastsuccess": "Mon May 19 2014 18:21:44 GMT+0000 (UTC)",
  "nextrun": "Mon May 26 2014 18:21:44 GMT+0000 (UTC)",
  "results": {
    "collection1": [
      {
        "property1": "CO2: 393.7 (ppm) - February 2014"
      }
    ]
  }
}

The required input for a Geckoboard “custom text widget” is like this:

{
  "item": [
    {
      "text": "CO2: 393.7 (ppm) - February 2014",
      "type": 0
    }
  ]
}

So my desired and rather simple mapping is something like this:

With JsonPath, the following simple expression is used to extract the required value from the Kimono JSON:

  $.results.collection1[0].property1

Breaking the above expression down into steps:

• the $ is the equivalent of the “root node” or // in XSL, so it selects the top level JSON
• then we select the results property
• then the first element in the collection1 collection
• and finally the property1 property which contains our desired value

Note how the . breaks up the elements in the expression, essentially the same as the / in XSL. This expression above will give us the relevant value and we can then pass that into our next process of building the Geckoboard JSON.

This feature is implemented like this:

	public String extractValueFromKimonoUsingJsonPath(String input) {
	JsonPath namePath = JsonPath.compile("$.results.collection1[0].property1");
	String geckoValue = namePath.read(input, String.class);
	System.out.println("Gecko:\n" + geckoValue);

	return geckoValue;
}

Building the Geckoboard JSON

Taking the value we’ve extract above, we then produce a Geckoboard compliant JSON payload using the Jackson ObjectMapper.

To produce JSON, all we need to do is populate a couple objects with the relevant data and then run the object tree through ObjectMapper. This creates a small overhead in forcing one to create a few basic object classes, but certainly seems the easiest way to produce compliant JSON. Alternatively, one could perhaps use a Groovy String as a template but that’s not exactly useful for anything other than likely this particular scenario with a new template needed in most situations.

Here’s the class that holds the actual “text” value:

	public class GeckoboardItem {
	private String text;
	private int type;
	
	public String getText() {
		return text;
	}

	public void setText(String text) {
		this.text = text;
	}

	public int getType() {
		return type;
	}

	public void setType(int type) {
		this.type = type;
	}
}

Here’s the method to actually map the object tree to a JSON string:

    public String buildGeckoboardJsonWithJackson(String input) {
	List<GeckoboardItem> items = new ArrayList<GeckoboardItem>();
	GeckoboardItem itemValue = new GeckoboardItem();
	itemValue.setText(input);
	itemValue.setType(0); //0 for no corner icon, 1 for error/red alert and 2 for info/grey alert
	items.add(itemValue);
	GeckoboardPayload<GeckoboardItem> payload = new GeckoboardPayload<GeckoboardItem>(items);

	String result = null;
	try {
		ObjectMapper mapper = new ObjectMapper();
		result = mapper.writeValueAsString(payload);
	} catch (JsonMappingException e) {
		System.out.println("Error: There was an issue when creating JSON string. " + e);
	} catch (JsonGenerationException e) {
		System.out.println("Error: There was an issue when creating JSON string. " + e);
	} catch (IOException e) {
		System.out.println("Error: There was an issue when creating JSON string. " + e);
	}

	System.out.println("Jackson:\n" + result);
	return result;
}

Deployment

For this particular little project I’ve simply added the new class into my existing Grails webapp that provides the Thrasher’s Wheat podcast feed, in time I may build out quite a few of these “glue” artefacts and roll a separate project.

For now, I’ve created a basic Grails controller that, in a synchronous manner, simply:

• receives the Geckoboard request
• calls out to the Kimono service
• receives the Kimono service response
• transforms the JSON into the relevant Geckoboard format
• responds back to Geckoboard with the new data

The method in my controller really is as simple as this:

	def co2_cape_grim() {
	String kimonoValue = kimonoToGeckoService.callKimono(Kimono.CO2_CAPE_GRIM, false)
	String value = kimonoToGeckoService.extractValueFromKimonoUsingJsonPath(kimonoValue)
	String result = kimonoToGeckoService.buildGeckoboardJsonWithJackson(value)
	render result
}

Hope this provides something useful to get the imagination going.