webofdatablog

Scottish government spending - CSVs please?

Tue, 24 Aug 2010 16:09:28 +0100

The Scottish Government is now publishing monthly reports of all items of expenditure over £25,000.

That’s great and a positive step towards transparency and open data. But can we have them as CSV files? or even Excel? Currently they are only available as PDF which makes it a pain to do anything with the data.

Chris Taggart has converted the first couple of reports to CSV and helpfully made them available here, but since this data was probably in a database or spreadsheet before being made into a PDF, it’s a bit daft to create all that extra work for anyone who wants to analyse the data.

They could leap in a single bound from one star to three on the Berners-Lee 5 stars of open linked data.

RDF datasets and graphs

Tue, 24 Aug 2010 15:50:39 +0100

I think we need to establish a common practice to link RDF datasets and named graphs.

I’ve been working recently on discovery of linked data: how people can find out easily what is available and how they can use it, either directly or by building it into new applications.

The dataset is clearly an important concept in this, albeit a rather vague one – essentially it’s a bunch of data that belong together somehow.

As most of you will already know, an ever increasing list of UK government datasets is catalogued at http://data.gov.uk/data, each with some supporting information on what it’s about, where it came from and where to go to access all the details.

Some of that data has been made available as Linked Data, with dereferenceable URIs and SPARQL endpoints, for example the information on schools available here. The Linked Data approach can be very powerful, but for a ‘data consumer’ it can also be difficult to know where to start. One of the things I’m currently working on is to publish some simple additional information that will hopefully make it easier to exploit the Linked Data part of data.gov.uk.

Which brings me back to describing Linked Data datasets. The de facto standard for this (and only show in town) is voiD which defines a vocabulary and recommends some good practices for describing a dataset.

Via the void:dataDump property, you can point to a location where you can get a copy of all the data in the dataset. And using dcterms:isPartOf, you can link the description of a resource back to the dataset that it’s part of.

OK, so far so good. However, one important thing that seems to be missing in this picture is how to restrict a SPARQL query to a particular dataset or shortlist of datasets.

The typical approach with statistical data has been to use SCOVO, soon to be usurped by the RDF Data Cube vocabulary from Dave Reynolds et al (still work in progress but hopefully soon to be released). Those approaches link individual observations back to the dataset they belong to, which makes it easy to limit a query to a particular dataset. But not all data fits that kind of pattern.

However this is exactly the reason that the named graph approach was created – as a convenient way of grouping a bunch of triples together and letting us talk about them – and it’s already supported by most RDF databases/quad stores.

VoiD lets you link a dataset to a SPARQL endpoint, but the voiD guide says “Note: It is assumed that the default graph of the SPARQL endpoint is the dataset itself”. This seems unnecessarily restrictive as it implies only one dataset per endpoint and a lot of the value of SPARQL and Linked Data in general is the ability to connect stuff across multiple datasets.

We can link named graphs to voiD datasets (by using dcterms:isPartOf for example) and if the data available through SPARQL endpoints is grouped into those named graphs, then it provides an easy mechanism for adding metadata and finding aids for the steadily increasing list of linked data datasets.

Linked Data in Edinburgh and Manchester

Fri, 23 Jul 2010 15:11:07 +0100

I’ve been at a couple of great Web of Data events in the last ten days or so.

On 13 July, I organised a Linked Data meetup in Edinburgh that I’m pleased to say went very well. Around 25 people showed up to hear interesting talks from Zach Beauvais of Talis (slides) and Paola di Maio of Strathclyde University (slides). There was a good mix of people already experienced with linked data and others who wanted to learn more about it – many of them with specific potential applications in mind.

There’s a fuller write-up at the SLDIG wiki.

Then a couple of days ago I went down to Manchester for the Vision and Media Transmission 6 event “Towards a web of data?” organised by Paul Collins of the White Room.

I presented on “Publishing Linked Data: Getting Started”. The other speakers were Paul Miller of Cloud of Data (and semantic podcaster extraordinaire) and Liz Turner of Iconomical, the creator of the well-known WhereDoesMyMoneyGo. Paul’s slides are here.

I very much enjoyed the chance to be involved and to meet with some of the thriving digital media community in Manchester.

[Credits: Manchester picture by Graham Smith, via Flickr).]

Public sector open data: big problems still to solve

Sun, 04 Jul 2010 10:09:52 +0100

We’re in the very early days of the UK government’s new approach to open data. While it seems to be going in the right direction and progress has in many respects been remarkably quick (eg this recent announcement on public sector transparency), clearly we still have a lot to learn and many problems to overcome.

Chris Taggart and Vicky Sargent’s recent
article on lessons learned from the Open Election Data project was extremely informative on how far we still have to go. I strongly recommend that you read it in full. It’s a salutary reminder that although activity around linked data is blooming, it’s very easy to forget that most people have no idea what we are talking about! That’s our failing, not theirs.

There is a need for education and awareness raising about the basics of web publishing in general. This ought to be a core activity of any organisation in the 21st century and will be the foundation for what we want to achieve with linked data. If more people in local government understand what is possible, that can feed through into setting priorities and selection of software tools and suppliers.

We in the tech community need to provide better software tools to assist the publishing process. They need to work alongside existing systems, not replace them, otherwise uptake will be impractically slow. A lot of the source information is already online but we can make it a lot more useful. Chris Taggart’s work with OpenlyLocal shows what can be achieved with a small amount of well targeted effort.

There is a lack of good learning resources aimed at the intelligent layman. We need introductory material aimed at particular user groups, such as local government, with examples that can be copied and adapted.

There are now a few success stories that illustrate the potential of linked data publishing and inspire more people to get involved. Let’s find more of those and get the word out to a broader community.

Register now for Scottish Linked Data meetup

Wed, 23 Jun 2010 08:32:44 +0100

You can now register for the Scottish Linked Data Interest Group meetup on 13 July.

The event is free but places are limited and there has already been a great reaction, so if you’d like to come please do sign up.

First meeting of the Scottish Linked Data Interest Group

Tue, 15 Jun 2010 17:45:45 +0100

The first meeting will be at 7pm on Tuesday 13 July at the Outhouse in Edinburgh.

I’ll set up an Eventbrite page for registration in the next few days and put together some more details about the event, but please put the date in your diary if you are interested.

Also, we’re on the lookout for 2 or 3 speakers for the event, so if you have something you would like to present on the topic of Linked Data, please let me know. As mentioned in my previous post, we welcome contributions from people with data to publish, people who are using linked data in interesting ways and people working on tools and methods that support the process.

Scottish Linked Data Interest Group

Mon, 24 May 2010 10:39:16 +0100

At the recent Open Knowledge Scotland meeting, Paola di Maio suggested setting up a Scottish interest group on linked data. Jeff Pan and I agreed with her and we decided to give it a try. I’ve volunteered to organise the first meeting.

We know that there’s a lot of interesting work going on in Scotland in this field and we want to provide an informal forum for people in the area to get together and exchange ideas – and hopefully encourage others to get involved.

Our initial discussions have led us to the following concept:

an emphasis on connecting different groups: people who have data to share, people who want to use data and people with tools for working with linked data. We want to help them to work together to achieve great things
a roughly 50/50 split of presentations and time for discussions and networking, in an informal friendly atmosphere
rotate the venue around the main cities of Scotland so we share out the travelling and access a big enough group of people to make the meetups work
meetings roughly four times a year

For the first meeting, I’m thinking of an evening meeting (probably 7pm-9pm) in Edinburgh, provisionally mid July, but I’d like to hear your ideas and preferences.

To get an idea of likely numbers, please leave a comment on this blog, or email me direct to let me know if you are interested in coming, whether you’d like to help organise this or subsequent meetups, whether you have something you want to present, what topics you’d like to see discussed and any thoughts you have on the way we should run the meetings. Thanks!

Got data, want linked data?

Wed, 31 Mar 2010 17:14:50 +0100

At Swirrl we’re working on a new platform for publishing Linked Data. (It’s called PublishMyData). It’s still at an early stage and we’d like to test our ideas against the constraints and objectives of the people who we hope might make use of it.

A good example of what we are thinking of is to create Linked Data for the 99% of datasets on data.gov.uk that are currently Excel, CSV, PDF etc rather than RDF (see Tom Morris’s survey of data.gov.uk formats) – though we’re not just interested in government datasets.

So if you have data you’d like to publish on the web as RDF/OWL (as well as HTML human-readable versions), but you haven’t done it because of lack of time, or lack of resources or technical knowledge – or indeed if you’ve recently been through a linked data publishing process – we’d love to hear your ideas and to find out if our initial plans are a good match for what you actually need. You can get in touch with me at bill@swirrl.com or @billroberts.

Scottish Data Now!

Fri, 19 Mar 2010 13:03:54 +0000

Tim Berners-Lee recently gave a short talk to TED, reporting back on a year of great progress since his 2009 cry of “Raw Data Now”. One of the big contributors to this has been the UK government via the data.gov.uk initiative, which I’m sure all readers of this blog are well aware of.

If you are from outside the UK (and for many Brits too), you might not be aware that the UK has a rather complicated system of government, where some responsibilities are handled centrally for the whole country and some are devolved to the Scottish government (and a different set of responsibilities to assemblies in Wales and Northern Ireland). Anyway, it seems that data.gov.uk is one of those initiatives that more or less stops at the Scottish border. This is mainly because most of the departments publishing their data have responsibility for England and Wales, with an equivalent but separate Scottish department in charge once you are north of Hadrian’s Wall. Or something like that – it’s complicated!

The Scottish government publishes quite a few high quality datasets through its own websites and some of those are linked from data.gov.uk (for example the Scottish Index of Multiple Deprivation and Abstract of Scottish Agricultural Statistics, or the sheep census as I like to think of it: 7.5 million in 2007 for those who are interested) but the concerted effort for publishing more data, in particular Linked Data, seems to be lacking. And because I live and work in Scotland, I want the same benefits here that are starting to appear in other parts of the UK.

I wrote today to John Swinney, Cabinet Secretary for Finance and Sustainable Growth in the Scottish government (he seemed like the most relevant person to ask), to encourage him to set up a similar initiative in Scotland – or to get the Scottish departments more actively involved in the existing UK scheme. Either way, it seems clear to me that this is something that can stimulate the high-tech economy, improve government efficiency through inter-departmental data re-use, and promote transparency and accountability of government. In other words it’s a Good Thing.

Scotland has a small but vibrant and growing web and digital media scene and we could do lots of great stuff with this. So we should start demanding our Scottish public data now!

We should also just start doing it and I’m currently working on a couple of examples to try to get the ball rolling and show what is possible.

For readers based in Scotland, if you want to express your opinion to the government, you can find contact information for the Scottish Cabinet here or contact your own elected representative.

Feed problems

Mon, 15 Feb 2010 11:03:34 +0000

Sorry for introducing lots of articles into your feed reader. After moving this blog over to its new platform, we had a few problems with the feed caching system, which meant that articles from Ric’s blog (ricroberts.com) found their way into my feed.

The problem is now fixed, so we should be back to normal service now (I hope!)