Around 150 participants joined the Open Aid Data Conference in Berlin. The event was full with discussions and exchange on how open data can be used to achieve more transparency in the developing aid sector. The first day was split into two workshops – an ‘Aidinfo Data Training’ and ‘Hackday,’ to explore potential data sets and applications to make developing aid more transparent.

In this post I would like to share my experiences from the Hackday as a co-organizer. There was an interesting mix of participants, from different backgrounds, who worked in teams on different issues greatly facilitated by Marek Tuszynski from Tactical Technology Collective. Here is a link summarizing the discussions, some available data sets, and these are the main questions we dealt with:

- What do we need to know about Open Data for aid transparency?
- What data is out there?
- Who benefits from developing aid? Organizations?
- How does an organization implement Open Data?

The last question was answered by one of the groups and put together in a document.
Other teams took a look at the kind of data offered in the German development aid sector. To begin with, the status of information is disappointing and proves the main purpose of the conference: Information is fragmented, almost no data is offered as open, and detailed data for financial spending is not available at all.

It is also impossible to find a list of organizations (NGOs, Government Institutions, etc.) funded by the Federal Ministry for Economic Cooperation and Development (BMZ). Not to mention a list if disbursed funds for each project. That led to one outcome at the Hackday: To initiate a freedom of information act request to the BMZ.

Another team found out that not a single German developing organization offers project information in an open data format, so that it could be easier analyzed. Furthermore, it is very difficult to find out in which countries all these organizations have projects. It can only be found out by clicking through all the websites of hundreds of organizations funded by the BMZ. Imagine how long that takes and how non-transparent that is!

However one interesting database is offered by the OECD, which is called Creditor Reporting System and offers project information for over 50 years. Member countries such as Germany contribute their data about their funded activities under the umbrella of the Official Development Assistance (ODA). We took a look at the data and potential visualizations.

As an outcome of the Hackday, it became clear that we need to invest more time into data analysis to bring more transparency in the sector. Therefore, I will start developing a data catalogue to monitor German aid money better. More to come soon.

Related posts:

1. Open Aid Data conference and Hackday Berlin The past year I have written on many occasions about...
2. Who to feed? The open vs. the commercial race for data Google Maps has been an incredible service in the past...

The past year I have written on many occasions about the potential of open data and why it is much needed particular for the development aid sector. So I am happy to announce a Hackday I am organizing as part of the Open Knowledge Foundation Germany. The Hackday is linked with the Open Aid Data conference held in Berlin, which is organized by Openaid.de, Boell-Foundation among others. Click here for full further information.

Join the Open Aid Data Hackday!

Help us find innovative solution for aid transparency and make development aid more effective.

Germany is one of the largest donors in development aid worldwide. Every year over 6 billion euros are spend alone by the Federal Ministry for Economic Cooperation and Development  to provide humanitarian relief and tackle poverty around the world. The Open Aid data conference will bring together practitioners from various organizations to discuss and exchange about new solutions and how technology, the Internet and particular open data can make aid more transparent – because not all of the money is spent effectively.

Prior to the conference, we will organize on the 28th of September a Hackday at the Böll-Foundation in Berlin to bring developers together to experiment on technical and data solutions to improve development aid. We are looking for programmers, designers, coders and others who want to learn more about the field of development aid and would like to share their wisdom. In the morning, we will introduce you to the theme and then brainstorm on possible approaches to make aid more transparent. During the rest of the day we want to work through a code sprint on a real solution. Be part of the event!

There are a range of activities around open aid data worldwide, such as the recent conference in Amsterdam or a Barcamp in Kathmandu for aid transparency. By the way, an interesting fact: The Kenyan government has offered an open data portal, while the German government is still debating on such a platform.

Open data and new bottom-up solutions for development aid are a rather new field but with some promising developments. Around data there is an initiative called IATI (International Aid Transparency Initiative), which propagates a common standard for data sets for financial and other project related data. So far in development cooperation, only a tiny fraction of financial data is openly available, which is, at the end of the day, the tax payers’ money. Watch this excellent movie from Publish What You Fund on why financial aid transparency is needed.

One driver of IATI is Aidinfo.org, a co-organizer of the conference and member of the IATI secretariat, who has done some pioneering work in the area. Check out the AidInfo Labs to see what is possible through such data sets. We are curious to hear your ideas and projects.

Another driver of open aid data is the World Bank, who will also present their work on the conference. The World Bank has not only opened up its data, but also made an app competition, where many great solutions have been developed, to use the data, for example games about development indicators, amazing visualizations and crowdsourcing approaches. The aim is to make development aid more effective. Initiatives, such as Ushahidi in Africa, demonstrate the potential  of new forms of technology. Come join us at our Hackday to network with great people from the community.

You can apply here or contact Christian.Kreutz {at} okfn.org for further questions. The Hackday is organized by the Open Knowledge Foundation Germany.

Related posts:

1. Results of the Open Aid Data Hackday Around 150 participants joined the Open Aid Data Conference in...
2. Who to feed? The open vs. the commercial race for data Google Maps has been an incredible service in the past...

Remember when you learnt cycling? The first time you stepped into a pedal and tried to balance the bicycle; the voice behind your neck telling you to keep pedaling and you will be fine. Learning cycling is a wonderful example of how difficult it is to “transfer” knowledge and that most of our wisdom is not just published in the Internet.

Have you ever heard of or read a book about how to learn cycling. I am sure there are books for that out there, but would you learn cycling from one? Without practice, patience and using all your senses, you will not succeed nor overcome the fear of falling down. Learning cycling shows how difficult it is to learn other than by just doing and experimenting.  About 80% of knowledge in our brain is tacit and cannot be written down.

I have recently stumbled over a nice article by Noam Cohen, in which he argues that the citation rules by Wikipedia, introduced as a quality mechanism, can also hinder knowledge sharing.

“In the case of dabba kali, a children’s game played in the Kerala state of India, there was a Wikipedia article in the local language, Malayalam, that included photos, a drawing and a detailed description of the rules, but no sources to back up what was written. Other than, of course, the 40 million people who played it as children.”

The same article also linked me to a great video called “People are Knowledge –  Exploring alternative methods of citation in Wikipedia” by Achal Prabhala.

At this point one could ask, what about social media? Knowledge sharing happens through conversations. But, let’s be honest, how many tweets would you need to explain sufficiently how to cycle? There is an inherent limitation in written exchange compared to face-to-face exchange. Ana what about video? Visual exchange can make learning easier; for example, I have read a dozen articles about how to repair my old espresso machine, but only the video explanation made me fully understand how to do it.
On the other side, Mike Davies argues in recent post:

“The lack of very local digital content is acute in Africa and is one reason even Google’s strategy is challenged here. Google Trader (among others) have offered cool new technologies, but without any real content they’re just not being used.” (http://blog.esoko.com/2011/08/mark-davies-part-ii-content-is-king.html)

But he also conludes that “we should also recognize that content can come from a multiple number of sources. Isn’t that the lesson we’ve learned over the last ten years? That content provided by your neighbour may be equally or even more relevant than that provided by your government, or by CNN.”

Through my experience with the hyperlocal open data platform in Frankfurt, I have realized how little information is available on the local level in Germany – information such as finding out why a red traffic light lasts longer or shorter (by the way,  there is a regulated framework called the “Richtlinien für Lichtsignalanlagen” – Guideline for Signal and Street lights). It is maybe a small detail, but that is what citizens care a lot about in Frankfurt.

So, it seems we need to work on two issues, making more knowledge explicit, particularly local content. And we also need to be aware that most knowledge won’t be on the Internet, and especially piles of opened data will not change that either.

No related posts.

Google Maps has been an incredible service in the past years. Not only it was Google’s engineers, who invented the slippy map, which revolutionized digital maps, but its approach to offer such a service for free and shock competitors with a free routing service. Google has a tremendous overview on all activities on the Internet; billions of search queries everyday say a lot about people’s personalities. With analytics in websites, Google tracks people’s paths from one page to the next.

We are just at the beginning of this massive data collection endavour. TomTom now throws out their gadgets for free just to get real-data from their users. Ironically, it came out recently that they sold the real-time traffic data to the Dutch police. That kind of data collection is not appreciated, all other data collection is agreed on with a small click by accepting the terms of service.

The author Daniel Suarez is worried about the future with his new book: “Understanding the Daemon.” It is still fiction, but that can change soon: “Computers have learnt from us with every Google search, with every „I like“-click. Now they are beginning to change us“. Every time we click, we feed the system called Internet and the outcome is not yet known. Ironically, the features of web 2.0 have incredibly helped to feed the system. Each recommendation, rating, each link, makes the data analysis better. But I am really worried that this is not always for the better. The open available data sets are peanuts compared to data sets of Google or Facebook likes. The question is also what data shall or can be public?

Data becomes more important than hardware. Apple is so eager to collect data that they spy on iPhone users activities and obliged users to do so over their terms of services. I have read a nice comparison: It is like buying a car and you are obliged to not use the seat belt. But things are not for free, although many services draw that illusion. Companies will at one point need to earn money with these adventures.

That’s why Google Maps has changed their terms recently. Websites, which use Google Maps are from now on obliged to “forward display any advertising delivered in the maps imagery“. An obvious and, from a company perspective, understandable move. Perhaps Google will invade millions of pages with advertisement soon. I am sure it is only a matter of time when these companies start to make money with personal data.

That’s why projects like Openstreetmaps are so important because there is an attempt to offer valuable geo data without restrictions. There is an uneven race for getting data open for more transparency and, for example, for better citizen services. Recently, companies, such as Nike, have started to provide open data. However, I get the sense it is just a public relation move; or will they soon provide data up and down the supply chain, so one can follow up how sneakers are being produced? The crowdsourcing potential for such open and free data will keep growing if more and more people join the process. However, these type of data collection is tiny compared to the huge commercial data sets. Ironically, commercial companies now “exploit” Openstreetmap data because it is so good.

That is one big reasons why I co-founded the Open Knowledge Foundation Germany two months ago – to work more on transparency projects.

Related posts:

1. Results of the Open Aid Data Hackday Around 150 participants joined the Open Aid Data Conference in...
2. Open Aid Data conference and Hackday Berlin The past year I have written on many occasions about...

David Wilcox asked me on Quora this question, which I also want to publish as a response here to discuss the topic further. Would be great to get some more thoughts on that topic from you. I imagine we can learn a lot from digital inclusion in Africa. Here are some points:

The art of improvisation

When it comes to access, the innovation under constraints is amazing in Africa. Look, for example, how the challenge of energy supply has been greatly mastered. I think that in Europe we address inclusion only from few angles and should be more creative. We could focus a lot more on mobile phones and offer real needed services even through SMS. Suddenly, we can potentially reach over 90% of people, but most important, we need to play a lot more with technology and hack it where we can. A lot is happening in this regard in the UK, on the contrary Germany, where technological skepticism is still hampering innovations, or where one faces legal implications when offering open wifi.

I like these to posts very much:

• Innovation from Constraint by Ethan Zuckerman
• If It Works in Africa, It Will Work Anywhere by Erik Hersman

However, to me one of the biggest challenges is media competency, and not only in Germany but in the rest of the world.

Service models

If we have not reached enough people through the Internet, it might be that most services do not address a real need and do not offer sufficient help. Isn’t the Internet in Europe largely focused towards the middle class? Where are web solutions or services focused on marginal groups? Here information literacy is the key: “… empower people in all walks of life to seek, evaluate, use and create information effectively to achieve their personal, social, occupational and educational goals” UNESCO

I wish there would be more solutions such as “Rate my Prison“ from the first social innovation camp or or “My Police“. Unfortunately “Rate my Prison” seem not to have been developed further.

Look at the public sector for example and see how little is offered here in Germany. There is a city website, but hardly any online services. On the other hand, Fixmystreet is still a rare exception. The whole world of apps unleashes here a new creativity, but if you really want to get inspired for future mobile services, you need to look at Africa. In Africa solutions such as Farmer’s friend (SMS price information) attempt to reach also poor people in remote areas. Where are such business models in Europe?

No related posts.

iPhone app to check the place of origin of eggs. Photo by Barcoo.com

As open data becomes more popular, I wonder where are the nonprofit and business models for open data? It is clear that somehow open data needs to generate revenues, because it will not only work with voluntary efforts. I did a little research to find interesting approaches to do more with open data.

A good starting point are existing open data initiatives, such as London or San Fransisco. One area of applications are all types of visualizations, which can help to highlight hidden information behind the data. A nice example is Betterflux, which offers a nice visualization tool for the open data World Bank API. Carolyn Mellor desribes in her post “Mining World Bank Data” how to offer paid analysis services using the World Bank API.

Fireworkers

Lorenz Matzat, a fellow blogger from the open data blog of the ZEIT magazine in Germany, wrote about an intriguing case to use open data at the Amsterdam fire brigade. Once a fire alarm starts, all sorts of data is collected about the location and the route to the emergency: Constructions on the way, latest updates from Openstreetmap, the type of house and if possible more data such as construction dates, materials, people living there, etc. A great case of how open public institutions themselves can benefit from open data. However, it is an example of how open data can easily collide with privacy. How many data should be freed for the sake of emergency.

Public transport

Everybody who has a smart phone might have already benefitted from a location-based public transport application, which gives you for example information on bus or train lines close to you. These applications would not have been possible without access to public transport information. In Germany, from my experience, in almost all cases the private applications are superior to the ones from public transport companies. An interesting example of what can be done with such data is the London Live Tube.

Also, ICommute takes the available data from the San Fransisco open data store and offers a mobility check tool. “ICommute SF helps you locate, organize and access route information and real-time arrival predictions for San Francisco’s Muni system. Get the most of public transit and improve your daily commute.” The app costs $2,99 dollars. I would be curious to know how many sales it takes to get at least the development costs back or even make a profit.

Kids life

Again, in San Fransisco an idea came up to provide better information for kids’ lives. “What choices are there as kids travelling to & from school”. After School provides a map for specific locations: Schools, libraries and playgrounds. It also offers places to eat – questionable places such as McDonalds. A commercial approach, again through an Iphone app, is done by MomMaps – It seems they do not offer a “Dadmaps.” Mommaps offers places such as parks, playgrounds, restaurants, museums in over a dozen cities in the USA. The app is for free, but I could not identify the business model.

Food

Nutrition is another interesting sector to use open data, which I discovered lately. Everyblock has for years food inspection data on their website and in the UK there is an Iphone app by the Lichfield district council: Ratemyplace. “Every time a council in the Ratemyplace scheme carries out an inspection of a food business’s kitchen, it’s listed on the Ratemyplace app.”

Another really interesting approach is Food Sprout. It combines different data sets to make transparent how the food is produced, up and down the supply chain. And they also come up with various revenue models. Check out the interview at the great Food and Tech blog. Interestingly companies seem to have growing interest to make their supply chain transparent in their corporate social responsibility efforts. These are the data sources of Food Sprout:

A last example for food is the whole potential behind barcode scanning – you take your mobile phone to the supermarket and scan products to get the information behind the fair trade certificate or behind the company. In the recent dioxin scandal in Germany, the company Barcoo took information from the ministry of agriculture in Germany, of which farms have intoxicated eggs and offer the info in their app. So, you can check in the supermarket the eggs that are fine and not with your mobile phone.

Conclusion

There are still very few business models for open data. Maybe because there is still little open data available and that might be hampering the development. Although if you look at Openstreetmap or CKAN, there are  large data sets offered. Besides Iphone apps, there is also no revenue model and any other is more of an experiment still. It seems way easier to start with open data as a nonprofit project.

Related posts:

1. Open Aid Data conference and Hackday Berlin The past year I have written on many occasions about...
2. Results of the Open Aid Data Hackday Around 150 participants joined the Open Aid Data Conference in...
3. Who to feed? The open vs. the commercial race for data Google Maps has been an incredible service in the past...

Let’s say one day you arrive at work – a mainly knowledge-driven organization, such as a consultancy, where you don’t have an office, not even a position, nor a particular function. So to start your day, you first get a selection of all projects, ideas and problems that your organization is dealing with at the moment.

Half of your working day is already subscribed to ongoing projects, and the other half you could jump into something new. You look at various open tasks, questions, ideas or requests for solutions – all these items have a chronology of contributions and interactions. You can see what has been already done and what is needed.  You find an interesting challenge, estimate the working time and send an invitation to a colleague, who has the skills and might be interested on working together on it.

Now, you have 20% of the day left. You take a look in your competence section and see several questions and help requests for topics. You pick a few tasks, which you can solve quickly and teach others how to do it themselves next time. The daily work plan is done and you go on to a workspace, where colleagues are gathered to work on your main project.

By the way, instead of having a boss, you have different scores you give yourself on your work performance. You might prefer the creativity score, which gives you a lot of time to find solutions and to push for innovations. Or you focus your work on your teaching score, which is evaluated by your colleagues. Or you pick another score, which fits best your working style. And to top it all, strategies do not exist either. This is done by a prediction market.

Is that absurd?

Quora also has an interesting solution to find information from within the ocean of questions. Words of questions become key words (tags), which are then associated with similar questions and clustered under one topic. Imagine such a thing in an organization. You would create organically an organizational wisdom. Why cannot whole projects be organized in such a fashion?

Of course that nice set of features does not automatically lead us to the utopian first part, but maybe it can contribute to it. If we look at the incredible inefficient and non-creative problem solutions capacities of organizations and companies and can overcome the cultural resistance, such open collaboration form would bring us closer to the utopian first part, where you work what you really want.

Related posts:

1. Learning cycling and the persistent illusion that all knowledge can be accessed online Remember when you learnt cycling? The first time you stepped...

Bing bird's eye view of the Brandburger Gate in Berlin

Digital cartography has made map making a lot easier. But If a map contains a lot of data or specific data, it can become a complex or costly adventure. Despite the efforts around open data, still the majority of data is not publicly available, and if so only for high costs. Crowdsourcing is one alternative to collect data for maptivism, but maybe some of these approaches will not soon be needed if automated mapping is further progressing.

Automated mapping

It is quite impressive and a bit scary to see the pace of innovation around digital recognition. Its aim is to make more information available from the offline world. Google is on the frontrun of digital recognition with another example: 3D trees in Google Earth. Google has chosen parks in 50 cities around the world to identify in an automated process trees out of satellite images.

“With 3D trees in Google Earth, we’ve brought characteristic trees to life, from the palm trees that dot San Francisco’s bayfront Embarcadero Street, to the olive trees that cling to the Acropolis in Athens, to the flowering dogwoods found in Tokyo’s parks. All told, there are around 50 different tree species to explore in Google Earth and counting!”

Consequences for mapping

A while ago I blogged about the crowdsourcing Urban Forest Map in San Francisco. Its goal is to map all trees in the city. Now at least the work for the park is not needed anymore if Google is giving out the data. Thousands of people from the Openstreetmap community use satellite imagery from Yahoo to draw shapes of buildings into maps. Is that becoming obsolete soon? What needs to be manually mapped? Of course a lot, because most of such data will not necessarily be publicly available. One example is real-time data. Check this post on maptivism: live tactical mapping for protest swarming.

Here, we are also getting in a dilemma. Such a virtualization of trees can contribute to the protection of forests. Imagine the mapping happens within days and deforestation in the rain forest can be act on quickly. However, what else can be mapped? If trees can be classified, all kind of objects can be classified if digital recognition software becomes increasingly powerful. Check for example the bird’s eye view from Bing, where you can see detailed aerial imagery (see image), not to mention Google street view.

No related posts.

Connection Speed in Europe

On my last post, I talked about how slow websites can trigger higher search costs. In this post I want to further elaborate on the bandwidth divide that exists within and between countries, and which is largely ignored by website developers. There is also the notion that we have unlimited capacities for websites – the more the features, the better the website. On the contrary, especially now with the increasing usage of the web, low-bandwidth websites are more important than ever.

Connection speed comparison

Average connection speeds by Pingdom

Check, for example, the latest “real” connection speed overview from Pingdom based on Akamai data. South Korea has been leading in terms of bandwidth for years, while other Western countries have considerable less capacity available. I did a little heat map to show the geographical variation across the world. Would you have guessed that Romania has the fastest connection speed in Europe?

On the bottom of the list are countries such as Nigeria, Indonesia and Iran, which have 1/3 of users with less than 256kbs per second connection. If that speed is really available, it still needs 6 seconds to load the Wikipedia page example from my last post. Take a book and select a page and then wait 6 seconds before you open it. Do that for a few pages and you will see how frustrating can slow speed can be when doing research.

Website speed and search engine ranking

Although the mobile web is growing exponential, only a minority offers customized websites because most such changes take time, skill and resources. Even Google is taking website speed into consideration. Its primary goal probably is to save resources for their crawlers. In the tool Google webmaster you can see your website’s speed performance. This example is with 5 second slower loading time than the majority of other websites. 

Google Webmaster Tools

Some hints to dive into website speed

You can also use tools such as webpagtest.org and see that, for example, the new World Bank page still has 900 kb to load. This means that with a fast connection, it still takes 8 seconds to load. Check here for details. With a low-bandwidth connection, which is the situation in many developing countries, it takes over one minute to load the front page.

Content Delivery Network

One other great approach is content distribution network (CDN). They distribute your files around the world and put them closer to the user’s end. So, if a user visits your site from Asia, then he gets the files from a server in Asia instead of Europe. Basically, you distribute the same files across servers. Wonder why is Google so fast? Because they have servers around the world. If your audience is let’s say in Uganda, you better not only host the website in the US, but also in Kampala or Kenya. However, it all depends on your audience.

No related posts.

Wikipedia (http://en.wikipedia.org/wiki/Wikipedia)

There are a number of reason why websites are slow; and the results of this lead to raising searching costs and leaving users. A faster website requires skills and resources often not available to great information-rich pages. With a low-bandwidth connection you have two amin major challenges: A slow connection needs to deal with increasing loaded websites, and low-bandwidth is often what you get with a mobile data connection.

Search costs

“Our experiments demonstrate that slowing down the search results page by 100 to 400 milliseconds has a measurable impact on the number of searches per user of -0.2% to −0.6%.”

“The most compelling results reveal that two seconds is the new threshold in terms of an average online shopper’s expectation for a web page to load and 40 percent of shoppers will wait no more than three seconds before abandoning a retail or travel site.”

Bandwidth is not keeping pace with page size

Credit: www.aptivate.org

No related posts.

How and where does most of our learning happens in our daily work life? The visualization above illustrates how intensively the learning process can be, as it can even happen in ordinary places such as the water cooler or the telephone.  At our most common daily places and through our daily used tools is where expertise or experiences (tacit knowledge) are exchanged, and so this is also how and where ideas are raised, e.g. during discussions in places such as coffee houses.

Social media offers some powerful new ways of knowledge sharing, but hence its often asynchronous exchange, it has its limitations with many technical barriers. Everybody using Skype conference calls can tell a story about the technical constraints that this implies. The filter problem is not close to be solved. Services such as paper.li give the impression that we are rather accelerating the information overload.

Social media can reach far more people (e.g. Twitter) and is often the only choice for distance exchange. But can it seriously compensate face-to-face learning? The visualization shall show that knowledge management with social media can support or extend existing practice of  sharing.  But technology-driven communication has many barriers we need to be aware of. It is these barriers we need to focus on more.

Related posts:

1. Learning cycling and the persistent illusion that all knowledge can be accessed online Remember when you learnt cycling? The first time you stepped...
2. Company Christian Kreutz, the writer of this blog, is also the...

Nowadays, if you want to start an open data project, you should rather check the availability of information first, and then imagine something useful because so little useful raw data is available. That’s why for the foreseeable future, at least in countries such as Germany, collecting documents is the way forward.

Frankfurt-Gestalten.de (OpenStreetMap Creative Commons CC-by-SA 2.0 Lizenz. Rendering © 2010 Cloudmade)

Although there are millions of documents available on the Internet, the most interesting ones are hidden in databases, protected as PDF files or only partially offered on websites. It often takes hours to get figures out of a PDF file to be used for analysis. For example, the budget of the city of Frankfurt is offered in a PDF page with more than one thousand pages.

How can you possibly draw different conclusions or see problems and even public misexpenditure? Even a local press paper does not have the resources (any more) to disclose the puzzle of large figure columns.  So I am glad to see my friends at Tactic Tools started a  project called Open Budget (Offener Haushalt) to shed some more light in the public budget of Germany’s government. Other great example comes from David McCandless, who presented them in a great TED presentation called “the beauty of data visualization”. However the remaining problem is that it takes a lot of time to extract  data, not talking about how to present it.

Number one and two are the perfect case. Number three can lead to incredible work. Imagine you copy and paste a PDF document of one thousand pages, which is probably printed as an Excel sheet version.  Number four is even possible with PDF files thanks to OCR, but scrapping can lead to a load of information.

Number 6 is a very different collective approach. For example, the widely cited Ushahidi is an instrument to offer new channels for data collection if one gets a critical base of contributors. The Guardian uses it these days to track the effects of budget cuts in the city of Leeds.

If you do not have the network or public relation budget to run such a crowd sourcing initiative, then you should think about collecting it from existing sources. For example, a lot of data is offered in RSS or XML format. An advantage is the way data is already referenced, such as date, key words and even locations if you are lucky. A nice tool in this regard is Drupal driven Managing News, which I use also for Create Frankfurt to geo-reference all incoming information.

Such aggregating tools allow automatic collection of data. So you can identify such information, subscribe to it and look at it from time to time. Two examples: Public transport congestion alerts or political municipality documents. A year of such data can give you some insights and might lead to interesting conclusions.

Related posts:

1. Who to feed? The open vs. the commercial race for data Google Maps has been an incredible service in the past...
2. Results of the Open Aid Data Hackday Around 150 participants joined the Open Aid Data Conference in...
3. Open Aid Data conference and Hackday Berlin The past year I have written on many occasions about...

A good list of keywords could be a ‘saviour’ while doing a research – it gives orientation, quick access and offers a cluster to find the right documentation. But such a simple list is often missing in most websites because it is not so easy to be achieved. And here a bottom up process, a crowdsourcing of a taxonomy, can be an option.

Image by Beth Kanter

The traditional way
Defining the keywords for a website, such as a navigation, often follows the logic of the organizational structure and hierarchy. It can work well finding information, but only with those ones familiar with the system, and it mostly ends with the ‘try on the search engine.’ After all, as Nielsen points out in his latest brief, users prefer logical structuring or prioritization by importance lists.

There is this wonderful example from the Steve Museums project, which years ago found out that 70% of the category defined by experts were not the ones that previous experts had put together. How is prioritization by importance possible when only 30% of keywords are overlapping?

Tagging through a folksonomy
Another approach is to let users tag content by themselves. Famous examples are tag clouds, which represent the most popular tags of users. Information junkies – like me – are big fans of them, but I have to admit that after various tests and interviews, they do not often work and seem confusing to most users although they are excellent to get a picture of the topics around your community.

Crowdsource your taxonomy
With the lessons learnt, I went to a different approach together with Fredrik Lassen using Drupal. For my open data project Create Frankfurt, we wanted to analyze and structure thousands of documents, so citizens can quickly find information next to the option of geo referenced data. The documents have to do with a variety of topics in urban development – a similar use case for an information management system of an organization.

So, first, we used an API called tagthe.net to automatically extract keywords. The results were more keywords than documents! Then, a tag cloud gave us an insight to major topics, but we had to erase 80% of tags, which did not help us, such as “implication” or “plan.” Here one could see the challenges of automated tagging – we still had more than 3,000 words left, so we started the crowdsourcing experiment.

We invited people with various backgrounds to crowd source a new taxonomy. We built a tool, where a user gets a random keyword that he needs to tag with a meta-keyword. We offered a few first meta-keywords, but left it open to users to create their own.

Over some weeks we had about two dozens of volunteers categorizing existing tags linked to thousands of documents. We are not finished yet, but we have already collected roughly one hundred categories. That is an easy list to deal with. You can merge similar concepts and get good working tags (often used) and less good ones.

Sure it is quite a process to undergo, but it can save tremendous search costs later on and might even involve less time when a few people have endless discussions on definitions.

Drop me an email if you want to know more about the tool and process.

No related posts.

The polyglot Internet is potentially possible, but often limited by its language barriers. A tiny percentage of the Internet content is translated. While famous US American blogs are read worldwide, Chinese, German, French or Indonesian blogs are often not noticed outside of their countries.

There is an interesting presentation from Ethan Zuckerman at TED, where he describes that we are too often “stuck” in our own language domain even though there are so many opportunities to read information or news from different countries, beyond our own. Still, automatic translation is improving year-by-year and many translators worldwide help to bridge the language gap.

Therefore, I was glad to see that a recent German article I wrote about Maptivism in the magazine Kulturaustausch, which was also published on the Kooptech Blog, has been picked by the European blog portal Wikio and translated into four different languages. (All my blog posts on Maptivism)

No related posts.

Picture by Andrew Turner's presentation "Drupal and the Geospatial Web"

Internet of Things

A rather old concept is becoming increasingly real thanks to RFID technology. I have previously blogged about its potential and consequences for development. RFID chips can be attached as stickers to objects, which then can disclose or collect information.
Over at the ReadWritewWeb blog there is a rather funny example of a social tennis racket, which could tell its story – where, with who and how it was played – with the help of such technology.
The logistic sector is using these technologies on a wide scale to track their packages, and soon millions of more objects will be connected to the Internet in the same way.

Location based information

Location based services have been around for a while, but it is lately when they are actually being tacked up. With services such as Foursquare or Gwozilla we not only send our location, but lots of additional information that is around the location. This is an amazing business concept, where users collect “for free” huge piles of information with all sorts of information:

• Location of bars, restaurants, shops, clubs
• An evaluation when a user sends statuses and comments
• Massive social profiles of movements and behaviors.

No surprise Twitter and Facebook have started to send their users updates from specific locations. I wrote a while ago about how you can see, on the example of Google Buzz, that such services are used around the world even in countries you would not imagine.

GPS is included in more and more devices, such as cars, mobile phones, bicycles and even washing powder (!).

World of sensors

Another approach to get huge amounts of data is through sensors, which measure all sorts of factors from our environment. The idea is that soon low-cost sensors will be available, for example, to measure noise, air quality or one’s physical condition. Such sensors can also be RFID chips, but can go even further. These sensors could be included in a watch or mobile phone; this way millions of people can deliver real time information. Sounds like science fiction, but there are already some crowdsourcing projects “using humans as sensors“. Citypulse wants to measure the air quality though the contributions from pedestrians.  And with a smart phone it is easy to join a project to measure the noise level worldwide.

The senseable city lab from MIT using such methods. For example, the “Trash Track” project, where 3000 sensors were added to trash bags to analyze the different ways pieces of trash are taken through the disposal chain.

Implications

The list can be easily extended; please add some further methods in the comment section. But why have I written this list? Because I want to describe how pervasive the process has gone and that it has had far reaching consequences, which not everybody is aware of.

First, positively, these means extend the richness of data on the Internet available. Second, it can offer more and better information faster. Third, thanks to open source and fairly cheap web services these tools become available for many more people. And fourth, if it is offered as open data to everyone, it can help create useful web services. I will elaborate further on user scenarios in upcoming posts.

However, I also have a lot of concerns and questions. What about ownership, often the lines blur. Who owns the data and to which I as an individual control the information about me anymore? Where does this data collection lead to in a few years time, when companies like Foursquare with millions of social profile data are in complete different hands? Do we really need to digitize everything that is possibly modifiable?

Michael Gurstein has a great post, in which he expresses concerns that open data might only help the people, who already have an information advantage (e.g. access, research skills).

What is the sense of collecting all this huge amounts of data? Or is a lot of that data collection nonsense, because it limits or has even little or no meaningfulness? I will elaborate further on these questions in my next posts.

First part of the post.

Related posts:

1. Who to feed? The open vs. the commercial race for data Google Maps has been an incredible service in the past...
2. Open Aid Data conference and Hackday Berlin The past year I have written on many occasions about...

The same way the fishing industry has found more efficient methods to get most fishes out of the oceans, is exactly how we find more ways to digitze information that was previously only available offline. Imagine a massive fishing-net bringing us the greatest fishes, but emptying the oceans. What would be then the fishing-net or, in this case, the opportunities and consequences of digitizing all the information? Nobody really knows.

Thanks to the Internet, we now double every two days all stored information. The estimated amount is 5 exabytes according to Eric Schmidt (Google) and it took human kind 2000 years to get a similar amount of archived information. Traditional governments and companies collect information and stored it as digital data. The non-profit sector is increasingly engaging in such efforts because technologies have become more widely available in many cases even as open source software.

Text recognition (OCR)

Text recognition software has become very sophisticated and can understand even hand written texts. Cloud services such as Evernote analyze each note uploaded for texts, being it a business card, wine label or any other document. Thanks to such softwares the UNdemocracy.com project scanned thousands of documents by the United Nations and offered better search capabilities. Another project by HP Labs in Bangalore wants to offer an email service written by pen on paper. An image through a mobile phone and text recognition shall make it possible. And if it is not recognized by Optical Character Recognition (OCR), then our support through the well-known reCaptcha helps make sense of words. For Recaptcha we help decipher texts from medieval books.

Voice recognition is far from being new, but it has become much better over the years, and its services are much easier available. Latest voice actions make such a service available for all Android driven mobile phones, where you can read outloud, for example, the text of an email. Ushahidi text-based service is now also available by voice to report about incidents. They work with Cloudvox a voice service application. This and services such as Twilio make such voice affordable and available to low budgets previously reserved only to companies and government.

The open source solution Freedom Fon even offers an interactive voice response system so, for example, iliterate people can provide information.

In recent years a lot of mobile softwares for data collection have been developed. Here are some open source solutions. Mobile phones are in the hand of half of the world population and many collect data passively or actively. Google collects already data through a GPS, if users have accepted to join it, for traffic information. In remote areas innovative solutions try to bridge the 160 character limit of SMS. A simple paper wheel is used to report critical health information from the country side in Cambodia.

More and more cameras and particular mobile phones have a GPS functionality and increasingly the photos are uploaded to the Internet. Flickr photos in the world map gives a first impression. Flickr has already 117,025,830 geotagged items. In few years in most locations around the world a series of blog posts are available. Google Street view will then “only” have the streets. Talking of Google, their Goggles service tries to deliver additional information on physical objects. Take a photo from an object such as a restaurant menu or a sight seeing spot and it will provide you with information and store the image in a database.

In the next days comes another post with more examples and thoughts on consequences.

Related posts:

1. Who to feed? The open vs. the commercial race for data Google Maps has been an incredible service in the past...
2. Open Aid Data conference and Hackday Berlin The past year I have written on many occasions about...
3. Learning cycling and the persistent illusion that all knowledge can be accessed online Remember when you learnt cycling? The first time you stepped...

If you sit in front a long list of information, it is often difficult to make quickly sense of it. If you look at a map of the same data, you might get a picture of it rather quickly.

1,470,000 US gallons of oil were leaking from an oil pipe of British Petrol in the Gulf of Mexico everyday. What does that tell you? When you look at this map from ifitweremyhome.com, then you get a better sense of it. The oil spill size was as big as Southern UK.

ifitweremyhome.com

If you do have some data, you can start working with a new great tool called openheatmap.com, however if not, then you need to collect it yourself.  One way is done by the grassrootmapping.org project, which has done a great initiative to document the oil spill on the coast line. Another one is the Oil Reporter from the Crisis Commons group. You “only” need an Iphone or Android driven phone and you can get the app to report where oil is found.

urbanforestmap.org

www.mappiness.org.uk

Context is king and therefore some other projects want you to share information to find out more about your environment. One example is the Urban Forest Map project: “The Urban Forest Map is a collaboration of government, nonprofits, businesses and you to build an inventory of San Francisco’s urban forest.” Citizens create an inventory of trees in their city and get exact information about the ecological impact.

Another mapping & crowdsourcing project is a research project, which founds to create a location based happiness index. Through an application, volunters are asked throughout the day about their mood and that information plus the location is then collected. Mappiness shall help to understand “how people’s feelings are affected by features of their current environment—things like air pollution, noise, and green spaces.”

A similar project called Mapumental, done by Mysociety. Watch the video for that amazing idea.

In Germany a similar project, called Mapnificient, has been done. At the moment we try to implement the same for Create Frankfurt.

No related posts.

For years, I have heard that the web is becoming a platform, where one can easily use a website as a tool box.  Community sites installed just with a few clicks and knowledge sharing communities are on their way. But as in most cases, there is always a hard point, and in this case is to get people involved. Although technology plays a less important role, it is still a critical factor whether people like to join and engage. Many ready-made-website fail to deliver the most important thing: To help people find stuff and help them exchange.

Technology constraints vs. user needs

Of course there are examples of simplified websites, which focus on user needs such as Gmail or Flickr. But knowledge sharing in a community is often way more complex if you have to combine different forms of media, a library of existing resources and so forth. I have tested numerous platforms and until today I have just been disappointed because each time I had certain needs and always had to put these under technology constraints. Although it should be the other way around and technology should help me make it better and easier.

Professional circles vs. passion driven communities

One big differentiation is important. I am talking here about professional communities, where people exchange about their work. These communities are much more difficult to establish. If people have a real desire to exchange ideas, for example, about their hobbies, a bizarre designed forum could work perfectly and would be more dynamic. There are many examples of dynamic communities that existed way before they were named Web2.0 or social media. To me, work related communities focus a lot on the knowledge management principle: Find the right information where and when it is needed.

Information seeking vs. engagement

I have less time. Nowadays, I can engage in so many communities, mostly when I am not even online. Why should I also join in your community? Perhaps because this community gives me the information I need: quick, easy and maybe even in high quality.  From this perspective, for example, a Ning website is a disaster. Flashy, blinky things focus on a personal presentation instead of orientation, coherence and relevance. Have you ever tried to find something on a Ning website?  How many communities are really active on Ning? Okay, to be fair, technology and a user centered website is one factor, but let me raise some more points:

• Most of these social network websites are features loaded and focus very little on plain people to people exchange.
• Although you have as a webmaster a set of options to change your network, it is not enough. You need to be able to tweak in detail to make a platform user-centered. Listen to your members and make changes according to them.
• Facebook, Linkedin, etc. are great tools to network and to mobilize, but offer almost no flexibility to build a community of practice. It is the Facebook way or no way. More important these big platforms have no interest in real exchange and learning. It is not their business concept.
• A questionable approach are widgets, where content is distributed all over the place. Photos here, documents there. A RSS feed is easily another information stream with little relevance.
• Each community has its own culture of exchange and different requirements. It is so difficult to find the right platform for that. Instead one needs to squeeze requirements to technology constraints.
• Confidentiality. Latest from Facebook, it is always a risk to give your data to such a platform. Also other platforms have a clear exit strategy for their users. You can leave easily and take your data with you.
• Community develop over time. It is impossible to foresee what is needed and what not for your platform in the future. Often a wonderful list of features is tempting, but so much is not really needed later on.
• If you want to make information finding for users the easiest possible way, the whole information structure behind a site becomes easily complex. The more is done through intelligently tagging in the background, the easier it becomes for users to find something.

Alternative?

Either you are lucky to find a service that offers you what you need or you opt for the usual stony way to build your own website. I know it is not a great alternative, but I believe we have to realize that so many web technologies are still in its infancy and we have just started to focus on the users as the center point of such technologies. I know it is not the best to shot at Ning as an example alone, but their latest turn in their payment policy shows a typical dilemma. Nobody knows where such providers are in a few years time.

No related posts.

There is often that notion that once you have access to the Internet or to other information and communication technologies (ICT), the whole world of information lies rights at your feet, so you only need to pick the best of it. But in contrary, it can become incredibly time consuming to verify information and to make yourself a trusted source. In the field of ICT4D, this issue is particularly important. In many cases people do not have years of experience working with ICTs and have actually learnt them just the auto-didactic way – using the Internet for their own benefit. Let’s take the case of farmers in rural areas of Africa.

Kenya. Photo: © Curt Carnemark / World Bank

Farmers in developing countries

The other day I had an interesting conversation with a colleague, who has been working already for decades in the rural development and agriculture field around the world. We talked about the potentials for ICT in agriculture and in specific farmers. One of the major challenges is neither access nor literacy, but simply trust. Why should a farmer trust an information coming from somewhere as an SMS? Farmers make careful elaborations, before they change certain practices. Information from a website can help, but at the end of the day what counts is the advice of trusted colleagues. So, we have to realize that information through ICTs often have only a small impact.

ICT and agriculture

In the case of agriculture, behavioural change through extension advice is even more difficult to happen through ICTs. For decades, it has been well known that advise has no effect if simply some guides and brochures are sent to farmers. More effective is a participatory process, where farmers learn from each other directly. So why should farmers change practices they have done for years when they get advice through SMS or any other channel? “It needs a lot more than ICTs,” pointed my colleague.

In contrast, market information particularly through mobile phones is more successful because of the price information, which are much easier to trust than a particular advice for the next cultivation. But even in this case, they had to be introduced in groups of trusted people. Otherwise, who would trust a SMS from anywhere? Would you? There have been cases where rumors spread through SMS have even led to violence.

Demystify the Internet in Rural Africa

Take yourself as an example, how many sources do you really read or how many people do you speak to before you can take the information for granted? ICTs allow for incredibly easy publishing and disseminating of information; but the information is still not worth a penny if it is not trusted. Linda Raftree has a great post, where she describes her experiences during ICT training courses in Ghana.

“Another question that surfaced was ‘Is the internet true?’” This led to a great discussion on how information comes from all sides, and that anyone can actually put information online. It’s truth, and anyone can’t believe everything one reads, it’s not regulated, you need to find a few sources and make some judgment calls.”

Although all this information is not new at all, so many ICT approaches forgot exactly about that challenge. The information has to only be delivered somewhere and that should bring change. The problem is that trust is built slowly through social relations and these take a lot longer to grow online.

No related posts.

There are more than one billion people, who are members of some sort of social network website. But, will these websites be the future platforms, where we can engage to create social networks? I doubt it. The future will go back to your address book, where it has always been.

Photo by Zheem from Flickr (CC)

Address book

An address book can combine contact details and much more information people add to it. Now, imagine these contacts are updated constantly adding new information available depending on each person and the social network they belong to. The HTC android driven mobile phone has already got that function. It connects you to certain social network websites and matches them to your address book. So, it really does not matter what website is behind it, you simply get an overview or stream of information of your friends’s and colleagues’s activities at various websites under each (online active) contact in your address book:

Own control vs. Facebook control

Photo by laihiu from Flickr (CC)

Now, the best way to decide what is public or private and who shall see what, would be if you have complete control over your information. In the case of Facebook, they want to do that job for you and that is the problem. If one looks at the continuous privacy issues of Facebook, it is about time that information gets protected from companies, which change their terms of services every second day (e.g. Apple).

Steve Boyd writes: “One simple observation is that users will not be able to get the privacy they want (or think they already have, or at least had in the past) in today’s Facebook.” and he concludes that Facebook, therefore, is not the future.

However, the answer of many social network websites is to collect as much personal information to become your personal address book. The Facebook friend finder stores all the information forever and therefore comes up with intriguing friend suggestions.

Reality check

The problem is that there is not really a serious initiative to work on an alternative open system, where users could control their data and give social networks access to the information they want to share. Still, in a call for a Facebook alternative, people donated in short time a hundred thousand dollars.

Another, albeit failed and disappointing, attempt was Open Social, initiated by Google about two years ago. The good news, however, is that even Facebook will have difficulties to be the decisive player. That might (hopefully) be history soon.

No related posts.