The benefits of crowdsourcing for crisis information were brought into the limelight in the aftermath of the Haiti earthquake, when Ushahidi, a platform originally developed to map reports of violence in Kenya after the post-election fallout in early 2008, was deployed to provide a map of vital information based on reports coming in from social media and other channels.
The information provided by Ushahidi helped people and organizations on the ground in Haiti direct rescue and other relief efforts, and find emergency care and other vital information. During the crisis, the Ushahidi team quickly saw the need for a tool that could not only take in crowdsourced information, but quickly and effectively filter the tremendous volumes of that information coming in, ultimately leading to today's launch of SwiftRiver, a new open source platform that will have the capability to not only gather as much information on a particular event from as many available sources as possible, but also deploy a range of tools to filter and validate the information provided. I had the chance to ask Jon Gosier, Director of Product at SwiftRiver and a TED '10 Senior Fellow a few questions about the new platform.
Allie: With the earthquakes in Haiti and Chile, and now the devastating floods in Pakistan, this year has brought the concept of crowdsourcing information during disasters and the benefit of applications like Ushahidi into the limelight. How did these events help inform and influence the development of SwiftRiver?
Jon: Haiti was truly a baptism by fire for myself and the rest of the team. Ushahidi brought me on board in December to begin building Swift. I spent the first month doing some research and inspecting the original prototypes and interviewing the staff about the problems they were facing dealing with crowdsourced info. I spent January designing the system and planning an 8 month workplan. Then, in February the earthquakes happened, and all that meticulous planning had to go out the window!
To make a long (stressful) story short, we were informed by what wasn't working. The fact that we had around 25,000 crowdsourced reports coming in per day with a small staff and a small group of volunteers processing that data was a problem we had to deal with. We had an event that devastated this tiny country and the people living there, and we were crowdsourcing info that had to be acted upon by organizations on the ground while we were thousands of miles away in Kenya and Uganda. So the information we passed along had to be vetted, it had to be true, and if it was an urgent situation (like someone being trapped or missing their child) the reports had to be escalated to proper authorities.
We had a finite number of people and a finite timeframe of response (people whose lives are being threatened have no extra time). On top of that, the number of incoming messages from SMS, Twitter, Email and the Web seemed to be growing exponentially with each passing hour. Since we couldn't take a collective 'time-out' to review every single message, we had to think of ways that we could use tech (algorithms) to maximize our time.
The goal of Swift is not to make algorithms replace human verification, but rather it's to speed up human operations.
Allie: If I understand correctly, the ultimate problem that SwiftRiver aims to solve is to help journalists and relief organizations more effectively and efficiently sort through the huge volume of real-time information during a crisis without sacrificing accuracy. How do you envision the open source platform ultimately moving from developers and into the hands of these users?
Jon: You're right, SwiftRiver is a platform not a product. It's made up of five different web APIs (Swift Web Services) and several different applications that combine those APIs in different ways. All of these products are free for basic use. For users who need premium access, they either take the code and build their own, or we offer paid options that they can use if they prefer not to do a lot of work.
What this means is any person, whether they are in Des Moines, Iowa or in Blantyre, Malawi have access to the same set of highly powerful, customizable intelligence gathering programs. A blogger might use our platform to look for data he writes about, an NGO might use it to respond to a disaster, a business might use it to perform market research... We tried to build a platform that's completely agnostic to the data it processes.
So the way I understand this question is, "how does the average person use SwiftRiver without getting into the all the minutia of code and setting things up?" The answer is that we've got several solutions in the works that will make it easy for everyone.
Allie: You’ve talked previously about the concept of distributed reputation and its importance in helping users prioritize trustworthy and nontrustworthy sources. How does that concept work within the SwiftRiver platform?
Jon: For us it's about history. When organizations 'crowdsource' data they are usually dealing with all these new sources of information that they've never interacted with before. So the information, the people sending it, their motives, their backgrounds... all these things become variables. This is why there's such a strong resistance by some humanitarian groups to using crowdsourced reports at all.
Our product we're working on that deals with this is called RiverID. RiverID builds a profile of users as they send in data. If the organization has information like phone numbers, email addresses, social media profiles and so on, they can build a profile for users who may repeatedly use the system.
You will still have a majority of first-time users and people who choose to remain anonymous, but at the very least you'll know if the same phone numbers or twitter accounts that were active during Haiti are also the same ones actively reporting during the floods in Pakistan. Also it benefits the users sending in reports. What if they want to be known and respected for how active they are at helping to vet crisis data? Now they can take their profile with them to new Swift communities that say, "I am person X, these are my details, and I was an active and trustworthy volunteer during the Haiti quakes."
Think of it like a passport that Swift users and administrators can opt-in to using. Facebook Connect lets you take your social profile with you to to different websites; we want to do something similar for people verifying crowdsourced data.
Allie: You make a good point in your UX Magazine article that the web is not a primary mode of communication in many of the countries where disasters occur. How can more traditional and offline modes of communications be incorporated using the SwiftRiver platform?
Jon: There's a A LOT of advanced intelligence and analytic software that focuses on web communication. There's also a lot that focus specifically on Twitter and email. There's virtually none that try to pull it all together to make connections between information that comes in across these channels.
Living in Uganda for the past two years, I saw a lot of humanitarian groups coming in with amazing products for both. Then they get out to a rural community here and everyone's using their mobile phone for communication. The usual response is, "we need to get these people on computers so we can help them use our Twitter apps." That sounds ridiculous, but it happens more than one would hope.
We took a different approach, because we wanted to be relevant in all parts of the world. How can we apply the same advanced semantic technologies we use for Twitter, Email and RSS feeds to SMS as well. So our goals were to:
- make our tools open source, so that anyone can use them,
- make them available for offline use (because not all real-time data is on the Web - again, SMS), and
- make it distributed so we don't touch anyone's data if they don't want us to.
David Kobia and Brian Herbert at Ushahidi just wrote a great plugin using CloudVox.com that allows people to use a regular phone to call in reports of incident. That data is then transcribed as text and submitted as a report to Ushahidi. So we can even do voice now thanks to them. We're also using the Google Languages API to offer Swift for 50+ languages that we'd never be able to do on our own.
Allie: A recent BBC article notes that the SwiftRiver platform could also be used by other industries and situations beyond crises. Can you give us an example or two of how that might work?
Jon: Oh sure, I can give a few actual examples of usage right now by some of our pilot partners. Most recently we were contacted by a medical company who wanted to use our natural language processing APIs to help sort internal communication within their organization to apply automated summaries, like who this message is about or what it's about, while using those key terms to find related messages across departments.
PubHealth.org is using our APIs to power their new blog. They wanted an aggregator that scrapes data from all across the web about public health information. Their problem then became: if we're scraping a thousand websites, what makes it to the front page? So we used our APIs to build an aggregator that was one part Techmeme.com and one part HuffingtonPost.com. Half the editorial decisions are made algorithmically and the other half by a human editor.
We've also been contacted by international news agencies, advertising agencies, artists, bloggers... even governments. It seems we've attempted to solve a problem that affects quite a broad group of people.
Tags | 
Fearless Focus: Harris Wofford, Peace Corps Co-founder
