Data for development: Supporting communities through data analytics

Editor’s note: Today’s guest post comes from Ryan Schlief, Executive Director of the International Accountability Project (IAP). Ryan tells us how IAP took part in one of DataKind’s  DataDives—a weekend-long problem-solving sprint—hosted by Google Cloud in June of 2018, where pro-bono data scientists came together with mission-driven organizations to address the negative effects of international development.

It was a Friday night when I arrived at Google’s office in New York City for the DataDive to join over 100 volunteer tech specialists and dig into how we can use data to help support communities. As more people arrived, the energy in the room was more comparable to a concert than a data event. It felt as though my favorite singer might show up at any moment to surprise us.

In this charged environment, I soon wondered if there could be a better way to spend a weekend than diving into data?

Using data to help support communities

IAP’s mission is to make information on development accessible to communities and activists. Imagine you find out a major highway or dam is being planned for construction near where you live. You’d probably want to know every detail about the project and how you could be a part of the decision-making process. Yet all too often, communities learn about these projects only after the plans and financing have been finalized—giving them fewer opportunities to provide feedback on the project and its potential impacts. IAP’s ongoing work to reinforce community-led development gives communities and activists access to information about proposed or even existing projects so they can get involved.  

To improve how a community can inform and participate in the current development process, the information exchange among governments, development banks, private actors, civil society and communities needed to change. That’s why we created the Early Warning System—to exchange accessible information about the newest proposed projects with the groups living and working nearest them. One important part of this Early Warning System is Nyali, the first civil society-led software to collect, standardize and organize proposed development projects the moment they are disclosed by one of thirteen major development banks. Nyali is the Chichewa word for ‘lantern’, a simple tool anyone can use to illuminate the darkness. It inspired us to create a simple process and an accessible tech tool that is able to illuminate the complex and opaque development finance system and encourage community-led development.

Once a project enters our database, the Early Warning System team verifies the project information and writes a summary or ‘snapshot’ of each project. We deliver these snapshots with original project documentation via email, secured mobile messaging, in person, or through one of our 400 civil society partners to the communities who need the information most. The Early Warning System database grows by about 7-10 projects each day, so we’re always looking for help reviewing projects, distributing snapshots and reinforcing community-led responses. Since accessibility is vital, the project summaries are translated into almost 100 languages.

Recently, we’ve come to realize that a publicly searchable database of reviewed and summarized projects could also benefit international civil society groups, development experts and academics who are also advocating for community-led development. This extends the potential for information-sharing. For example, if a development bank does not provide all the information a community may need about a project, a news organization could report on these missing details. Local governments could publicly promote projects to encourage public support or private investment.  A civil society newsletter could document local participation in a project. A business magazine could share more details about the project’s investors.

Using machine learning to automatically match news articles with projects

For the DataDive, we asked our volunteer tech specialists to help us automate the matching of news articles with the projects in our database through the power of machine learning. We presented visual graphs, illustrations, work stream charts, and blog posts to introduce our work to these volunteer tech specialists, and soon enough we had assembled a dedicated and informed team who immediately set out building a software solution.

While my favorite band did not show up that Friday night, two of our partners did. Accountability Counsel challenged the tech specialists to improve how it tracks and analyzes the dozens of complaints filed with development banks. And Inclusive Development International presented the challenge of creating a search engine which would allow its internal staff to find project information on the websites of these development banks.

Through Saturday morning and night, music, laughter and good food kept us going. The DataDive volunteers tested and experimented their ideas in small group meetings or in online spaces like Slack or GitHub. A wide array of doughnuts welcomed us on Sunday morning. After a collective meeting to tie up loose ends of code, my IAP colleague Preksha and two amazing tech specialists who led our team presented the weekend’s work. In preliminary tests, the software was able to predict likely matches between news articles and projects tracked in our database with 67 percent accuracy. And since we’re taking advantage of machine learning, our software  will learn from itself and the predictive rate will improve over time. We hope in the next year, as we test the software internally, we’ll be able to share news updates with partners around the world on the projects they care about the most.

Because of resource and skill constraints, many civil society groups are not able to maximize data, whether internally to understand their effectiveness, or externally to strengthen their activism. Data and technology also can be intimidating. Dedicated support from DataKind and the tech specialists at the DataDive helped us break down these limitations and transform our work. Isn’t that a worthwhile way to spend a weekend in New York City?

The DataDive was possible through the generous support of DataKind, 11th Hour Project, Elsevier Foundation, Google Cloud, Teradata and more than 100 volunteer tech specialists. To read a live blog of the DataDive weekend, check out this excellent post by Elsevier Foundation.