'BRIDGE' to Somewhere: Progress to Date

Technical development of the first Basic Registry of Identified Global Entities (BRIDGE) release is currently about halfway to completion. If you've been following the project, you know that BRIDGE aims to revolutionize data interoperability in the global social sector by uniquely identifying all the world's NGOs in one database. In the post below, BRIDGE project manager Chad McEvoy ([email protected]) checks in with a progress report. The post originally appeared on the Markets for Good site and is reprinted here with permission.

Considering its ambitious scope, the project has unfolded as efficiently as we could have hoped for, given that it's dependent on collaboration between four distinct partner organizations -- the Foundation Center, GlobalGiving, GuideStar, and TechSoup Global -- represented by contributing team members on both sides of the Atlantic, who are in turn coordinating with a Polish software development firm, all the while consulting with a technical advisory group composed of six extremely busy specialists and sector thought leaders — themselves spanning three countries. Nonetheless, I am happy to report that the foundation of BRIDGE has been laid and we are making steady progress.

From its inception, BRIDGE has been conceived of as something that will have the potential for unexpected positive outcomes.

Just as no one involved in the creation of Universal Product Codes (UPCs) in the 1970s could have anticipated or planned for the development of the current crop of smart phone scanner apps, we expect BRIDGE to provide a foundation for future innovation, but we don't yet know precisely where that will take us. We know BRIDGE will have far-reaching implications, perhaps revolutionizing philanthropic information-sharing, but we can only begin to imagine the breadth of the project's ultimate impact.

In recognition (or perhaps even celebration) of this, we have built flexibility into both the process and the product. For example, we are using the Scrum Agile software development methodology — a process that allows for on-the-fly adaptations and adjustments. Agile is particularly well suited to a project such as BRIDGE where we know generally what we want to accomplish and the unmet needs we want to serve, but we don't necessarily know precisely how everything will fit together in the end. This is in contrast to more traditional ways of building things, such as an actual bridge, for which explicit and largely static specifications about the end product are necessary to have in hand before any work can begin.

The creation of BRIDGE has been a balancing act. We have prioritized the big decisions so we may preserve our future capacity to accommodate unanticipated needs while still maintaining a coherent overarching vision. So far this approach has led us to develop a cloud-based non-relational (NoSQL) database utilizing open source technologies. For now, even the BRIDGE numbering system itself — which aims to assign every nonprofit and NGO, as well as their programs and projects and other social sector entities across the globe, a unique identifier, like a numerical fingerprint — will be basically unstructured so as to allow for maximum future flexibility. And the selection of a non-relational database will allow BRIDGE to easily scale as it grows larger. Developing such a flexible system removes any limit to the ultimate size of the database while preserving performance speeds. It also potentially will allow us to better accommodate currently unforeseen and less structured data varieties with an eye toward the emergence of Big Data in the social sector.

Being globally hosted in the cloud will also be a great advantage to BRIDGE. Being cloud-based means we can continually grow without needing to invest in physical hardware infrastructure upfront. If we need more capacity, we can automatically allow for it; if we don't, we don't even have to pay for it. It also means the database will be fast and reliable anytime from anywhere in the world that there's a decent Internet connection. Ultimately, being in the global data cloud is ideal for BRIDGE, not just from a technical standpoint but also metaphorically as the project becomes an international tool for use by anyone — from Patagonia to Siberia — seeking to improve collaboration and communication in the social sector.

Okay, enough with the platitudes; let's get wonky for a moment. Here is an overview of what has been accomplished to date (and I promise a translation will follow):

• creation & configuration of the MongoDB database, development of the BRIDGE data schema, and importation of the initial sample data from each of the core team member organizations;
• development of the initial API functions, basic user interface, and BRIDGE ID generation mechanisms;
• creation & tuning of the data de-duplication process and matching algorithm, including geo-coding, addresses normalization, threshold generation, and fuzzy matching;
• implementation of data indexing & retrieval through ElasticSearch; and
• configuration & deployment of test systems to Amazon cloud servers.

In plain English, what this all means is that the infrastructure of BRIDGE has been delivered and the first iterations of the most important initial features of the system have been created and are now in the process of being tested, tuned, and refined. The core structure is there, and now we just have to make it function well. And that's exactly what we'll be doing over the course of the next few months. Among other things, we plan to create and deploy the first iteration of the secure user interfaces that the core team member organizations will begin using for in-depth testing of the tool. In addition, we are going to create a user management system and perform several improvements to the API layer. Most critically, perhaps, is that we expect to spend a lot of time focusing on deeply customizing the de-duplication process to the specific needs and idiosyncrasies of the global social sector.

In human terms, BRIDGE is a vital cog in the greater mission to change lives. It is a platform that will, at a minimum, allow us to answer simple but perennially vexing questions that stunt effective interventions, questions such as: Who's doing what — and where? Are our efforts too concentrated, or not concentrated enough, in specific geographies — and on specific problems? Beyond questions such as these, the exciting thing is that we can begin to generate new uses, tools, and insights as the platform matures and people around the world put it to their particular uses.

Stay tuned. We look forward to sharing more updates with you along the way.

-- Chad McEvoy