From Bluelink, a member of Cohort 4.

Here at Bluelink, we build data integrations for Democratic campaigns and organizations. After launching hundreds of pipelines for the 2020 election cycle and beyond, we’ve learned a lot about what it takes to build a good data integration. If you’re a political tech startup looking to enable integrations between your product and the rest of the political tech ecosystem, here’s our guide to getting started.

What are integrations and why are they important?

Syncing data between all the systems used is critical for campaigns and movement groups to be as effective as possible and provide a good supporter experience. Many of us have experienced what happens when campaigns don’t have that data integration. I donated to one campaign six times — and they only asked me to volunteer after the sixth time. A friend donated to another campaign and a week later got a text asking if they support the candidate. How much more effective could these campaigns have been if they had complete data synchronization between their donation, volunteer, and texting systems?

As a political tech vendor, you can help enable this better supporter experience by enabling and encouraging data integrations between your product and others. In fact, you will find some clients will require integration support to their existing toolset before they are willing to sign up. They might want the data collected from your product synced to VAN or EveryAction, where they can have one view of all their voters. Or to Mailchimp, where they can email their supporters and follow up with groups of people. Larger campaigns and organizations with their own data analysts might want data synced to Redshift or BigQuery so they can run their own analyses on it. 

Customers might also ask for data to flow in the other direction, from other products to yours. For example, they might want to cut lists of voters in another tool and then be able to upload that to your tool to contact those voters.

By enabling these use cases, you will not only improve your product and help your business grow but also drive the better supporter experience and outcomes we want to see across the progressive space.

Designing your product with integrations in mind

As a busy startup, you probably won’t build out a bunch of integrations right away, but there are some things you can do to make integrations easier to enable in the future. Here are our top recommendations:

(1) Think about what data you might need for integrations — even if you don’t need it right now.

If you’ll want to build out integrations in the future, it will be easier if your product already collects the data you’ll need. It’s especially important if current customers will want to sync over past data.

For example, if your product is a texting tool, you might let customers upload lists of phone numbers to send texts to, along with other data like each person’s name to customize texts. Your tool probably doesn’t use email addresses so you might not think to include email address as a column in the file. However, many CRMs use email address as a primary identifier, so making it easy to upload a list that includes it and returning it as part of a campaign result would make it possible to sync results back to many CRMs.  

(2) Make sure your data can be read by computers, not just humans

The first users of your data might be people looking at and interpreting the results. But when you build out automated integrations, computers will need to be able to understand each piece of data. 

For example, we worked with a vendor that allowed clients to customize the opt-in checkboxes on their UI. Some clients wrote, “Click here to opt in to our mailing list”. Others wrote things like “Check this box to be removed from our mailing list”. Having seen the UI, it’s easy for a human to figure out if a user should be opted in or out — but very hard for a computer. The vendor solved this by adding an additional field with pre-defined options for the behavior of the checkbox. That way there was a computer-understandable description of the checkbox as well as the text that end users actually saw.

(3) Consider how your data maps to other systems.

As you make decisions about your product, it can be useful to look at other products in political tech and think about how your data will be able to map to those systems in the future.

For example, if your product asks survey questions to users, your customers might want those responses synced to EveryAction. EveryAction only allows multiple-choice questions, so it will be easier to sync your data over if your questions are also multiple-choice, rather than free response. Or you could make a plan to group free-response answers into discrete buckets in the future.

Of course, you won’t just build your product to optimize for data integration — you’ll want to balance that with other considerations like user experience. But it’s good to keep in mind how your choices will affect future integrations and take it into account when making decisions.

(4) Consider how data from other systems can make your tool better

While your first automated integrations might focus on getting results out of your system, you should also consider how getting data into your system in a simple way can make it more effective. There are two aspects of data ingestion to consider. The first is administrative. If your system needs inputs, like a target list, think about how to make sure that data imports cleanly and easily. You might also want to import data that “closes the loop” after a user leaves your platform, like donation records or clicks, so you can do analytics and provide reporting on campaign success. The more you can automate this process, the happier and more successful your customers will be.

The second, more subtle, question you should ask yourself is how your tool can be used more effectively with more information. For example, if you are a fundraising tool you might not see a direct need to import volunteer history. But supporters like to be acknowledged for all their work, so maybe you want to allow for a custom message thanking them for their last shift, with the exact date and time. Once you start down this path, you will be impressed by the number of ideas you, and your customers can come up with on how richer data can improve the effectiveness of your tool.

Building your data integration tech 

Once you’ve thought about the data you’ll collect, send, and receive, the next step in enabling integrations is to start building the tech. One way to enable integrations is to write code to sync data to each third-party system yourself. But it can be easier and a lot more flexible to let other people build their own integrations too. The best way to enable this is to make your data available to other (approved) software developers and data analysts and let them build out integrations. There are a few ways you can make your data available:

• APIs — This is by far the most common way to let people access your data. APIs let you control what data is available and in what format. They are flexible — you can change your underlying database structure while keeping your APIs consistent. They’re the best way to let people write data to your system as the other options are generally read-only.
• SQL mirror — A SQL mirror is a copy of your database that people can query directly. This can be good for data analytics use cases because it’s already in a queryable format. It’s good if users need access to every single piece of data you collect. It’s also good for extremely large amounts of data that could be slower to send via API.
• CSV dumps – While the previous two options are most common, any way of making your data available is a good start. Writing data to a CSV once a day and making that available to others can be a quicker and easier way to provide data for others to use if you don’t have the resources to develop another method.

Finally, once your data is accessible to others, how do you actually get the data flowing to other systems? One option (of course 🙂 ) is to use Bluelink. We can ingest data from your API, SQL mirror, or whatever you provide, and sync it to dozens of political tech products and other commonly used software. There are other good options out there too! Products like Zapier can enable integrations with many commercial products. Parsons can make it easy to build your own integrations with Python. 

When you’re deciding what platform to use to build out integrations, here are some things to evaluate:

• Scalability. Political data is unusual in that you might have only small amounts of data for most of the year, and then huge amounts of data leading up to an election, especially during GOTV. And of course, the few days with the most data are the ones where it matters most that the data gets synced successfully. The solution you use needs to work not just for your average load but for your heaviest load as well.
• Reliability. The third-party systems you integrate with won’t be available 100% of the time. Systems go down for maintenance; they have bugs; they fail due to too much load. Again, you are most likely to see issues during the more critical times leading up to an election. Your integrations platform should be able to retry individual API calls as well as entire pipelines to keep your data flowing even in flaky conditions. It’s good to make pipelines idempotent as much as possible to allow retries to occur without the risk of duplicating data. 
• Security. Political data is an enticing target for hackers. And data pipelines handle a lot of sensitive data — both the data flowing through the pipeline and the credentials used to access that data. Your integrations platform should keep that data encrypted as much as possible and access the smallest amount of data possible.

No matter what you use to build integrations, what matters is that you’re empowering Democratic organizations to make the most of their data and enable the best supporter experience possible. And if you want to discuss further, we love talking all things data so contact us any time at hello@bluelink.org!