Legislator Profile - Campaign Finance Data Pipeline

[Mephistic]

Summary

As part of the Legislator Profile project, we want to make the Campaign Finance data (made publicly available via OCPF) associated with a Legislator visible to our users through the new Legislator Profile page. We don't currently scrape Campaign Finance data, so we'll need to add a new scraper that plumbs this data into our system to make it available for the Front End.

Success Criteria

• New scraper that checks OCPF regularly for new data and plumbs any new data into our system (likely a new firestore collection e.g. campaignFinance)
  • This will need to be queryable by Legislator (Ideally by the Member Code - since that's what the Legislator Profile is keyed off - so we may need to do a little work here to map legislators to members depending on the format of the OCPF data)
  • Depending on the volume/format of the data dump from OCPF, it may be worthwhile to pre-compute the key statistics here by legislator (e.g. total raised, total spent, small donors, and breakdown by industry) and save the summaries separately.
  • The Legislator Profile Frontend will (for now) only display the most recent election cycle's data
• One-off script to backfill historical data into the new collection (ideally going back to ~2009/2010 - the earliest year we're likely support since that's when the MA Legislature API came out)

Screenshots

Additional Links

• Mockup: https://maple-leg-profiles.netlify.app/
• OCPF: https://www.ocpf.us/Data/Downloads