Legislator Profile - Elections Data Pipeline

[Mephistic]

Summary

For the Legislator Profile project, we want to show past elections data for each legislator. We don't currently have that data in our backend, so we need to create a scraper to pull that data into our system to make it available for the front-end to display.

Election Results are available from the Secretary of State's website (e.g. https://electionstats.state.ma.us/elections/search/year_from:2022/year_to:2026/office_id:9/stage:General)

Success Criteria

• New Scraper that runs daily to check the Secretary of State's website for new election results and pull them into our system if so
  • This will likely demand a new firestore collection (e.g. electionResults)
  • We (currently) only care about General Election results, not primaries
  • This data updates very infrequently (outside special elections, each seat is only up once every two/four years).
• Create a script to run this scraper against historical data (going back to ~2009/2010 - the earliest court we're likely to support) once )
  • Data before 2025 is unlikely to ever change - we can grab all historical data in one fell swoop, then only monitor for new data

Additional Links

• Secretary of State Election Results Website: https://electionstats.state.ma.us/elections/search/year_from:2022/year_to:2026/office_id:9/stage:General
• Mockup: https://maple-leg-profiles.netlify.app/