Repository navigation
[Q2-P2] Add synthetic monitoring with continuous SDK health checks #28
Copy link
Copy link
Closed
Labels
priority: mediumShould be fixed eventuallyShould be fixed eventuallyquadrant: q2Important, Not Urgent (Schedule)Important, Not Urgent (Schedule)type: monitoringMonitoring, observability, and alertingMonitoring, observability, and alerting
Description
Problem
We only test SDK when:
- Someone runs tests manually
- CI runs on commits
- Customers use it in production
Gap: No continuous validation that SDK works in production
Solution
Add synthetic monitoring that runs realistic SDK queries continuously:
# monitoring/synthetic/sdk_health_check.py
import schedule
import time
from oilpriceapi import OilPriceAPI
def health_check():
"""Run realistic SDK queries."""
client = OilPriceAPI(api_key=MONITOR_KEY)
checks = [
('current_price', lambda: client.prices.get('WTI_USD')),
('1_week_historical', lambda: client.historical.get(
'WTI_USD',
(datetime.now() - timedelta(days=7)).date(),
datetime.now().date()
)),
('1_month_historical', lambda: client.historical.get(
'WTI_USD',
(datetime.now() - timedelta(days=30)).date(),
datetime.now().date()
)),
]
for name, check in checks:
try:
start = time.time()
result = check()
duration = time.time() - start
send_metric(f'synthetic.{name}.success', 1)
send_metric(f'synthetic.{name}.duration', duration)
except Exception as e:
send_metric(f'synthetic.{name}.failure', 1)
send_alert(f'Synthetic check failed: {name}: {e}')
# Run every 5 minutes
schedule.every(5).minutes.do(health_check)Deploy
# Deploy as cron job or lambda
apiVersion: batch/v1
kind: CronJob
metadata:
name: sdk-synthetic-monitoring
spec:
schedule: "*/5 * * * *" # Every 5 minutes
jobTemplate:
spec:
template:
spec:
containers:
- name: health-check
image: oilpriceapi/sdk-monitor:latest
env:
- name: MONITOR_API_KEY
valueFrom:
secretKeyRef:
name: monitoring-secrets
key: api-keyAlerts
ALERT: Synthetic check failure rate >10% over 15min
ALERT: Synthetic check duration >2x baseline
ALERT: Synthetic check not running (missing data)
Acceptance Criteria
- Synthetic monitoring script created
- Deployed and running every 5 minutes
- Alerts configured
- Dashboard shows synthetic check results
- Runbook for responding to failures
Estimated Effort
Time: 4 hours
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
priority: mediumShould be fixed eventuallyShould be fixed eventuallyquadrant: q2Important, Not Urgent (Schedule)Important, Not Urgent (Schedule)type: monitoringMonitoring, observability, and alertingMonitoring, observability, and alerting
Activity
coderabbitai commentedon Dec 17, 2025
📝 CodeRabbit Plan Mode
Generate an implementation plan and prompts that you can use with your favorite coding agent.
Examples
🔗 Similar Issues
Related Issues
👤 Suggested Assignees
🧪 Issue enrichment is currently in open beta.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord or schedule a call!
feat: Complete Q1 & Q2 QA improvements (#20-28)
feat: Complete Q1 & Q2 QA improvements (#20-28)
karlwaldman commentedon Jul 25, 2026
Completion receipt for PR #74 /
c2f6a882b1aa483815b9191e8c83a1a25a7c224f:sdk-health-30169188492exists and expires 2026-08-24The schedule is hourly rather than every five minutes: this keeps detection bounded while avoiding roughly 8,640 CI starts/month and unnecessary API quota. Push/PR live checks cover changes between scheduled runs.