ASN Karma

ASN Karma is a Go pipeline for building ASN-level risk datasets from observed BlackRoute evidence. It aggregates hostile IP/CIDR records by autonomous system, scores abuse exposure with an auditable rule set, and emits release artifacts for security analytics, fraud/risk enrichment, traffic policy, and network operations.

---

Latest Release

Fresh dataset artifacts are published by the scheduled build. The links below point at the latest GitHub Release assets.

Last dataset build: 2026-06-17T09:23:39Z

Open latest GitHub release

Artifact	Download	Description
index.json	download	Machine-readable release manifest
asn-risk.jsonl	download	Primary JSONL risk dataset
asn-changes.jsonl	download	ASN delta feed since previous build
asn-summary.csv	download	CSV summary for review and reporting
asn-evidence-table.md	download	Markdown table of top ASN evidence counts
asn-profiles.tar.gz	download	Per-ASN JSON profiles
source-impact.csv	download	Source contribution breakdown
country-risk.csv	download	Country-level operational rollup
high-risk-asn-critical.txt	download	Critical ASN tier
high-risk-asn-high.txt	download	High ASN tier
high-risk-asn-watch.txt	download	Watch ASN tier
high-risk-asn-prefixes-critical.txt	download	Derived critical ASN announced prefixes
high-risk-asn-prefixes-high.txt	download	Derived high ASN announced prefixes
high-risk-asn-prefixes-watch.txt	download	Derived watch ASN announced prefixes
report.md	download	Markdown dataset report
release-notes.md	download	Release summary and top ASN table
run_stats.json	download	Build metadata and tier counts
checksums.txt	download	SHA256 checksums for release artifacts

Overview

ASN Karma consumes BlackRoute JSONL records and produces an ASN risk layer designed for operational use. The output is intentionally explainable: each ASN record includes score, tier, observed record counts, source diversity, top threat labels, and build metadata.

The project treats ASN expansion as derived intelligence. Source evidence comes from observed IP/CIDR records only; generated ASN prefix lists are output artifacts, not feedback into the evidence stream.

System Behavior

BlackRoute JSONL
  -> parse observed IP/CIDR evidence
  -> enrich records without ASN via Team Cymru bulk whois
  -> aggregate records by ASN
  -> compute source diversity and threat label distribution
  -> apply scoring policy from configs/scoring.json
  -> write JSONL, CSV, TXT tiers, and run statistics

Stage	Responsibility	Current implementation
Ingest	Read BlackRoute-style JSONL with tolerant field mapping	internal/blackroute
Enrich	Map observed IP/CIDR records to ASN, country, and routed prefix	internal/enrich
Model	Normalize observed records and aggregate by ASN	internal/model
Scoring	Apply deterministic score and tier policy	internal/scoring
Output	Emit release artifacts for machines and operators	internal/output
Automation	Build and publish artifacts from GitHub Actions	.github/workflows/build.yml

Features

• Go CLI with no runtime service dependency.
• Team Cymru bulk whois enrichment for upstream records without ASN metadata.
• Deterministic ASN scoring from local configuration.
• JSONL primary output for downstream data pipelines.
• CSV summary for analyst workflows.
• Text tier files for infrastructure policy integration.
• 7/30/90 day history signals for persistence and trend.
• Confidence scoring alongside risk scoring.
• Per-ASN profile archive and derived announced-prefix artifacts.
• SHA256 checksums for release artifacts.
• GitHub Actions workflow for scheduled dataset builds.
• Explicit expanded_prefixes_are_evidence: false field in risk records.
• Local smoke-test fixture under data/blackroute.example.jsonl.

Quick Start

go test ./...
go run ./cmd/asn-karma \
  -input data/blackroute.example.jsonl \
  -out release \
  -readme README.md

The command writes release artifacts into release/.

release/
  index.json
  asn-risk.jsonl
  asn-changes.jsonl
  asn-summary.csv
  asn-evidence-table.md
  asn-profiles.tar.gz
  source-impact.csv
  country-risk.csv
  high-risk-asn-critical.txt
  high-risk-asn-high.txt
  high-risk-asn-watch.txt
  high-risk-asn-prefixes-critical.txt
  high-risk-asn-prefixes-high.txt
  high-risk-asn-prefixes-watch.txt
  report.md
  release-notes.md
  run_stats.json
  checksums.txt

Installation

From Source

git clone https://github.com/ipanalytics/ASN-Karma.git
cd ASN-Karma
go build -o bin/asn-karma ./cmd/asn-karma

Requirements

Component	Version
Go	1.22 or newer
Input dataset	BlackRoute JSONL
Runtime	Linux, macOS, or containerized CI

Usage

Run against a local BlackRoute export:

asn-karma \
  -input data/blackroute.jsonl \
  -config configs/scoring.json \
  -out release

ASN enrichment is enabled by default. For offline parser tests against data that already contains ASN fields:

asn-karma \
  -input data/blackroute.example.jsonl \
  -out release \
  -asn-enrich=false

Use a fixed build timestamp for reproducible test output:

asn-karma \
  -input data/blackroute.example.jsonl \
  -out /tmp/asn-karma-release \
  -built-at 2026-06-15T00:00:00Z

Run directly with Go:

go run ./cmd/asn-karma -input data/blackroute.jsonl -out release

Outputs

Artifact	Format	Purpose
index.json	JSON	Machine-readable release manifest with sizes and SHA256 hashes
asn-risk.jsonl	JSONL	Primary machine-readable ASN risk dataset
asn-changes.jsonl	JSONL	Delta feed since previous build
asn-summary.csv	CSV	Compact review and reporting table
asn-evidence-table.md	Markdown	Top ASN evidence table used by README and release notes
asn-profiles.tar.gz	tar.gz	Per-ASN JSON profiles with risk, history, confidence, and derived prefixes
source-impact.csv	CSV	Source contribution and ASN impact summary
country-risk.csv	CSV	Country-level operational rollup
high-risk-asn-critical.txt	TXT	Strict action tier
high-risk-asn-high.txt	TXT	Challenge or rate-limit tier
high-risk-asn-watch.txt	TXT	Enrichment and logging tier
high-risk-asn-prefixes-critical.txt	TXT	Derived announced prefixes for critical ASN tier
high-risk-asn-prefixes-high.txt	TXT	Derived announced prefixes for high ASN tier
high-risk-asn-prefixes-watch.txt	TXT	Derived announced prefixes for watch ASN tier
report.md	Markdown	Rendered release report with deltas, countries, and source impact
release-notes.md	Markdown	GitHub Release body with run summary and top ASN table
run_stats.json	JSON	Build metadata and tier counts
checksums.txt	TXT	SHA256 checksums for release artifacts

Changes Since Previous Build

The scheduled build updates this table from asn-changes.jsonl. It shows the largest ASN-level deltas compared with the previous persisted history snapshot.

Last updated: 2026-06-17T09:23:39Z

ASN	Name	Country	Change	Previous	Current	Evidence Delta
AS132203	TENCENT-NET-AP-CN - Tencent Building, Kejizhongyi Avenue, CN	SG	evidence_increased	20518	21586	+1068
AS43515	YOUTUBE - Google Ireland Limited, IE	US	evidence_decreased	1493	578	-915
AS16509	AMAZON-02 - Amazon.com, Inc., US	US	evidence_increased	360959	361804	+845
AS17561	LCS-AS-AP - LARUS Limited, HK	SC	evidence_increased	11353	12126	+773
AS396982	GOOGLE-CLOUD-PLATFORM - Google LLC, US	US	evidence_decreased	51545	50852	-693
AS31898	ORACLE-BMC-31898 - Oracle Corporation, US	US	evidence_increased	29913	30600	+687
AS16276	OVH - OVH SAS, FR	FR	evidence_increased	37021	37604	+583
AS14061	DIGITALOCEAN-ASN - DigitalOcean, LLC, US	US	evidence_decreased	169742	169190	-552
AS19527	GOOGLE-2 - Google LLC, US	US	evidence_increased	315	817	+502
AS4134	CHINANET-BACKBONE - No.31,Jin-rong Street, CN	CN	evidence_decreased	78169	77770	-399
AS15169	GOOGLE - Google LLC, US	US	evidence_increased	1036	1405	+369
AS17497	LGHL-AS-AP - Liasail Global Hongkong Limited, HK	SC	evidence_increased	11367	11713	+346
AS210874	box-broadband - Box Broadband Limited, GB	US	risk_level_changed	336	2	-334
AS24940	HETZNER-AS - Hetzner Online GmbH, DE	DE	evidence_increased	23170	23497	+327
AS14618	AMAZON-AES - Amazon.com, Inc., US	US	evidence_increased	85728	86029	+301
AS8560	IONOS-AS - IONOS SE, DE	DE	evidence_increased	4638	4923	+285
AS197540	netcup-AS - netcup GmbH, DE	DE	evidence_increased	4656	4928	+272
AS20326	TERASWITCH - TeraSwitch Networks Inc., US	GB	evidence_decreased	2600	2332	-268
AS27385	QUALYS - QUALYS, Inc., US	US	evidence_increased	1785	2040	+255
AS36352	AS-COLOCROSSING - HostPapa, US	US	evidence_increased	29491	29736	+245
AS44559	ITHOSTLINE - IT HOSTLINE LTD, CY	SC	evidence_increased	2362	2601	+239
AS8075	MICROSOFT-CORP-MSN-AS-BLOCK - Microsoft Corporation, US	US	evidence_increased	47084	47313	+229
AS202656	XServerCloud - Ivanov Vitaliy Sergeevich, UA	SC	evidence_decreased	4273	4057	-216
AS6079	RCN-AS - RCN, US	US	evidence_decreased	9386	9174	-212
AS63949	AKAMAI-LINODE-AP - Akamai Connected Cloud, SG	US	evidence_increased	12488	12673	+185

Risk Record

When ASN records are available, asn-risk.jsonl contains one JSON object per ASN:

{
  "asn": 64500,
  "asn_name": "Example Hosting",
  "country": "US",
  "risk_score": 39,
  "risk_level": "low",
  "confidence_score": 40,
  "confidence": "low",
  "recommended_action": "no_action",
  "observed_records": 2,
  "unique_observed_cidrs": 2,
  "source_count": 2,
  "source_diversity": 2,
  "top_threat_labels": {
    "c2_ioc": 1,
    "malware_host_active": 1,
    "network_scan_or_abuse": 1
  },
  "evidence_window_days": 30,
  "persistence_days_30d": 1,
  "active_days_7d": 1,
  "active_days_30d": 1,
  "active_days_90d": 1,
  "first_seen": "2026-06-15",
  "last_seen": "2026-06-15",
  "trend": "new",
  "evidence_delta_1d": 2,
  "expanded_prefix_count": 0,
  "expanded_prefixes_are_evidence": false,
  "large_cloud": false,
  "watchlist": false,
  "built_at": "2026-06-15T00:00:00Z"
}

If a build is explicitly allowed to complete with zero ASN records, asn-risk.jsonl contains a single build_status JSON object explaining that no ASN records were produced. Scheduled production builds do not use -allow-empty; an empty ASN dataset fails before release publication.

Data Contracts

Schemas are kept under docs/schema/:

Schema	Covers
docs/schema/asn-risk.schema.json	asn-risk.jsonl records
docs/schema/asn-changes.schema.json	asn-changes.jsonl records
docs/schema/index.schema.json	index.json release manifest
docs/schema/run-stats.schema.json	run_stats.json

Integration Examples

Operational examples are available under examples/:

File	Target
examples/cloudflare-waf.md	Cloudflare WAF ASN policy
examples/nginx-map.md	NGINX enrichment map pattern
examples/opnsense-alias.md	OPNsense firewall aliases
examples/splunk-lookup.md	Splunk CSV lookup
examples/clickhouse-ingest.sql	ClickHouse JSONL ingestion

Scoring Policy

Scoring is configured in configs/scoring.json.

Signal	Role
Source diversity	Rewards corroboration across feeds
Threat severity	Weights labels such as C2, malware hosting, spam, and scanning
Recent activity	Captures observed volume in the build window
Abuse density proxy	Gives smaller concentrated abuse surfaces weight
Cybercrime prefix bonus	Adds weight for severe infrastructure labels
Large cloud penalty	Reduces broad-provider overclassification
Allowlist penalty	Suppresses known infrastructure where appropriate
Watchlist flag	Adds context without turning context into evidence

Risk tiers are emitted as critical, high, watch, or low.

Operational Notes

• Treat asn-risk.jsonl as the canonical artifact.
• Use TXT tier files as policy inputs only after local validation.
• Keep scoring changes reviewable; policy drift should be visible in config diffs.
• Do not feed derived ASN prefix expansion back into source evidence.
• Verify downloaded artifacts with checksums.txt.
• ASNs marked review_required=true are large cloud, backbone, CDN, or major hosting networks; they are capped to review/watch policy unless local telemetry supports enforcement.
• Large cloud and CDN networks need provider-aware handling in production policy.
• Run builds on a schedule after the upstream BlackRoute release has completed.

Project Scope

ASN Karma focuses on ASN-level aggregation, scoring, and artifact generation. It is designed to sit between raw IP reputation feeds and downstream enforcement, enrichment, or analytics systems.

Planned extension points include:

• Optional release signing.
• GitHub Pages dataset index.

Use Cases

• Enrich SIEM, SOAR, and data lake events with ASN risk context.
• Feed WAF, CDN, and edge policy with conservative ASN tiers.
• Track abuse concentration across hosting providers and network operators.
• Support fraud and risk pipelines with infrastructure-level features.
• Build daily ASN exposure reports for security operations.

Limitations

ASN-level scoring is coarse by design. It should be combined with local telemetry, asset context, customer impact analysis, and provider-specific knowledge before enforcement.

Team Cymru enrichment uses current BGP attribution. For historical analysis, run the scorer against input that already carries time-appropriate ASN metadata.

Directory Structure

.
├── cmd/asn-karma/              # CLI entrypoint
├── configs/                    # scoring and policy configuration
├── data/                       # local fixtures and input data
├── data/history/               # persisted daily ASN history state
├── docs/schema/                # JSON schema contracts
├── examples/                   # integration examples
├── internal/blackroute/         # BlackRoute JSONL ingest
├── internal/enrich/             # ASN enrichment adapters
├── internal/model/              # normalized records and aggregation
├── internal/output/             # release artifact writers
├── internal/scoring/            # scoring policy implementation
├── release/                     # generated artifacts
├── site/                        # README and documentation assets
└── .github/workflows/           # scheduled build automation

Deployment

The repository includes a scheduled GitHub Actions workflow:

on:
  schedule:
    - cron: "47 4 * * *"
  workflow_dispatch:

The workflow tests the Go code, downloads the latest BlackRoute JSONL release, builds ASN Karma artifacts, updates the README evidence table, and publishes the generated files as a GitHub release.

For self-hosted deployments, run the CLI from cron, systemd timers, Kubernetes CronJobs, or an existing data orchestration system. The process is batch-oriented and writes immutable output files for each run.

Example Kubernetes CronJob command

command:
  - /usr/local/bin/asn-karma
  - -input
  - /data/blackroute.jsonl
  - -config
  - /config/scoring.json
  - -out
  - /release

License

MIT license.

Disclaimer

ASN Karma provides infrastructure risk signals derived from public abuse evidence. Operators are responsible for applying local policy, validation, and impact controls before enforcement.