Skip to content

Preheat/initial contribution #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
StompieVision_Labs (masterstompie) wants to merge 7 commits into
base: main
Choose a base branch
from
+5,675 −0
Open

Preheat/initial contribution #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8834195
Add preheat: Apple content cache readiness tool
masterstompie Oct 2, 2026
81507bf
Add preheat: Apple content cache readiness tool
masterstompie Oct 2, 2026
7aabcfc
Add preheat
masterstompie Oct 2, 2026
26fcbbd
Add preheat
masterstompie Oct 2, 2026
4d8ef09
Add missing license headers to preheat scripts
masterstompie Oct 2, 2026
06bfd98
Add missing license headers to remaining preheat scripts
masterstompie Oct 2, 2026
d20ca55
Merge branch 'main' into preheat/initial-contribution
daniel-maclaughlin Oct 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions sales/preheat/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Preheat

See from Jamf Pro whether each Apple content cache is working, which devices it serves, and whether it already holds the OS update they are about to ask for. Preheat it when it does not.

> **Status: early, working, looking for testers.** Proven end to end in one lab: one cache
> server, three Macs, three iPads and an iPhone, through three real point releases. Not yet
> done: an update enforced by a Blueprint across many devices, and a second cache server in a
> second office. The full account is in
> [What is proven, and what is not](docs/FINDINGS.md#what-is-proven-and-what-is-not).
> Issues and pull requests welcome — open one at https://github.com/masterstompie/preheat/issues.

![Jamf Pro dashboard with the content cache Smart Groups](docs/images/dashboard-smart-groups.png)

*The day-to-day view: one cache server, ready for the next OS push, serving two Macs and four mobile devices. "Wiped recently" reads 1 because this cache's drive was erased the evening before: the tile to alert on, doing its job. "On but not chosen" and "Chosen but not caching" are the two that should always read 0. (Two captures of one page, joined. The "Preheat test" groups belong to the lab.)*

## What it tells you
Apple content caching keeps one copy of each update in the office, and saves the internet link
every time a second device asks for it. When it works it needs no help. But Apple gives an
administrator no way to see whether it is working, what it holds, or which devices can reach it.

Preheat answers one question per cache server, judged only against the devices that use that
server: does it already hold the OS update each of them is about to download?

| Verdict | Meaning |
|---|---|
| READY | every file those devices will ask for is in the cache, complete |
| PARTIAL | some of them are |
| NOT READY | none of them is; or the cache is down; or the server has gone silent |

The verdict is an extension attribute on the server's record, so Smart Groups, the dashboard and
email alerts work on it like on anything else in Jamf Pro. Beside it, Preheat shows:

- what each cache holds, by name: "iOS 27.0 (24A437) delta from 26.7 for iPhone17,3"
- which cache each Mac, iPhone, iPad and Apple TV would use
- whether the cache is earning its keep: the share of what it delivered that did not come from Apple
- whether a cache was wiped, is low on space, or is caching when nobody chose it to

When a cache is not ready, Preheat can fill it before the devices ask, with no test device at
that site: see [Preheating](docs/PREHEATING.md).

## How it works
- Jamf Pro's inventory says which hardware and which OS build every device has.
- Apple's own lookup service, the one every device asks, says which file that hardware and build
would download next.
- An extension attribute on each cache server lists the files its cache holds.
- An extension attribute on each Mac says which cache that Mac would use, as macOS itself decides.
- `admin/readiness-check.py` puts the four together and writes the verdict.

Nothing with a password runs on a device, and nothing is installed by hand on any Mac. More in
[Reference](docs/REFERENCE.md).

## What you need
- Jamf Pro 11.31 or later, with computers enrolled and sending inventory, the cache servers
among them. Mobile devices too, if you want iPhone, iPad, Apple TV or Vision Pro readiness.
- A Jamf Pro API role and client.
- An admin Mac, or a scheduled job, with Python 3.8 or later and no extra packages.
- One or more Macs running content caching, each with a fixed address and the Xcode Command Line
Tools.
- macOS 27 on the cache servers, if you want their live status and to configure them by Blueprint.

The full list, with the API privileges each feature needs, is in
[Requirements](docs/SETUP.md#requirements).

## Setup, in short
| Do this | With |
|---|---|
| Create the API role and client, and store the credentials in your keychain | `python3 admin/readiness-check.py --store-credentials` |
| Create the extension attributes | `python3 admin/sync-eas.py --create`, and two text fields by hand |
| Run the check | `python3 admin/readiness-check.py`, and with `--write` to store the verdict |
| Create the Smart Groups | `python3 admin/create-smart-groups.py` |
| Have each cache server name its files | `python3 admin/create-labels-policy.py` |
| Check your work | `python3 admin/doctor.py` |
| Keep it current | run the check with `--write` on a schedule; hourly is plenty |

Each step, with its tables, is in [Jamf setup](docs/SETUP.md#jamf-setup). The same installation
as Terraform is in [`terraform/`](terraform/README.md). Every script answers `--help` and changes
nothing when asked: [docs/USAGE.md](docs/USAGE.md).

## Test without Jamf
python3 admin/readiness-check.py --inventory-json admin/sample-inventory.json
python3 admin/readiness-check.py --inventory-json admin/sample-inventory-large-site.json --platforms all # two public addresses, two peer caches

Each sample carries the answers Apple gave when it was recorded (2026-09-28), so it gives the same
result whatever Apple has released since. `--record-answers` records them again.

![Readiness check output for the sample fleet](docs/images/readiness-check-output.png)

*Report for the sample two-office fleet: Office X ready, Office A missing the Mac14,2 asset.*

## Read more
| Page | What is in it |
|---|---|
| [Setup](docs/SETUP.md) | Requirements, the extension attributes as Jamf's form asks for them, setup step by step, where things live |
| [Preheating](docs/PREHEATING.md) | Filling a cache before the devices ask: OS updates, hardware not yet enrolled, installers, Xcode, Aerials, and what cannot be preheated |
| [Blueprints](docs/BLUEPRINTS.md) | Declarative updates in two waves, a group that holds on the previous OS, update settings, and configuring the cache servers themselves |
| [Findings](docs/FINDINGS.md) | What was measured: what is proven, what one release weighs, what a full cache does, thin links and outages |
| [Caveats](docs/CAVEATS.md) | What can go wrong, each one seen in the lab; and two scripts for when clients do not see the cache |
| [Reference](docs/REFERENCE.md) | Every piece, where each fact comes from, large sites, mobile devices, live status, why a Mac downloaded at 1 pm, hardware coverage, the rest of the Jamf platform |
| [Usage](docs/USAGE.md) | Every script and every option |

## Prior art
Charles Edge's [precache](https://github.com/krypted/precache) (2016 to 2020) pre-filled the old
macOS Server Caching Service with installers, IPSWs and App Store apps, and could take its device
list from Jamf Pro. Content caching has since moved into macOS itself and updates have become
per-model deltas. Preheat starts from the same idea, that a cache can be filled before devices ask,
and adds the parts that did not exist then: knowing what a cache holds, and whether it is ready for
the devices that use it.

## Bill of Materials

| Component | Kind | Version | License |
|---|---|---|---|
| [jamf/jamfplatform](https://registry.terraform.io/providers/jamf/jamfplatform) | Terraform provider | ≥ 0.29 | Jamf — verify before redistribution |

Python dependencies: none (stdlib only). Shell dependencies: `/usr/bin/curl` (system, macOS built-in).

## Privacy

Preheat does not collect telemetry, send analytics, or phone home. It reads your Jamf Pro inventory
and queries Apple's public software update API (`gdmf.apple.com`), three other Apple catalogs for
file names (`swscan.apple.com`, `configuration.apple.com`, `devimages-cdn.apple.com`) and AppleDB
(`api.appledb.dev`). No device data leaves your network except in those outbound lookup calls, which carry no personally
identifiable information. See [Jamf's Privacy Policy](https://www.jamf.com/legal/privacy-policy/).

## License

Copyright © 2026 Jamf Software LLC. All rights reserved.

Distributed under the Jamf Concepts Use Agreement. See LICENSE for the full terms.
154 changes: 154 additions & 0 deletions sales/preheat/admin/cache-collect.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
#!/bin/bash
# Copyright 2026, Jamf Software LLC.
# This work is licensed under the terms of the Jamf Source Available License
# https://github.com/jamf/scripts/blob/main/LICENCE.md
# cache-collect.sh -- READ-ONLY inspection of an Apple content caching server.
#
# Purpose: gather everything needed to design a Jamf Pro Extension Attribute that
# reports which macOS updates are cached. Makes NO changes: the cache database is
# copied to a temp folder before it is queried, and the service is never touched.
#
# Run on the caching Mac: sudo bash admin/cache-collect.sh
# Output: /Users/Shared/cache-report-<date>.txt (plus a .json of macOS-update rows)

set -u
case "${1:-}" in -h|--help) /usr/bin/sed -n '2,/^set -u/p' "$0" | /usr/bin/sed -e '/^set -u/d' -e 's/^# \{0,1\}//'; exit 0;; esac
if [[ $EUID -ne 0 ]]; then echo "Run with sudo: sudo bash $0"; exit 1; fi

STAMP=$(date +%Y%m%d-%H%M%S)
REPORT="/Users/Shared/cache-report-$STAMP.txt"
JSON="/Users/Shared/cache-macos-assets-$STAMP.json"
WORK=$(mktemp -d /tmp/cache-collect.XXXXXX)
trap 'rm -rf "$WORK"' EXIT

exec > >(tee "$REPORT") 2>&1
section(){ printf '\n\n===== %s =====\n' "$1"; }

section "SYSTEM"
sw_vers
echo "model: $(sysctl -n hw.model)"
echo "board id: $(sysctl -n hw.target)"
echo "uptime: $(uptime)"
echo "hostname: $(scutil --get LocalHostName 2>/dev/null)"

section "CONTENT CACHE STATUS (AssetCacheManagerUtil status)"
AssetCacheManagerUtil status 2>&1

section "CONTENT CACHE STATUS JSON"
AssetCacheManagerUtil -j status 2>&1

section "CONTENT CACHE SETTINGS JSON"
SETTINGS_JSON=$(AssetCacheManagerUtil -j settings 2>&1)
echo "$SETTINGS_JSON"

section "LAUNCHD SERVICE STATE"
launchctl print system/com.apple.AssetCache 2>&1 | head -40
echo "--- processes ---"
pgrep -lf AssetCache 2>&1

# ---- locate the data folder (default or custom DataPath) ----
DATA_PATH=$(echo "$SETTINGS_JSON" | python3 -c '
import json,sys
try:
d=json.load(sys.stdin); r=d.get("result",d)
print(r.get("DataPath") or "")
except Exception: print("")' 2>/dev/null)
[[ -z "$DATA_PATH" ]] && DATA_PATH="/Library/Application Support/Apple/AssetCache/Data"
DB="$DATA_PATH/AssetInfo.db"

section "DATA FOLDER: $DATA_PATH"
ls -la "$DATA_PATH" 2>&1 | head -40
echo "--- disk usage of data folder ---"
du -sh "$DATA_PATH" 2>&1
echo "--- largest 25 files under data folder ---"
find "$DATA_PATH" -type f -size +100M -exec ls -l {} \; 2>/dev/null | sort -k5 -n -r | head -25

if [[ ! -f "$DB" ]]; then
echo "AssetInfo.db not found at $DB -- stopping database section."
else
# ---- copy DB (and WAL/SHM if present) so we never touch the live file ----
cp "$DB" "$WORK/AssetInfo.db"
[[ -f "$DB-wal" ]] && cp "$DB-wal" "$WORK/AssetInfo.db-wal"
[[ -f "$DB-shm" ]] && cp "$DB-shm" "$WORK/AssetInfo.db-shm"
Q(){ sqlite3 -readonly "$WORK/AssetInfo.db" "$@"; }

section "DATABASE SCHEMA"
Q ".schema"

section "ZASSET COLUMNS"
Q -header -column "PRAGMA table_info(ZASSET);"

section "ROW COUNTS PER TABLE"
for t in $(Q "select name from sqlite_master where type='table';"); do
printf '%-20s %s\n' "$t" "$(Q "select count(*) from $t;")"
done

section "ASSET SUMMARY BY URI PREFIX (first 2 path components)"
Q -header -column "
select substr(ZURI,1,instr(substr(ZURI,2),'/')+1) as prefix,
count(*) as n,
round(sum(ZTOTALBYTES)/1073741824.0,2) as GB
from ZASSET group by prefix order by GB desc limit 40;"

section "ASSET SUMMARY BY FILE EXTENSION"
Q -header -column "
select case when instr(ZURI,'.')=0 then '(none)' else lower(replace(ZURI, rtrim(ZURI, replace(ZURI,'.','')), '')) end as ext,
count(*) as n, round(sum(ZTOTALBYTES)/1073741824.0,2) as GB
from ZASSET group by ext order by GB desc limit 30;"

section "SAMPLE OF 15 URIS (to see exact format)"
Q "select ZURI from ZASSET order by ZTOTALBYTES desc limit 15;"

section "macOS UPDATE CANDIDATES (all columns, line mode)"
MATCH="ZURI like '%MacSoftwareUpdate%' or ZURI like '%InstallAssistant%' or ZURI like '%.ipsw%' or ZURI like '%macOSUpd%' or ZURI like '%SoftwareUpdate%' or ZURI like '%.aea%' or ZURI like '%FCS/%'"
Q -header -line "select *,
datetime(ZCREATIONDATE + 978307200,'unixepoch') as created_utc,
datetime(ZLASTACCESSED + 978307200,'unixepoch') as last_accessed_utc
from ZASSET where $MATCH order by ZTOTALBYTES desc;"

section "macOS UPDATE CANDIDATES (compact)"
Q -header -column "select round(ZTOTALBYTES/1073741824.0,2) as GB,
datetime(ZCREATIONDATE + 978307200,'unixepoch') as created_utc,
ZURI
from ZASSET where $MATCH order by ZTOTALBYTES desc;"

section "ON-DISK SIZE CHECK FOR CANDIDATES (is the file fully present?)"
# Try to relate DB rows to files on disk by GUID / checksum-looking columns.
Q -json "select * from ZASSET where $MATCH;" > "$JSON"
[[ -s "$JSON" ]] || echo "[]" > "$JSON"
echo "JSON written to $JSON ($(python3 -c "import json;print(len(json.load(open('$JSON'))))") rows)"
python3 - "$JSON" "$DATA_PATH" <<'PY'
import json,sys,os
rows=json.load(open(sys.argv[1])); data=sys.argv[2]
# Layout learned 2026-09-17: each asset's bytes live in <DataPath>/<ZGUID>/<chunk files>
for r in rows:
uri=r.get("ZURI",""); total=r.get("ZTOTALBYTES") or 0; guid=r.get("ZGUID") or ""
d=os.path.join(data,guid); ondisk=0; files=0
if guid and os.path.isdir(d):
for root,_,fs in os.walk(d):
for f in fs:
try: ondisk+=os.path.getsize(os.path.join(root,f)); files+=1
except OSError: pass
pct=(100.0*ondisk/total) if total else 0
state="COMPLETE" if total and ondisk>=total else ("PARTIAL" if ondisk else "NO DATA ON DISK")
print(f"\n{uri}\n guid={guid} files={files} ondisk={ondisk} total={total} ({pct:.1f}%) -> {state}")
print(f" ZMD5OFFSET={r.get('ZMD5OFFSET')} ZCHECKSUM={r.get('ZCHECKSUM')!r}")
PY

section "OTHER TABLES (first 20 rows each, may explain completeness/state)"
for t in $(Q "select name from sqlite_master where type='table' and name not in ('ZASSET');"); do
echo "--- $t ---"; Q -header -column "select * from $t limit 20;"
done
fi

section "RECENT CONTENT CACHE LOG (last 30 min, errors/faults + status summaries)"
log show --last 30m --predicate 'subsystem == "com.apple.AssetCache"' --style compact 2>/dev/null \
| grep -Ei 'error|fault|fail|shut|deactivat|activat|start|stopp|status|registration' | tail -80

section "RECENT MACOS UPDATE REQUESTS SEEN BY THE CACHE (last 24h, if any)"
log show --last 24h --predicate 'subsystem == "com.apple.AssetCache"' --style compact 2>/dev/null \
| grep -Ei 'MacSoftwareUpdate|InstallAssistant|ipsw|\.zip|\.aea|FCS' | tail -40

section "DONE"
echo "Report: $REPORT"
echo "JSON: $JSON"
52 changes: 52 additions & 0 deletions sales/preheat/admin/create-labels-policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/usr/bin/env python3
# Copyright 2026, Jamf Software LLC.
# This work is licensed under the terms of the Jamf Source Available License
# https://github.com/jamf/scripts/blob/main/LICENCE.md
"""
Put admin/preheat-labels.py into Jamf Pro as a script, and create the policy that runs it on cache servers:
General: trigger Recurring Check-in, frequency Ongoing Scripts: preheat-labels.py
Maintenance: Update Inventory Scope: Smart Group "Content Caching - Servers"
So at every check-in a cache server names any new files it holds, then sends inventory, and the decoded EA
(ea-content-cache-decoded.sh) is current without anyone running the admin script. Disable the policy to stop.
Run again after editing preheat-labels.py: the script in Jamf is updated, the policy is left alone.
Needs API privileges: Create/Read/Update Scripts, Create/Read Policies, Read Smart Computer Groups.
Usage: python3 admin/create-labels-policy.py [--group "Content Caching - Servers"] [--disabled]
"""
import os, sys, argparse, urllib.error, urllib.request
from xml.sax.saxutils import escape
sys.path.insert(0, os.path.dirname(__file__)); rc = __import__("readiness-check")
SCRIPT_NAME = "preheat-labels.py"; POLICY_NAME = "Preheat - Refresh cache labels"

def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--group", default="Content Caching - Servers"); ap.add_argument("--disabled", action="store_true", help="create the policy switched off")
args = ap.parse_args()
j = rc.jamf_from_credentials()
local = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), SCRIPT_NAME)).read()
scripts = j.call("/api/v1/scripts?page=0&page-size=500").get("results", [])
hit = next((s for s in scripts if s["name"] == SCRIPT_NAME), None)
body = {"name": SCRIPT_NAME, "info": "Names the OS update files a content cache holds. No Jamf credentials. See the preheat README.",
"notes": "Managed by admin/create-labels-policy.py", "priority": "AFTER", "scriptContents": local}
if hit:
sid = hit["id"]
if (hit.get("scriptContents") or "").replace("\r\n", "\n").strip() == local.strip(): print(f"script {SCRIPT_NAME}: up to date (id {sid})")
else: j.call(f"/api/v1/scripts/{sid}", "PUT", {**hit, **body}); print(f"script {SCRIPT_NAME}: updated (id {sid})")
else:
sid = j.call("/api/v1/scripts", "POST", body)["id"]; print(f"script {SCRIPT_NAME}: created (id {sid})")
if any(p["name"] == POLICY_NAME for p in j.call("/JSSResource/policies")["policies"]):
print(f"policy '{POLICY_NAME}': already exists, left alone"); return
groups = j.call("/api/v2/computer-groups/smart-groups?page=0&page-size=500").get("results", [])
gid = next((g["id"] for g in groups if g["name"] == args.group), None)
if not gid: sys.exit(f"Smart Group '{args.group}' not found; run admin/create-smart-groups.py first")
xml = f"""<policy><general><name>{escape(POLICY_NAME)}</name><enabled>{str(not args.disabled).lower()}</enabled>
<trigger>CHECKIN</trigger><trigger_checkin>true</trigger_checkin><frequency>Ongoing</frequency></general>
<scope><computer_groups><computer_group><id>{gid}</id></computer_group></computer_groups></scope>
<scripts><size>1</size><script><id>{sid}</id><priority>After</priority></script></scripts>
<maintenance><recon>true</recon></maintenance></policy>"""
req = urllib.request.Request(j.base + "/JSSResource/policies/id/0", data=xml.encode(), method="POST",
headers={"Authorization": "Bearer " + j.token, "Content-Type": "application/xml"})
try:
with urllib.request.urlopen(req, timeout=60) as r: print(f"policy '{POLICY_NAME}': created, scoped to '{args.group}'", r.read().decode()[-40:])
except urllib.error.HTTPError as e: sys.exit(f"policy create refused HTTP {e.code}: {e.read().decode()[:300]}")

if __name__ == "__main__": main()
Loading
Loading