system/nxpkg: network sync, install hardening, and CLI completion

[aviralgarg05]

Note: Please adhere to Contributing Guidelines.

Summary

This is another slice of the ongoing GSoC 2026 Dynamic ELF loading and
nxpkg work for NuttX, following on from the initial package lifecycle
helper merged in #3474.

An earlier draft PR (#3474) carries the initial nxpkg package
lifecycle helper — local metadata parsing, a local install / list
flow, compatibility checks, transaction-state handling, and integrity
verification. That initial slice was intentionally local-only: no
network sync, no update/rollback execution, no remove.

Series order for this PR:

• base series context: system/nxpkg: add a basic package lifecycle helper #3474 (merged)
• technical dependency: after system/nxpkg: add a basic package lifecycle helper #3474
• independent of: apps: add missing module metadata for executable tools #3519, examples/sotest: support packaged shared library fixtures #3526, examples/elf: extend nxpkg validation coverage #3531

This PR fills in most of what #3474 explicitly deferred, plus hardening
found while getting the package path solid enough to build a GUI
frontend on top of it (a separate follow-on PR).

Concretely, it:

• adds bounded memory-safety helpers (pkg_zalloc/pkg_malloc/pkg_free)
  and explicit size limits used throughout the rest of the package path
• hardens on-device storage read/write against untrusted or partially
  written input, rather than assuming every file it reads back was
  written by a well-behaved prior run
• rejects path-traversal sequences in manifest name/version before
  they are spliced into an on-device path — a manifest's name/version
  ultimately comes from a repository index, which should not be
  implicitly trusted to be well-formed
• supports loading a package index from an arbitrary path, not just a
  single hardcoded location
• adds real network sync and artifact acquisition (a plain HTTP GET,
  verified via SHA-256 rather than requiring TLS or any auth — the
  intent is that any static file host works, no special server needed)
• rewrites install to support network sources and to reclaim state
  left behind by an interrupted previous install/download, instead of
  getting stuck on stale partial state
• wires sync / remove / rollback / available into the CLI —
  previously stub error paths
• raises PKG_INDEX_MAX now that the in-memory index is heap-allocated
  rather than a fixed-size stack/static array
• adds an optional manifest icon field and the source-resolution logic
  for it (consumed by the nxstore GUI frontend)
• fixes a lost-update race on the shared installed-packages database:
  two independent installs of different packages could each read the
  on-disk installed-state file, mutate their own in-memory copy, and
  write back — the second writer's write would silently discard the
  first writer's already-applied change. Fixed with a global installed-
  state lock so concurrent installs serialize around that shared file
  instead of clobbering each other
• documents the repository layout and how to stand up a local package
  server (system/nxpkg/README.txt)
• fixes a real build break: pkg_runtime_compat() unconditionally
  referenced CONFIG_ARCH_BOARD, a Kconfig string symbol with no
  default clause under ARCH_BOARD_CUSTOM — on a custom board it is
  left completely undefined rather than defined-but-empty, so the
  previous code failed to compile the moment CONFIG_SYSTEM_NXPKG was
  enabled on a custom-board config. Fixed by falling back to
  CONFIG_ARCH_BOARD_CUSTOM_NAME (only #define'd when
  ARCH_BOARD_CUSTOM is selected), and routed pkg_error()/pkg_info()
  through syslog instead of stdio, since neither is visible to a
  supervisor process with no attached console

Impact

User impact:

• nxpkg now supports the full lifecycle the initial slice deferred:
  sync, install (local or network source), update, remove,
  rollback, list, available
• fixes a real compile failure on any board selected via
  ARCH_BOARD_CUSTOM (a custom board directory, as opposed to one of
  the named boards ARCH_BOARD has a default clause for) with
  CONFIG_SYSTEM_NXPKG enabled

Build impact:

• no new Kconfig dependencies beyond what system/nxpkg: add a basic package lifecycle helper #3474 already required
  (NETUTILS_CJSON); network sync additionally needs NETUTILS_WEBCLIENT

Runtime impact:

• adds network I/O and a global install-state lock to the install path;
  local-only installs are otherwise unaffected in behavior
• pkg_error()/pkg_info() now log via syslog instead of directly to
  stdio

Compatibility impact:

• none for existing applications unless CONFIG_SYSTEM_NXPKG is enabled
• the on-disk installed-state format gained fields (icon source,
  lock file); already-installed state from system/nxpkg: add a basic package lifecycle helper #3474's format is not
  migrated, since system/nxpkg: add a basic package lifecycle helper #3474 itself is not yet in a released version

Testing

Host used for build:

• OS: macOS 26.5 arm64
• compiler: xtensa-esp-elf-gcc (crosstool-NG esp-14.2.0_20241119) 14.2.0

Target used for verification:

• arch: xtensa
• board/config used during runtime verification: esp32s3-touch-lcd-7:nsh
• board: Waveshare ESP32-S3-Touch-LCD-7 (custom board directory, not
  yet upstream)

This series was developed and continuously exercised on real hardware
over the course of building the nxstore GUI frontend on top of it (a
separate follow-on PR) — every nxpkg code path this PR touches is
what that frontend calls into for every install/launch/close cycle
across dozens of manual test passes on the physical board, including:

• network sync against a local static-file repository server
• installing packages whose version strings are longer than a single
  character (this specifically exercises a since-fixed FAT 8.3
  short-filename issue in a related, already-shipped part of the
  storage path — not part of this diff, but this PR's install/sync
  rewrite is what exercises that path end-to-end)
• concurrent-install handling for the installed-state lock fix, by
  triggering installs of two different packages back to back and
  confirming both ended up correctly recorded in the installed
  database afterward, rather than one silently overwriting the other

Focused verification for the final commit in this PR (the
ARCH_BOARD_CUSTOM build fix):

$ xtensa-esp32s3-elf-gcc -c -fsyntax-only \
    -I <nuttx>/include -isystem <nuttx>/include -D__NuttX__ -D__KERNEL__ \
    -I <apps>/include -I system/nxpkg \
    system/nxpkg/pkg_compat.c system/nxpkg/pkg_log.c

compiles clean against this project's own generated
include/nuttx/config.h for esp32s3-touch-lcd-7:nsh, where
CONFIG_ARCH_BOARD is confirmed absent and only
CONFIG_ARCH_BOARD_CUSTOM_NAME is defined — reproducing the exact
build-break condition and confirming the fallback resolves it.

This is a draft PR — opening now to get the series in front of
reviewers; happy to split further if any single commit above should be
its own PR.

---

Update: the nxstore GUI frontend built on top of this is now open as #3643 (stacked on this PR — see that PR's series-order note). The games/NXDoom bring-up/crash fixes referenced above as a follow-on are open as #3644 (independent of this PR).