Skip to content

Add Hugo Algolia indexing - #968

Open
MuhammadAashirAslam wants to merge 6 commits into
precice:hugofrom
MuhammadAashirAslam:hugo-algolia
Open

Add Hugo Algolia indexing#968
MuhammadAashirAslam wants to merge 6 commits into
precice:hugofrom
MuhammadAashirAslam:hugo-algolia

Conversation

@MuhammadAashirAslam

Copy link
Copy Markdown
Collaborator

This PR replaces the previous Jekyll Algolia plugin with a Hugo JSON export and a standalone Node.js
indexing tool.

layouts/index.algolia.json

This is a Hugo output template, not the Algolia index itself. During the Hugo build, it collects searchable
pages and writes them to public/algolia.json.

For each page, it stores:

  • The page title and URL
  • Rendered HTML and plain text
  • Headings and section information
  • Tags, categories, keywords, and modification date
  • The configured nodes to index, such as paragraphs, code blocks, and tables

Pages marked with search: exclude are skipped. The template also uses the existing content compatibility
layer so imported documentation is indexed after its legacy markup has been converted to Hugo-compatible
output.

tools/algolia-index.mjs

This is the standalone command-line indexer. It reads the JSON generated by Hugo and converts it into
records that Algolia can search.

The script:

  • Parses the generated HTML with Cheerio
  • Extracts headings, anchors, paragraphs, code blocks, and tables
  • Removes scripts, styles, iframes, and alert content from searchable text
  • Keeps tutorial introductions ahead of informational callouts
  • Generates deterministic object IDs so repeated runs update the same records
  • Adds heading hierarchy, snippets, highlighting fields, and ranking metadata
  • Splits oversized records when they exceed the configured limit
  • Validates the export before any upload takes place
  • Applies the Algolia searchable attributes, facets, snippets, and ranking settings
  • Replaces the target index atomically using Algolia’s replaceAllObjects operation

The command supports a dry-run mode:

npm run algolia:index -- --dry-run

A dry run performs all parsing and validation locally without contacting Algolia.

For an actual upload, the script reads the write credential from the environment. The application ID and
index name are supplied by the workflow, while the write key remains a repository secret.

test/algolia-index.test.mjs

These tests use Node.js’s built-in test runner, so no additional test framework is required.

The tests cover the important indexing behavior:

  • Valid Hugo exports produce the expected Algolia records
  • Heading and anchor metadata are preserved
  • Object IDs are deterministic
  • Alert boxes are excluded so tutorial results begin with their introduction
  • Oversized records are split without exceeding the configured limit
  • Invalid JSON schemas and invalid command-line options are rejected

They run with:

npm run test:algolia

package.json

This file defines the Node.js tooling used by the Algolia implementation.

It provides:

  • npm run algolia:index for transforming and uploading records
  • npm run test:algolia for running the test suite
  • The algoliasearch dependency for communicating with Algolia
  • The cheerio dependency for parsing rendered HTML
  • The minimum supported Node.js version

The workflow installs these dependencies with npm ci, which uses the lockfile rather than resolving
versions again.

package-lock.json

package-lock.json is generated automatically by npm from package.json. It records the exact versions,
download locations, and integrity checksums of the direct and transitive Node.js dependencies.

It is committed so that local development and GitHub Actions install the same dependency tree every time.
It should not be edited manually; it should be regenerated with npm when package.json changes.

Workflow

The workflow performs the indexing in several controlled stages:

  1. Checks out the repository with full history.
  2. Installs the configured Go, Hugo, and Node.js versions.
  3. Runs npm ci using package-lock.json.
  4. Runs the Algolia unit tests.
  5. Verifies Hugo Modules.
  6. Builds the Hugo site and generates public/algolia.json.
  7. Runs the indexer in dry-run mode to validate the generated records.
  8. Uploads the validated records to Algolia.

The application ID and index name are configured to match the Hugo configuration. The workflow reads only
ALGOLIA_API_KEY from GitHub repository secrets for the write credential.

Comment thread .github/workflows/update-algolia.yml Outdated
@MakisH MakisH added the technical Technical issues on the website label Aug 11, 2026
MuhammadAashirAslam added a commit to MuhammadAashirAslam/precice.github.io that referenced this pull request Aug 13, 2026
Comment thread tools/package-lock.json
Comment thread tools/package.json

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates Algolia indexing from the prior Jekyll plugin approach to a Hugo-generated JSON export (public/algolia.json) plus a standalone Node.js indexer that validates/transforms the export and (optionally) uploads records to Algolia via GitHub Actions.

Changes:

  • Adds a Node.js CLI (tools/algolia-index.mjs) to validate Hugo’s export, create/split Algolia records, and upload them with deterministic object IDs.
  • Introduces a Hugo output template (layouts/index.algolia.json) to emit the Algolia export during the Hugo build.
  • Adds Node tooling/tests plus a revamped scheduled workflow to build, validate (dry-run), and publish the Algolia index.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tools/algolia-index.mjs New Node.js indexer: parses Hugo export HTML, builds/splits records, and uploads with Algolia settings.
test/algolia-index.test.mjs Adds Node test-runner coverage for record creation, deterministic IDs, callout exclusion, and splitting behavior.
package.json Defines Node module tooling (scripts/deps) and declares the supported Node engine range.
package-lock.json Locks Algolia/Cheerio (and transitive) dependency versions for reproducible CI installs.
layouts/index.algolia.json Hugo output template that collects searchable pages and emits the JSON export.
config/_default/hugo.toml Updates Algolia index name to the new Hugo-based index.
.github/workflows/update-algolia.yml Reworks the scheduled/manual workflow to build Hugo export, run tests, dry-run validate, and upload records.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/algolia-index.mjs
Comment thread tools/algolia-index.mjs
Comment thread tools/algolia-index.mjs
Comment thread package.json
MuhammadAashirAslam and others added 3 commits August 14, 2026 18:13
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

technical Technical issues on the website

Projects

Development

Successfully merging this pull request may close these issues.

3 participants