Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ jobs:
id: version
run: node scripts/check-version.mjs src/joomla-ws-api.json CHANGELOG.md "$RUNNER_TEMP/base.json"

# Also generates the PhpStorm collection
- name: Build
run: npm run build

Expand Down
62 changes: 62 additions & 0 deletions .github/workflows/phpstorm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: PhpStorm branch

# On every push to main, the PhpStorm HTTP Client collection is generated from the spec and committed to the
# phpstorm branch, when it changed. The commit message names the spec version and the main commit, for instance
# "Generated from 6.2.0 (main 9bd9ea4)". The first run creates the branch.
# The fixed collection of each release is the release file joomla-ws-api-<version>-phpstorm.zip.

on:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: phpstorm-branch
cancel-in-progress: false

jobs:
phpstorm:
name: Update the phpstorm branch
# Not in forks: there the phpstorm branch can be the head of a pull request.
# Only from main: a manual run from another branch must not push to phpstorm.
if: github.repository == 'joomla-projects/API-Endpoints' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7

- uses: actions/setup-node@v7
with:
node-version: 22

# The generator only uses Node.js itself: no npm ci needed
- name: Generate the collection
run: node scripts/generate-phpstorm.mjs

- name: Commit to the phpstorm branch
run: |
version=$(node -p "require('./src/joomla-ws-api.json').info.version")
branch="$RUNNER_TEMP/phpstorm"
if git ls-remote --exit-code --heads origin phpstorm > /dev/null; then
git fetch --quiet --depth 1 origin phpstorm
git worktree add --quiet --detach "$branch" FETCH_HEAD
else
echo "::notice::The phpstorm branch does not exist yet: creating it"
git worktree add --quiet --orphan -b phpstorm-generated "$branch"
fi
cd "$branch"
git rm -r --quiet --ignore-unmatch .
cp -a "$GITHUB_WORKSPACE/dist/phpstorm/." .
git add -A
if git diff --cached --quiet; then
echo "The collection did not change."
exit 0
fi
git -c user.name="github-actions[bot]" -c user.email="41898233+github-actions[bot]@users.noreply.github.com" \
commit --quiet -m "Generated from $version (main ${GITHUB_SHA::7})"
git push --quiet origin HEAD:refs/heads/phpstorm
git log -1 --stat --format='%s'
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Release

# Push a tag that equals info.version of src/joomla-ws-api.json, for instance 6.2.1:
# git tag -a 6.2.1 -m "Joomla Web Services API 6.2.1" && git push origin 6.2.1
# The release gets the files joomla-ws-api-<version>.json, .yaml, .html and .sha256.
# The release gets the files joomla-ws-api-<version>.json, .yaml, .html, -phpstorm.zip and .sha256.
# The highest version becomes the "latest" release:
# https://api.github.com/repos/joomla-projects/API-Endpoints/releases/latest

Expand Down Expand Up @@ -57,7 +57,7 @@ jobs:
| grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 1)
latest=false
[ "$highest" = "$GITHUB_REF_NAME" ] && latest=true
gh release create "$GITHUB_REF_NAME" dist/* \
gh release create "$GITHUB_REF_NAME" dist/joomla-ws-api-* \
--verify-tag \
--title "Joomla Web Services API $GITHUB_REF_NAME" \
--notes-file notes.md \
Expand Down
19 changes: 18 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ The specification is one file: [`src/joomla-ws-api.json`](src/joomla-ws-api.json
| Not yet released (`main`) | https://raw.githubusercontent.com/joomla-projects/API-Endpoints/main/src/joomla-ws-api.json |

Each release has these files, with its version in the name: `joomla-ws-api-<version>.json`, the same spec in
YAML (`.yaml`), readable documentation (`.html`, Redoc) and checksums (`.sha256`).
YAML (`.yaml`), readable documentation (`.html`, Redoc), the [PhpStorm collection](#phpstorm-collection)
(`-phpstorm.zip`, since 6.2.1) and checksums (`.sha256`).

For AI agents and other tools: the JSON of the latest release has the version in `tag_name`, and the download
URL of each file in `assets[].browser_download_url`. For instance:
Expand Down Expand Up @@ -51,6 +52,22 @@ The [`bruno`](https://github.com/joomla-projects/API-Endpoints/tree/bruno) branc
[Bruno](https://www.usebruno.com) collection of the Joomla 6 API endpoints, made by René Kreijveld
([@renekreijveld](https://github.com/renekreijveld)). It is not yet up to date with this specification.

## PhpStorm collection

A collection for the HTTP Client of PhpStorm and other JetBrains IDEs: a request for every operation of the
specification, with its example request body. It is generated from the specification, so it is always up to date
with it.

- **A release**: each release since 6.2.1 has the collection of that version in
`joomla-ws-api-<version>-phpstorm.zip`.
- **The [`phpstorm`](https://github.com/joomla-projects/API-Endpoints/tree/phpstorm) branch**: the collection of the
newest specification in `main`, also with improvements that are not released yet. It is updated automatically on
every change of `main`.

Its [README](scripts/phpstorm/README.md) explains how to set it up. To generate the collection yourself, run
`npm run phpstorm`: it writes the collection in `dist/phpstorm/`, from
[`scripts/generate-phpstorm.mjs`](scripts/generate-phpstorm.mjs) and the files in [`scripts/phpstorm`](scripts/phpstorm).

## Credits

The specification was made by Sergey Tolkachyov ([@sergeytolkachyov](https://github.com/sergeytolkachyov)) for the Joomla Manual
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"lint": "redocly lint src/joomla-ws-api.json --format=stylish",
"quality": "node scripts/check-quality.mjs src/joomla-ws-api.json",
"routes": "node scripts/check-routes.mjs src/joomla-ws-api.json",
"phpstorm": "node scripts/generate-phpstorm.mjs",
"check": "npm run format:check && npm run validate:schema && npm run lint && npm run quality",
"build": "node scripts/build.mjs"
},
Expand Down
58 changes: 57 additions & 1 deletion scripts/build.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
// joomla-ws-api-<version>.json the spec, unchanged
// joomla-ws-api-<version>.yaml the same spec in YAML
// joomla-ws-api-<version>.html readable documentation (Redoc)
// joomla-ws-api-<version>-phpstorm.zip the PhpStorm HTTP Client collection (scripts/generate-phpstorm.mjs)
// joomla-ws-api-<version>.sha256 checksums of the files above
// The collection is also left unzipped in dist/phpstorm/.
//
// Usage: node scripts/build.mjs [--tag <tag>]
// --tag: fail unless the tag equals info.version of the spec (used by the release workflow).
Expand All @@ -12,6 +14,7 @@ import { execFileSync } from 'node:child_process';
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import zlib from 'node:zlib';
import { loadSpec } from './lib.mjs';

const SRC = 'src/joomla-ws-api.json';
Expand All @@ -34,13 +37,66 @@ const redocly = (...a) => {
execFileSync(process.execPath, [path.join('node_modules', '@redocly', 'cli', 'bin', 'cli.js'), ...a], { stdio: 'inherit' });
};

/**
* Writes a zip file of the files in dir, in the folder root of the zip. Fixed dates, so a build is reproducible.
* Only a flat folder: a subfolder is an error.
*/
function zip(dir, root, target) {
const entries = fs.readdirSync(dir, { withFileTypes: true });
const other = entries.filter((e) => !e.isFile()).map((e) => e.name);
if (other.length) throw new Error(`zip: ${dir} may only contain files, not: ${other.join(', ')}`);
const names = entries.map((e) => e.name).sort();
const local = [];
const central = [];
let offset = 0;
for (const name of names) {
const data = fs.readFileSync(path.join(dir, name));
const deflated = zlib.deflateRawSync(data, { level: 9 });
const fileName = Buffer.from(`${root}/${name}`);
const crc = zlib.crc32(data);
// version 2.0, flags (UTF-8 names), deflate, time 00:00, date 1980-01-01, crc, sizes, name length, extra length
const fields = (b, at) => {
b.writeUInt16LE(20, at);
b.writeUInt16LE(0x0800, at + 2);
b.writeUInt16LE(8, at + 4);
b.writeUInt16LE(0, at + 6);
b.writeUInt16LE(0x21, at + 8);
b.writeUInt32LE(crc, at + 10);
b.writeUInt32LE(deflated.length, at + 14);
b.writeUInt32LE(data.length, at + 18);
b.writeUInt16LE(fileName.length, at + 22);
};
const header = Buffer.alloc(30);
header.writeUInt32LE(0x04034b50, 0);
fields(header, 4);
const entry = Buffer.alloc(46);
entry.writeUInt32LE(0x02014b50, 0);
entry.writeUInt16LE(20, 4);
fields(entry, 6);
entry.writeUInt32LE(offset, 42);
local.push(header, fileName, deflated);
central.push(entry, fileName);
offset += header.length + fileName.length + deflated.length;
}
const directory = Buffer.concat(central);
const end = Buffer.alloc(22);
end.writeUInt32LE(0x06054b50, 0);
end.writeUInt16LE(names.length, 8);
end.writeUInt16LE(names.length, 10);
end.writeUInt32LE(directory.length, 12);
end.writeUInt32LE(offset, 16);
fs.writeFileSync(target, Buffer.concat([...local, directory, end]));
}

const base = `joomla-ws-api-${version}`;
const files = [`${base}.json`, `${base}.yaml`, `${base}.html`];
const files = [`${base}.json`, `${base}.yaml`, `${base}.html`, `${base}-phpstorm.zip`];

// The spec is one self-contained file: the release JSON is the source file as it is.
fs.copyFileSync(SRC, `${DIST}/${base}.json`);
redocly('bundle', SRC, '--output', `${DIST}/${base}.yaml`);
redocly('build-docs', SRC, '--output', `${DIST}/${base}.html`, '--title', `Joomla ${version} Web Services API`);
execFileSync(process.execPath, ['scripts/generate-phpstorm.mjs', SRC], { stdio: 'inherit' });
zip(`${DIST}/phpstorm`, `${base}-phpstorm`, `${DIST}/${base}-phpstorm.zip`);

const sums = files.map((f) => `${crypto.createHash('sha256').update(fs.readFileSync(path.join(DIST, f))).digest('hex')} ${f}`);
fs.writeFileSync(`${DIST}/${base}.sha256`, sums.join('\n') + '\n');
Expand Down
137 changes: 137 additions & 0 deletions scripts/generate-phpstorm.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
#!/usr/bin/env node
// Generates the PhpStorm HTTP Client collection in dist/phpstorm/ from the spec: one .http file per tag, with a
// request for each operation, and the files of scripts/phpstorm/ (README and environment files). The request
// bodies are the examples of the spec. The collection is not committed to main: the release has it as a zip file,
// and the phpstorm branch has the collection of the latest main.
//
// Usage: node scripts/generate-phpstorm.mjs [spec.json] (default: src/joomla-ws-api.json)

import fs from 'node:fs';
import path from 'node:path';
import { deref, loadSpec, METHODS } from './lib.mjs';

const OUT = 'dist/phpstorm';
const STATIC = 'scripts/phpstorm';
const specFile = process.argv[2] ?? 'src/joomla-ws-api.json';

// Name of the {id} variable, by the last word of the tag
const ID_NAMES = {
Categories: 'category_id',
Clients: 'client_id',
Form: 'contact_id',
Fields: 'field_id',
Groups: 'group_id',
Levels: 'level_id',
Items: 'menu_item_id',
Styles: 'template_style_id',
Content: 'language_id',
Overrides: 'override_key',
Requests: 'request_id',
Consents: 'consent_id',
Adapters: 'adapter_id',
Newsfeeds: 'feed_id',
Articles: 'article_id',
};

// Default values of the path variables
const DEFAULTS = {
adapter_id: 'local-images',
override_key: 'COM_EXAMPLE_KEY',
lang_code: 'en-GB',
component_name: 'com_content',
path: 'local-images:/joomla_black.png',
};

/** "Banners Categories" -> "category_id". Content history uses the ID of the item it belongs to. */
function idName(tag) {
const words = tag.split(' ');
const last = words.at(-1);
if (last === 'History') return words.at(-3).replace(/s$/, '').toLowerCase() + '_id';
return ID_NAMES[last] ?? last.replace(/s$/, '').toLowerCase() + '_id';
}

const firstSentence = (text) => text.trim().split(/(?<=\.)\s/)[0];

function bodyExample(content) {
const media = content['application/json'] ?? Object.values(content)[0];
if ('example' in media) return media.example;
if (media.examples) return deref(spec, Object.values(media.examples)[0]).value;
return {};
}

const spec = loadSpec(specFile);
const files = new Map();

// The operations in the order of the spec (operations() of lib.mjs sorts the methods)
const ordered = Object.entries(spec.paths).flatMap(([route, pathItem]) =>
Object.keys(pathItem)
.filter((method) => METHODS.includes(method))
.map((method) => ({ route, method, operation: pathItem[method], pathItem })),
);

for (const { route, method, operation, pathItem } of ordered) {
const tag = operation.tags[0];
const parameters = [...(pathItem.parameters ?? []), ...(operation.parameters ?? [])].map((p) => deref(spec, p));
const variables = {};
const notes = [];
let url = route;
for (const p of parameters.filter((p) => p.in === 'path')) {
let name = p.name;
if (name === 'id') name = (p.description ?? '').includes('ID of the version') ? 'version_id' : idName(tag);
url = url.replace(`{${p.name}}`, `{{${name}}}`);
variables[name] = p.example ?? DEFAULTS[name] ?? 1;
if (p.description) notes.push(`{{${name}}}: ${firstSentence(p.description)}`);
}
const security = operation.security ?? spec.security ?? [];
const requestBody = deref(spec, operation.requestBody);
if (!files.has(tag)) files.set(tag, []);
files.get(tag).push({
summary: operation.summary,
method: method.toUpperCase(),
url: `{{base_url}}${url}`,
variables,
notes: [...new Set(notes)],
query: parameters.filter((p) => p.in === 'query').map((p) => p.name),
updateToken: security.some((s) => 'JUpdateToken' in s),
body: requestBody ? bodyExample(requestBody.content) : null,
});
}

/** The contents of the .http file of one tag. */
function render(tag, requests) {
const lines = [
`# ${tag}`,
'#',
`# Generated from the OpenAPI spec of the Joomla Web Services API ${spec.info.version}.`,
'# Set base_url and api_key in http-client.env.json and http-client.private.env.json.',
'',
];
const variables = {};
for (const r of requests) {
for (const [name, value] of Object.entries(r.variables)) variables[name] ??= value;
}
for (const [name, value] of Object.entries(variables)) lines.push(`@${name} = ${value}`);
if (Object.keys(variables).length) lines.push('');

for (const r of requests) {
lines.push(`### ${r.summary}`, `# @name ${r.summary}`);
for (const note of r.notes) lines.push(`# ${note}`);
if (r.query.length) lines.push(`# Optional query parameters: ${r.query.join(', ')}`);
lines.push(`${r.method} ${r.url}`);
lines.push(r.updateToken ? 'X-JUpdate-Token: {{update_token}}' : 'X-Joomla-Token: {{api_key}}');
lines.push('Accept: application/vnd.api+json');
if (r.body !== null) lines.push('Content-Type: application/json', '', JSON.stringify(r.body, null, 2));
lines.push('');
}
return lines.join('\n').trimEnd() + '\n';
}

/** "Banners Categories" -> "banners-categories.http" */
const fileName = (tag) => `${tag.toLowerCase().replace(/[^a-z0-9]+/g, '-')}.http`;

const requests = [...files.values()].reduce((n, r) => n + r.length, 0);
fs.rmSync(OUT, { recursive: true, force: true });
fs.mkdirSync(OUT, { recursive: true });
fs.cpSync(STATIC, OUT, { recursive: true });
for (const [tag, tagRequests] of files) fs.writeFileSync(path.join(OUT, fileName(tag)), render(tag, tagRequests));
console.log(`${OUT}: ${files.size} files, ${requests} requests`);
1 change: 1 addition & 0 deletions scripts/phpstorm/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
http-client.private.env.json
Loading
Loading