diff --git a/.changeset/web-workspaces.md b/.changeset/web-workspaces.md new file mode 100644 index 00000000..09610831 --- /dev/null +++ b/.changeset/web-workspaces.md @@ -0,0 +1,7 @@ +--- +"@matdata/yasgui": minor +--- + +Add read-only **Web** workspaces (#256): queries published as plain `.rq` files on any web server (GitHub Pages, a static host), listed in a `workspace.json` with optional folder labels, endpoint aliases and per-query name, description and endpoint. Query files may start with grlc-style `#+ name:`, `#+ description:` and `#+ endpoint:` lines, which are removed from the query text. Opening a query switches the tab to its endpoint. + +Add one in Settings → Workspaces with the type **Web (read-only)**, or share a link with `?workspace=`: whoever opens it is asked whether to add the workspace, and then sees it in the query browser. Embedding pages can call `yasgui.addWebWorkspace(url)`, or add `{ type: "http", manifestUrl }` through `persistentConfig.addOrUpdateWorkspace`. diff --git a/docs/user-guide.md b/docs/user-guide.md index c37a75af..b5470ab4 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -58,6 +58,7 @@ - [Workspaces](#workspaces) - [SPARQL workspaces](#sparql-workspaces) - [Git based workspaces](#git-based-workspaces) + - [Web workspaces](#web-workspaces) - [Managed queries](#managed-queries) - [Query History and Persistence](#query-history-and-persistence) - [Share Queries](#share-queries) @@ -1362,6 +1363,7 @@ This feature is designed for **power users** who want to: **Not for new user onboarding:** This is not primarily intended as a way to distribute sample queries to help new users get started with an endpoint. If you need to provide example queries for onboarding, consider: +- Publishing them as a read-only [web workspace](#web-workspaces): plain `.rq` files plus a `workspace.json` on any web server, shared with one link - Using the developer configuration to embed queries in your Matgui instance - Using the snippets functionality for quick templates - Implementing a custom wrapper that loads example queries programmatically @@ -1372,7 +1374,7 @@ This feature is designed for **power users** who want to: 1. Open **Settings** (gear icon in the top bar) 2. Go to the **Workspaces** tab -3. Add one or more workspaces (Git or SPARQL) and choose a **Default workspace** +3. Add one or more workspaces (SPARQL, Git or Web) and choose a **Default workspace** 4. (Optional) Click **Validate access** to test the configuration and show how many managed queries are already saved in that workspace **Read-only workspaces:** tick **Read-only** in a workspace's settings (or set `readOnly: true` when adding it through `persistentConfig.addOrUpdateWorkspace`) to protect a shared or published set of queries. Queries in a read-only workspace can be browsed and opened, but: @@ -1565,6 +1567,58 @@ The token must have access to the specific repository you're configuring as a wo - The repository can contain other files; only `.rq` and `.sparql` files will be shown in the query browser. - Authentication/permission errors may not always display clear error messages - check browser console if saves fail silently. +##### Web workspaces + +A web workspace is a read-only set of queries published as plain files on a web server, such as GitHub Pages, a university web space or any static file host. Nothing has to be loaded into a SPARQL store and no Git token is needed. MatGUI reads one `workspace.json` file that lists the query files. + +**Add one:** in **Settings → Workspaces**, click **Add workspace**, pick the type **Web (read-only)** and enter the URL of the `workspace.json`. The label and description are filled in from the file when you leave them empty. Web workspaces are always read-only (see [Read-only workspaces](#workspaces)), and have no version history. + +**Share one with a link:** add `?workspace=` to the address of MatGUI, for example `https://matgui.matdata.eu/?workspace=https://example.org/queries/workspace.json`. Whoever opens the link is asked whether to add the workspace, and then sees its queries in the query browser. Nothing runs until they open a query and press Run. If the workspace is already configured, the link just shows it. The **Copy link** button next to a web workspace in the settings copies such a link for the current page. Embedding pages can do the same with `yasgui.addWebWorkspace(url)`. + +**Publish one:** put the query files and a `workspace.json` in a folder on the web server: + +```json +{ + "label": "Example queries", + "description": "Optional description of the workspace", + "endpoints": { + "era": "https://graph.data.era.europa.eu/repositories/rinf-plus" + }, + "folders": { + "era": { "label": "Railway infrastructure", "endpoint": "era" } + }, + "queries": [ + "era/operational-points.rq", + { + "url": "./other/example.rq", + "name": "Name shown in the query browser", + "description": "Shown in the query preview", + "endpoint": "era", + "folder": "era" + } + ] +} +``` + +- `queries` (required) lists the query files, because a web server cannot list a folder. An entry is a URL, or an object with a `url` and optional `name`, `description`, `endpoint` and `folder`. URLs are relative to `workspace.json`; absolute URLs work too. +- The folder of a query in the query browser is the folder of its file, unless the entry sets `folder`. `folders` gives a folder a readable `label` and a default `endpoint` for the queries in it and its sub-folders. +- `endpoints` defines short aliases that `endpoint` values can use. An `endpoint` can also be a full URL. Opening a query switches the tab to its endpoint. +- `label` and `description` are used as the workspace's label and description when it is added. +- Other keys are ignored. + +A query file can start with header lines in the `#+ key: value` form that [grlc](https://grlc.io) uses. MatGUI reads `name`, `description` and `endpoint` from them and removes them from the query text it shows: + +```sparql +#+ name: Operational points per country +#+ description: Counts the operational points in RINF per member state. +#+ endpoint: era +SELECT ?country (COUNT(?op) AS ?ops) WHERE { ... } +``` + +A value in `workspace.json` wins over the file header; without either, the file name is used as the name (`operational-points.rq` becomes "operational points"). + +**Web server requirements:** MatGUI fetches the files from the browser, so the server must allow cross-origin requests (send `Access-Control-Allow-Origin: *`). GitHub Pages and `raw.githubusercontent.com` do this. On an `https` page the files must be served over `https` too. Files are fetched without cookies or credentials, so they must be public. Changes on the server show up after reloading MatGUI. + #### Managed queries **Browse and open managed queries:** diff --git a/packages/yasgui/src/Tab.ts b/packages/yasgui/src/Tab.ts index e3e071fe..0e74bf27 100644 --- a/packages/yasgui/src/Tab.ts +++ b/packages/yasgui/src/Tab.ts @@ -19,7 +19,8 @@ import EndpointSelect from "./endpointSelect"; import "./tab.scss"; import { getRandomId, default as Yasgui, YasguiRequestConfig } from "./"; import * as OAuth2Utils from "./OAuth2Utils"; -import type { ManagedTabMetadata } from "./queryManagement/types"; +import type { BackendType, ManagedTabMetadata } from "./queryManagement/types"; +import { queryIdFromMetadata, versionRefFromTag, versionTagFromRef } from "./queryManagement/queryRefs"; import { hashQueryText } from "./queryManagement/textHash"; import SaveManagedQueryModal from "./queryManagement/SaveManagedQueryModal"; import { saveManagedQuery } from "./queryManagement/saveManagedQuery"; @@ -170,14 +171,11 @@ export class Tab extends EventEmitter { } private getManagedQueryIdFromMetadata(meta: ManagedTabMetadata): string | undefined { - if (meta.backendType === "git") return (meta.queryRef as any)?.path as string | undefined; - return (meta.queryRef as any)?.managedQueryIri as string | undefined; + return queryIdFromMetadata(meta); } - private versionRefFromVersionTag(backendType: "git" | "sparql", versionTag: string | undefined) { - if (!versionTag) return undefined; - if (backendType === "git") return { commitSha: versionTag }; - return { managedQueryVersionIri: versionTag }; + private versionRefFromVersionTag(backendType: BackendType, versionTag: string | undefined) { + return versionRefFromTag(backendType, versionTag); } private getQueryTextForSave(): string { @@ -254,11 +252,7 @@ export class Tab extends EventEmitter { const backend = getWorkspaceBackend(workspace, { persistentConfig: this.yasgui.persistentConfig }); - const expectedVersionTag = (() => { - if (!meta?.lastSavedVersionRef) return undefined; - if (meta.backendType === "git") return (meta.lastSavedVersionRef as any)?.commitSha; - return (meta.lastSavedVersionRef as any)?.managedQueryVersionIri; - })(); + const expectedVersionTag = versionTagFromRef(meta.backendType, meta.lastSavedVersionRef); try { await backend.writeQuery(queryId, this.getQueryTextForSave(), { expectedVersionTag }); @@ -363,9 +357,7 @@ export class Tab extends EventEmitter { const expectedVersionTag = (() => { // A version tag only applies when saving over the same query in the same workspace. if (meta?.workspaceId !== workspace.id) return undefined; - if (!meta?.lastSavedVersionRef) return undefined; - if (meta.backendType === "git") return (meta.lastSavedVersionRef as any)?.commitSha; - return (meta.lastSavedVersionRef as any)?.managedQueryVersionIri; + return versionTagFromRef(meta.backendType, meta.lastSavedVersionRef); })(); try { @@ -675,7 +667,9 @@ export class Tab extends EventEmitter { const workspace = this.yasgui.persistentConfig.getWorkspace(meta.workspaceId); const crumbs = [workspace?.label || meta.workspaceId]; const path = (meta.queryRef as any)?.path as string | undefined; - if (meta.backendType === "git" && path) crumbs.push(...path.split("/").slice(0, -1).filter(Boolean)); + if ((meta.backendType === "git" || meta.backendType === "http") && path && !/^[a-z][a-z0-9+.-]*:/i.test(path)) { + crumbs.push(...path.split("/").slice(0, -1).filter(Boolean)); + } return crumbs; } diff --git a/packages/yasgui/src/TabContextMenu.ts b/packages/yasgui/src/TabContextMenu.ts index de828244..7322a48b 100644 --- a/packages/yasgui/src/TabContextMenu.ts +++ b/packages/yasgui/src/TabContextMenu.ts @@ -5,7 +5,7 @@ import Tab from "./Tab"; import { TabListEl } from "./TabElements"; import { cloneDeep } from "lodash-es"; import { getWorkspaceBackend } from "./queryManagement/backends/getWorkspaceBackend"; -import type { GitQueryRef, SparqlQueryRef } from "./queryManagement/types"; +import { queryIdFromMetadata } from "./queryManagement/queryRefs"; import "./TabContextMenu.scss"; export interface TabContextConfig { name: string; @@ -137,10 +137,7 @@ export default class TabContextMenu { if (meta) { const workspaceConfig = this.yasgui.persistentConfig?.getWorkspaces().find((w) => w.id === meta.workspaceId); if (workspaceConfig) { - const queryId = - meta.backendType === "git" - ? (meta.queryRef as GitQueryRef).path - : (meta.queryRef as SparqlQueryRef).managedQueryIri; + const queryId = queryIdFromMetadata(meta); if (queryId) { const backend = getWorkspaceBackend(workspaceConfig, { persistentConfig: this.yasgui.persistentConfig, diff --git a/packages/yasgui/src/index.ts b/packages/yasgui/src/index.ts index ac9aa42b..8aef69c3 100644 --- a/packages/yasgui/src/index.ts +++ b/packages/yasgui/src/index.ts @@ -17,7 +17,14 @@ import GraphPlugin from "@matdata/yasgui-graph-plugin"; import TablePlugin from "@matdata/yasgui-table-plugin"; import { moveTabIdToFront, removeTabId } from "./tabNavigationHistory"; import * as dialogs from "./dialogs"; -import { setDialogHost, showToast } from "./dialogs"; +import { setDialogHost, showAlert, showConfirm, showToast } from "./dialogs"; +import HttpWorkspaceBackend from "./queryManagement/backends/HttpWorkspaceBackend"; +import { asWorkspaceBackendError } from "./queryManagement/backends/errors"; +import { + findWebWorkspace, + getWorkspaceUrlFromLink, + removeWorkspaceParamFromLink, +} from "./queryManagement/workspaceLink"; import "@matdata/yasgui-chart-plugin/dist/yasgui-chart-plugin.min.css"; import "@matdata/yasgui-explain-plugin/dist/yasgui-explain-plugin.min.css"; import "@matdata/yasgui-graph-plugin/dist/yasgui-graph-plugin.min.css"; @@ -319,6 +326,8 @@ export class Yasgui extends EventEmitter { this.rootEl.appendChild(this.queryBrowser.getElement()); let executeIdAfterInit: string | undefined; let optionsFromUrl: PersistedTabJson | undefined; + // Read before the share link below, which clears the hash + const workspaceFromLink = this.config.populateFromUrl ? this.takeWorkspaceUrlFromLink() : undefined; if (this.config.populateFromUrl) { optionsFromUrl = shareLink.getConfigFromUrl(Tab.getDefaults(this)); if (optionsFromUrl) { @@ -379,7 +388,12 @@ export class Yasgui extends EventEmitter { } else { this.sidePanel.restore(); } - if (this.quickStart && this.config.quickStartOnFirstVisit && !QuickStart.hasBeenSeen()) { + if (workspaceFromLink) { + // After the first paint, so the dialog opens over the interface + window.setTimeout(() => { + if (this.rootEl.isConnected) void this.addWebWorkspace(workspaceFromLink); + }, 300); + } else if (this.quickStart && this.config.quickStartOnFirstVisit && !QuickStart.hasBeenSeen()) { // After the first paint, so the tour opens over the interface it explains const quickStart = this.quickStart; QuickStart.markSeen(); @@ -388,6 +402,63 @@ export class Yasgui extends EventEmitter { }, 300); } } + /** The `?workspace=` URL of the page, removed from the address bar so a reload does not ask again */ + private takeWorkspaceUrlFromLink(): string | undefined { + if (typeof window === "undefined" || !window.location) return undefined; + const manifestUrl = getWorkspaceUrlFromLink(window.location.href); + if (!manifestUrl) return undefined; + try { + window.history.replaceState(window.history.state, "", removeWorkspaceParamFromLink(window.location.href)); + } catch { + // The parameter stays in the address bar; the workspace is only added once anyway + } + return manifestUrl; + } + + /** + * Add a read-only web workspace from the URL of its workspace.json, after the user confirms, and show it in + * the query browser. A workspace that is already configured is shown without asking. + * Used for links with `?workspace=`; embedding pages can call it too. + */ + public async addWebWorkspace(manifestUrl: string): Promise { + const existing = findWebWorkspace(this.persistentConfig.getWorkspaces(), manifestUrl); + if (existing) { + this.queryBrowser.showWorkspace(existing.id); + return existing.id; + } + + let info: { label?: string; description?: string; queryCount: number }; + try { + info = await new HttpWorkspaceBackend({ id: "preview", label: "", type: "http", manifestUrl }).getManifestInfo(); + } catch (e) { + await showAlert(`The workspace from this link could not be loaded. ${asWorkspaceBackendError(e).message}`, { + host: this.rootEl, + title: "Workspace not added", + }); + return undefined; + } + + const label = info.label || new URL(manifestUrl).host; + const queries = `${info.queryCount} ${info.queryCount === 1 ? "query" : "queries"}`; + const ok = await showConfirm( + `Add the read-only workspace "${label}" (${queries}) published at ${manifestUrl}? You can browse and open its queries. A query only runs when you press Run.`, + { host: this.rootEl, title: "Add workspace", confirmLabel: "Add workspace", cancelLabel: "Not now" }, + ); + if (!ok) return undefined; + + const id = `ws_web_${Date.now().toString(36)}_${Math.random().toString(16).slice(2, 8)}`; + this.persistentConfig.addOrUpdateWorkspace({ + id, + type: "http", + label, + description: info.description, + manifestUrl, + readOnly: true, + }); + this.queryBrowser.showWorkspace(id); + return id; + } + /** * Run the query of a tab opened from a share link, unless that would be unsafe without the user reviewing it */ diff --git a/packages/yasgui/src/queryManagement/QueryBrowser.ts b/packages/yasgui/src/queryManagement/QueryBrowser.ts index 5342ed39..dde60aac 100644 --- a/packages/yasgui/src/queryManagement/QueryBrowser.ts +++ b/packages/yasgui/src/queryManagement/QueryBrowser.ts @@ -8,6 +8,8 @@ import { asWorkspaceBackendError, isWorkspaceBackendError, WorkspaceBackendError import { getEndpointToAutoSwitch } from "./openManagedQuery"; import { hashQueryText } from "./textHash"; import type { BackendType, VersionRef, ManagedTabMetadata } from "./types"; +import { clearHttpWorkspaceCache } from "./backends/HttpWorkspaceBackend"; +import { queryIdFromMetadata, queryRefFromId, versionRefFromTag } from "./queryRefs"; import { normalizeQueryFilename } from "./normalizeQueryFilename"; import { isWorkspaceReadOnly } from "./readOnlyWorkspace"; import SaveManagedQueryModal from "./SaveManagedQueryModal"; @@ -78,6 +80,7 @@ export default class QueryBrowser { public invalidateAndRefresh(workspaceId?: string) { // Clear caches so new/renamed queries show up immediately. if (!workspaceId || this.treeWorkspaceId === workspaceId) { + clearHttpWorkspaceCache(); this.folderEntriesById.clear(); this.folderLoadingById.clear(); this.folderErrorById.clear(); @@ -339,6 +342,17 @@ export default class QueryBrowser { else this.open(openerEl); } + /** Open the query browser on a workspace, at its root folder */ + public showWorkspace(workspaceId: string, openerEl?: HTMLElement) { + this.selectedWorkspaceId = workspaceId; + this.yasgui.persistentConfig.setActiveWorkspaceId(workspaceId); + this.currentFolderId = undefined; + this.resetTreeState(); + this.searchEl.value = ""; + this.invalidateAndRefresh(); + this.open(openerEl); + } + private getWorkspaces(): WorkspaceConfig[] { return this.yasgui.persistentConfig.getWorkspaces(); } @@ -425,7 +439,7 @@ export default class QueryBrowser { return `⚠️ Authentication failed. Please check your workspace configuration and credentials in Settings → Workspaces.`; } if (err.code === "FORBIDDEN") { - return `⚠️ Access forbidden. Please verify your ${workspaceType === "git" ? "token permissions" : "endpoint authentication"} in Settings → Workspaces.`; + return `⚠️ Access forbidden. Please verify your ${workspaceType === "git" ? "token permissions" : workspaceType === "http" ? "web server access" : "endpoint authentication"} in Settings → Workspaces.`; } if (err.code === "NETWORK_ERROR") { return `⚠️ Unable to connect to workspace. Please check your workspace configuration and network connection.`; @@ -536,8 +550,7 @@ export default class QueryBrowser { } private getManagedQueryIdFromMetadata(meta: ManagedTabMetadata): string | undefined { - if (meta.backendType === "git") return (meta.queryRef as any)?.path as string | undefined; - return (meta.queryRef as any)?.managedQueryIri as string | undefined; + return queryIdFromMetadata(meta); } private findOpenManagedTabId(workspaceId: string, backendType: BackendType, queryId: string): string | undefined { @@ -616,9 +629,7 @@ export default class QueryBrowser { } private versionRefFromVersionTag(backendType: BackendType, versionTag: string | undefined): VersionRef | undefined { - if (!versionTag) return undefined; - if (backendType === "git") return { commitSha: versionTag }; - return { managedQueryVersionIri: versionTag }; + return versionRefFromTag(backendType, versionTag); } /** @@ -657,7 +668,7 @@ export default class QueryBrowser { const managedMetadata: ManagedTabMetadata = { workspaceId, backendType, - queryRef: backendType === "git" ? { path: queryId } : { managedQueryIri: queryId }, + queryRef: queryRefFromId(backendType, queryId), lastSavedVersionRef: this.versionRefFromVersionTag(backendType, read.versionTag), lastSavedTextHash: hashQueryText(read.queryText), }; @@ -808,13 +819,14 @@ export default class QueryBrowser { } } - // History - menu.appendChild( - makeItem("History", async () => { - if (!this.queryHistoryModal) this.queryHistoryModal = new QueryHistoryModal(this.yasgui); - await this.queryHistoryModal.show(this.selectedWorkspaceId!, backend, backend.type, entry.id, entry.label); - }), - ); + // History (plain files on a web server have none) + if (backend.type !== "http") + menu.appendChild( + makeItem("History", async () => { + if (!this.queryHistoryModal) this.queryHistoryModal = new QueryHistoryModal(this.yasgui); + await this.queryHistoryModal.show(this.selectedWorkspaceId!, backend, backend.type, entry.id, entry.label); + }), + ); // Rename if (backend.renameQuery) { diff --git a/packages/yasgui/src/queryManagement/QueryHistoryModal.ts b/packages/yasgui/src/queryManagement/QueryHistoryModal.ts index eac4e0e3..5942e167 100644 --- a/packages/yasgui/src/queryManagement/QueryHistoryModal.ts +++ b/packages/yasgui/src/queryManagement/QueryHistoryModal.ts @@ -2,6 +2,7 @@ import type Yasgui from "../index"; import { addClass, removeClass } from "@matdata/yasgui-utils"; import { getWorkspaceBackend } from "./backends/getWorkspaceBackend"; import type { BackendType, ManagedTabMetadata, ReadResult, VersionInfo, VersionRef } from "./types"; +import { queryRefFromId, versionRefFromTag } from "./queryRefs"; import { hashQueryText } from "./textHash"; import { getEndpointToAutoSwitch } from "./openManagedQuery"; import { asWorkspaceBackendError } from "./backends/errors"; @@ -318,12 +319,12 @@ export default class QueryHistoryModal { tab.setQuery(read.queryText); - const versionRef: VersionRef | undefined = this.versionRefFromVersionTag(backendType, read.versionTag); + const versionRef: VersionRef | undefined = versionRefFromTag(backendType, read.versionTag); const managedMetadata: ManagedTabMetadata = { workspaceId, backendType, - queryRef: backendType === "git" ? { path: queryId } : { managedQueryIri: queryId }, + queryRef: queryRefFromId(backendType, queryId), lastSavedVersionRef: versionRef, lastSavedTextHash: hashQueryText(read.queryText), }; @@ -332,12 +333,6 @@ export default class QueryHistoryModal { this.close(); } - private versionRefFromVersionTag(backendType: BackendType, versionTag: string | undefined): VersionRef | undefined { - if (!versionTag) return undefined; - if (backendType === "git") return { commitSha: versionTag }; - return { managedQueryVersionIri: versionTag }; - } - private formatDate(isoDate: string): string { const d = new Date(isoDate); if (isNaN(d.getTime())) return "Unknown date"; diff --git a/packages/yasgui/src/queryManagement/WorkspaceSettingsForm.ts b/packages/yasgui/src/queryManagement/WorkspaceSettingsForm.ts index 1ea4dde5..f8619847 100644 --- a/packages/yasgui/src/queryManagement/WorkspaceSettingsForm.ts +++ b/packages/yasgui/src/queryManagement/WorkspaceSettingsForm.ts @@ -5,6 +5,10 @@ import { validateWorkspaceConfig } from "./validateWorkspaceConfig"; import { getWorkspaceBackend } from "./backends/getWorkspaceBackend"; import { asWorkspaceBackendError } from "./backends/errors"; import { countManagedQueries } from "./countManagedQueries"; +import HttpWorkspaceBackend from "./backends/HttpWorkspaceBackend"; +import { isWorkspaceReadOnly } from "./readOnlyWorkspace"; +import { createWorkspaceLink } from "./workspaceLink"; +import { showCopyValue } from "../dialogs"; export interface WorkspaceSettingsFormOptions { persistentConfig: PersistentConfig; @@ -101,7 +105,7 @@ ORDER BY ?workspace`; const help = document.createElement("div"); help.textContent = - "Configure managed-query workspaces (Git or SPARQL). Credentials are stored locally and are never shown again after entry."; + "Configure managed-query workspaces (SPARQL, Git, or read-only Web). Credentials are stored locally and are never shown again after entry."; addClass(help, "settingsHelp"); header.appendChild(label); @@ -176,7 +180,7 @@ ORDER BY ?workspace`; const nameEl = document.createElement("div"); addClass(nameEl, "workspaceListLabel"); - nameEl.textContent = (workspace.label || workspace.id) + (workspace.readOnly ? " (read-only)" : ""); + nameEl.textContent = (workspace.label || workspace.id) + (isWorkspaceReadOnly(workspace) ? " (read-only)" : ""); row.appendChild(nameEl); const actions = document.createElement("div"); @@ -223,6 +227,24 @@ ORDER BY ?workspace`; actions.appendChild(editBtn); actions.appendChild(validateBtn); + if (workspace.type === "http" && typeof window !== "undefined") { + // A link that adds this workspace for whoever opens it + const linkBtn = document.createElement("button"); + linkBtn.type = "button"; + linkBtn.textContent = "Copy link"; + linkBtn.title = "Copy a link that adds this workspace for whoever opens it"; + addClass(linkBtn, "secondaryButton"); + linkBtn.onclick = async () => { + const link = createWorkspaceLink(window.location.href, workspace.manifestUrl); + try { + await navigator.clipboard.writeText(link); + status.textContent = "✓ Link copied"; + } catch { + void showCopyValue("Copy the link to this workspace", link); + } + }; + actions.appendChild(linkBtn); + } actions.appendChild(removeBtn); row.appendChild(actions); row.appendChild(status); @@ -368,8 +390,12 @@ ORDER BY ?workspace`; const optGit = document.createElement("option"); optGit.value = "git"; optGit.textContent = "Git"; + const optHttp = document.createElement("option"); + optHttp.value = "http"; + optHttp.textContent = "Web (read-only)"; typeSelect.appendChild(optSparql); typeSelect.appendChild(optGit); + typeSelect.appendChild(optHttp); if (existing) { typeSelect.value = existing.type; @@ -408,6 +434,58 @@ ORDER BY ?workspace`; const typeValue = existing?.type ?? (typeSelect.value as WorkspaceConfig["type"]); + // Web workspaces are always read-only + readOnlyInput.disabled = typeValue === "http"; + if (typeValue === "http") readOnlyInput.checked = true; + else if (!existing) readOnlyInput.checked = false; + + if (typeValue === "http") { + const httpHelp = document.createElement("div"); + addClass(httpHelp, "settingsHelp"); + httpHelp.textContent = + "Queries published as files on a web server (for example GitHub Pages). Enter the URL of the workspace.json that lists them. The server must allow cross-origin requests (CORS)."; + + const manifestUrlInput = document.createElement("input"); + manifestUrlInput.type = "url"; + manifestUrlInput.placeholder = "https://example.org/queries/workspace.json"; + manifestUrlInput.value = existing && existing.type === "http" ? existing.manifestUrl : ""; + addClass(manifestUrlInput, "settingsInput"); + + // Fill an empty label/description from the manifest, and show how many queries it lists + manifestUrlInput.onchange = async () => { + const manifestUrl = manifestUrlInput.value.trim(); + if (!manifestUrl) return; + status.textContent = "Loading workspace.json…"; + try { + const info = await new HttpWorkspaceBackend({ + id: "preview", + label: "", + type: "http", + manifestUrl, + }).getManifestInfo(); + if (manifestUrlInput.value.trim() !== manifestUrl) return; + if (!labelInput.value.trim() && info.label) labelInput.value = info.label; + if (!descriptionInput.value.trim() && info.description) descriptionInput.value = info.description; + status.textContent = `✓ Found ${info.queryCount} ${info.queryCount === 1 ? "query" : "queries"}`; + } catch (e) { + if (manifestUrlInput.value.trim() !== manifestUrl) return; + status.textContent = asWorkspaceBackendError(e).message; + } + }; + + dynamic.appendChild(httpHelp); + dynamic.appendChild(this.wrapField("workspace.json URL", manifestUrlInput)); + + (dynamic as any).__getConfig = (): WorkspaceConfig => ({ + id: existing?.id || newWorkspaceId(), + type: "http", + label: labelInput.value.trim(), + description: descriptionInput.value.trim() || undefined, + manifestUrl: manifestUrlInput.value.trim(), + }); + return; + } + if (typeValue === "git") { const gitHelp = document.createElement("div"); addClass(gitHelp, "settingsHelp"); @@ -712,7 +790,8 @@ ORDER BY ?workspace`; status.textContent = ""; const getConfig = (dynamic as any).__getConfig as undefined | (() => WorkspaceConfig); if (!getConfig) return; - const config: WorkspaceConfig = { ...getConfig(), readOnly: readOnlyInput.checked || undefined }; + const base = getConfig(); + const config: WorkspaceConfig = { ...base, readOnly: base.type === "http" || readOnlyInput.checked || undefined }; if (config.type === "git" && !isNonEmpty(config.auth.token)) { status.textContent = "Git token is required."; diff --git a/packages/yasgui/src/queryManagement/backends/HttpWorkspaceBackend.ts b/packages/yasgui/src/queryManagement/backends/HttpWorkspaceBackend.ts new file mode 100644 index 00000000..49ac65c7 --- /dev/null +++ b/packages/yasgui/src/queryManagement/backends/HttpWorkspaceBackend.ts @@ -0,0 +1,238 @@ +import type { FolderEntry, HttpWorkspaceConfig, ReadResult, VersionInfo, WriteQueryOptions } from "../types"; +import type { WorkspaceBackend } from "./WorkspaceBackend"; +import { WorkspaceBackendError } from "./errors"; +import { + folderDefaultEndpoint, + nameFromFileUrl, + normalizeFolderPath, + parseHttpWorkspaceManifest, + parseQueryFile, + resolveEndpointAlias, + type HttpWorkspaceQueryEntry, + type ParsedHttpWorkspace, +} from "./httpWorkspaceManifest"; + +/** How long a fetched manifest or query file is reused before it is fetched again */ +const CACHE_TTL_MS = 60_000; + +type CacheEntry = { at: number; value: Promise }; + +// Backends are created anew for every call of getWorkspaceBackend, so the cache lives at module level. +const manifestCache = new Map>(); +const fileCache = new Map>(); + +function cached(cache: Map>, key: string, load: () => Promise): Promise { + const hit = cache.get(key); + if (hit && Date.now() - hit.at < CACHE_TTL_MS) return hit.value; + const value = load(); + cache.set(key, { at: Date.now(), value }); + // Never keep a failure: the next call tries again. + value.catch(() => { + if (cache.get(key)?.value === value) cache.delete(key); + }); + return value; +} + +/** Forget fetched manifests and query files, so the next read fetches them again */ +export function clearHttpWorkspaceCache() { + manifestCache.clear(); + fileCache.clear(); +} + +function isMixedContent(url: string): boolean { + try { + return ( + typeof window !== "undefined" && + window.location?.protocol === "https:" && + new URL(url, window.location.href).protocol === "http:" + ); + } catch { + return false; + } +} + +async function fetchText(url: string, what: string): Promise { + if (isMixedContent(url)) { + throw new WorkspaceBackendError( + "NETWORK_ERROR", + `The ${what} is served over http, and browsers block it on an https page. Use an https URL: ${url}`, + ); + } + let res: Response; + try { + res = await fetch(url, { credentials: "omit", headers: { Accept: "application/json, text/plain, */*" } }); + } catch { + throw new WorkspaceBackendError( + "NETWORK_ERROR", + `Could not load the ${what} (${url}). Check the URL, and that the web server allows cross-origin requests (CORS).`, + ); + } + if (res.status === 404 || res.status === 410) { + throw new WorkspaceBackendError("NOT_FOUND", `The ${what} was not found (${url})`); + } + if (res.status === 401) throw new WorkspaceBackendError("AUTH_FAILED", `The ${what} needs a login (${url})`); + if (res.status === 403) throw new WorkspaceBackendError("FORBIDDEN", `Access to the ${what} is forbidden (${url})`); + if (res.status === 429) throw new WorkspaceBackendError("RATE_LIMITED", `Too many requests for the ${what} (${url})`); + if (!res.ok) { + throw new WorkspaceBackendError("UNKNOWN", `Could not load the ${what} (${url}): HTTP ${res.status}`); + } + return res.text(); +} + +/** + * A read-only workspace published as plain files on a web server (GitHub Pages, any static host): + * a `workspace.json` manifest that lists `.rq` files. See httpWorkspaceManifest.ts for the format. + */ +export default class HttpWorkspaceBackend implements WorkspaceBackend { + public readonly type = "http" as const; + + constructor(private readonly config: HttpWorkspaceConfig) {} + + private loadManifest(): Promise { + const manifestUrl = this.config.manifestUrl?.trim(); + if (!manifestUrl) return Promise.reject(new WorkspaceBackendError("NOT_FOUND", "No workspace.json URL configured")); + return cached(manifestCache, manifestUrl, async () => { + const text = await fetchText(manifestUrl, "workspace.json"); + let json: unknown; + try { + json = JSON.parse(text); + } catch { + throw new WorkspaceBackendError("UNKNOWN", `workspace.json is not valid JSON (${manifestUrl})`); + } + try { + return parseHttpWorkspaceManifest(json, manifestUrl); + } catch (e) { + throw new WorkspaceBackendError("UNKNOWN", e instanceof Error ? e.message : String(e)); + } + }); + } + + private loadFile(entry: HttpWorkspaceQueryEntry): Promise { + return cached(fileCache, entry.url, () => fetchText(entry.url, "query file")); + } + + private async findEntry(queryId: string): Promise<{ manifest: ParsedHttpWorkspace; entry: HttpWorkspaceQueryEntry }> { + const manifest = await this.loadManifest(); + const entry = manifest.queries.find((q) => q.id === queryId); + if (!entry) throw new WorkspaceBackendError("NOT_FOUND", `Query not found: ${queryId}`); + return { manifest, entry }; + } + + /** The name of a query: from the manifest, else the file header, else the file name */ + private async labelOf(entry: HttpWorkspaceQueryEntry): Promise { + if (entry.name) return entry.name; + try { + const { meta } = parseQueryFile(await this.loadFile(entry)); + if (meta.name?.trim()) return meta.name.trim(); + } catch { + // A file that cannot be loaded is still listed; opening it shows the error. + } + return nameFromFileUrl(entry.url); + } + + private async toQueryEntries(entries: HttpWorkspaceQueryEntry[]): Promise { + return Promise.all( + entries.map( + async (q): Promise => ({ + kind: "query", + id: q.id, + label: await this.labelOf(q), + parentId: q.folder || undefined, + }), + ), + ); + } + + private folderLabel(manifest: ParsedHttpWorkspace, path: string): string { + return manifest.folders[path]?.label || path.split("/").pop() || path; + } + + async validateAccess(): Promise { + await this.loadManifest(); + } + + async listFolder(folderId?: string): Promise { + const manifest = await this.loadManifest(); + const folder = normalizeFolderPath(folderId); + const prefix = folder ? `${folder}/` : ""; + + // Folders: every folder that holds a query, plus the folders named in the manifest + const allFolders = new Set(); + const addWithParents = (path: string) => { + const parts = path.split("/").filter(Boolean); + for (let i = 1; i <= parts.length; i++) allFolders.add(parts.slice(0, i).join("/")); + }; + manifest.queries.forEach((q) => q.folder && addWithParents(q.folder)); + Object.keys(manifest.folders).forEach(addWithParents); + + const folders: FolderEntry[] = []; + for (const path of allFolders) { + if (!path.startsWith(prefix)) continue; + const rest = path.slice(prefix.length); + if (!rest || rest.includes("/")) continue; + // Hide folders that are empty, all the way down + if (!manifest.queries.some((q) => q.folder === path || q.folder.startsWith(`${path}/`))) continue; + folders.push({ + kind: "folder", + id: path, + label: this.folderLabel(manifest, path), + parentId: folder || undefined, + }); + } + + const queries = await this.toQueryEntries(manifest.queries.filter((q) => q.folder === folder)); + + const byLabel = (a: FolderEntry, b: FolderEntry) => a.label.localeCompare(b.label); + return [...folders.sort(byLabel), ...queries.sort(byLabel)]; + } + + async searchByName(query: string): Promise { + const q = query.trim().toLowerCase(); + if (!q) return []; + const manifest = await this.loadManifest(); + const all = await this.toQueryEntries(manifest.queries); + return all.filter((e) => e.label.toLowerCase().includes(q)).sort((a, b) => a.label.localeCompare(b.label)); + } + + async readQuery(queryId: string): Promise { + const { manifest, entry } = await this.findEntry(queryId); + const { meta, queryText } = parseQueryFile(await this.loadFile(entry)); + const endpoint = + entry.endpoint || meta.endpoint?.trim() || folderDefaultEndpoint(entry.folder, manifest.folders) || undefined; + return { + queryText, + description: entry.description || meta.description?.trim() || undefined, + associatedEndpoint: resolveEndpointAlias(endpoint, manifest.endpoints), + }; + } + + async writeQuery(_queryId: string, _queryText: string, _options?: WriteQueryOptions): Promise { + throw new WorkspaceBackendError("FORBIDDEN", "Web workspaces are read-only"); + } + + async listVersions(_queryId: string): Promise { + // Plain files have no history + return []; + } + + async readVersion(queryId: string, _versionId: string): Promise { + return this.readQuery(queryId); + } + + getQueryUri(queryId: string): string | undefined { + const manifestUrl = this.config.manifestUrl?.trim(); + if (!manifestUrl) return undefined; + try { + // Same resolution as the manifest parser: ids are relative to the manifest's directory. + return new URL(queryId, manifestUrl).toString(); + } catch { + return undefined; + } + } + + /** The label and description that the manifest gives the workspace, if any */ + async getManifestInfo(): Promise<{ label?: string; description?: string; queryCount: number }> { + const manifest = await this.loadManifest(); + return { label: manifest.label, description: manifest.description, queryCount: manifest.queries.length }; + } +} diff --git a/packages/yasgui/src/queryManagement/backends/getWorkspaceBackend.ts b/packages/yasgui/src/queryManagement/backends/getWorkspaceBackend.ts index 3ec98091..a9eebbc1 100644 --- a/packages/yasgui/src/queryManagement/backends/getWorkspaceBackend.ts +++ b/packages/yasgui/src/queryManagement/backends/getWorkspaceBackend.ts @@ -3,6 +3,7 @@ import type { WorkspaceBackend } from "./WorkspaceBackend"; import GitWorkspaceBackend from "./GitWorkspaceBackend"; import SparqlWorkspaceBackend from "./SparqlWorkspaceBackend"; import ReadOnlyWorkspaceBackend from "./ReadOnlyWorkspaceBackend"; +import HttpWorkspaceBackend from "./HttpWorkspaceBackend"; import { GithubProviderClient } from "./GithubProviderClient"; import { GitlabProviderClient } from "./GitlabProviderClient"; import { BitbucketProviderClient } from "./BitbucketProviderClient"; @@ -109,7 +110,8 @@ export function getWorkspaceBackend( }, ): WorkspaceBackend { const backend = createWorkspaceBackend(config, options); - return config.readOnly ? new ReadOnlyWorkspaceBackend(backend) : backend; + // Web workspaces are always read-only, whatever the config says + return config.readOnly || config.type === "http" ? new ReadOnlyWorkspaceBackend(backend) : backend; } function createWorkspaceBackend( @@ -120,6 +122,7 @@ function createWorkspaceBackend( ): WorkspaceBackend { const registered = registeredBackends.get(config.id); if (registered) return registered; + if (config.type === "http") return new HttpWorkspaceBackend(config); if (config.type === "git") { const client = (GithubProviderClient.canHandle(config) && new GithubProviderClient()) || diff --git a/packages/yasgui/src/queryManagement/backends/httpWorkspaceManifest.ts b/packages/yasgui/src/queryManagement/backends/httpWorkspaceManifest.ts new file mode 100644 index 00000000..7364dc55 --- /dev/null +++ b/packages/yasgui/src/queryManagement/backends/httpWorkspaceManifest.ts @@ -0,0 +1,243 @@ +/** + * The `workspace.json` manifest of a web (HTTP) workspace, and the `#+ key: value` header of its query files. + * + * ```json + * { + * "label": "Example queries", + * "description": "Optional", + * "endpoints": { "era": "https://graph.data.era.europa.eu/repositories/rinf-plus" }, + * "folders": { "era": { "label": "ERA", "endpoint": "era" } }, + * "queries": [ + * "./era/operational-points.rq", + * { "url": "./other/example.rq", "name": "…", "description": "…", "endpoint": "era", "folder": "era" } + * ] + * } + * ``` + */ + +export interface HttpWorkspaceManifestFolder { + label?: string; + description?: string; + /** Default endpoint (alias or URL) for the queries in this folder and its sub-folders */ + endpoint?: string; +} + +export interface HttpWorkspaceManifestQuery { + url: string; + name?: string; + description?: string; + endpoint?: string; + folder?: string; +} + +export interface HttpWorkspaceManifest { + label?: string; + description?: string; + endpoints?: Record; + folders?: Record; + queries: Array; +} + +/** A query of the manifest, with its URL resolved and its folder known */ +export interface HttpWorkspaceQueryEntry { + /** Query id: the path relative to the manifest's directory, or the absolute URL for a file elsewhere */ + id: string; + /** Absolute URL of the query file */ + url: string; + /** Folder path ("" for the root) */ + folder: string; + /** Name from the manifest (overrides the file header) */ + name?: string; + /** Description from the manifest (overrides the file header) */ + description?: string; + /** Endpoint (alias or URL) from the manifest (overrides the file header) */ + endpoint?: string; +} + +export interface ParsedHttpWorkspace { + label?: string; + description?: string; + endpoints: Record; + folders: Record; + queries: HttpWorkspaceQueryEntry[]; +} + +function isRecord(value: unknown): value is Record { + return !!value && typeof value === "object" && !Array.isArray(value); +} + +function optionalString(value: unknown): string | undefined { + return typeof value === "string" && value.trim() ? value.trim() : undefined; +} + +export function normalizeFolderPath(path: string | undefined): string { + return (path || "") + .split("/") + .map((p) => p.trim()) + .filter((p) => p && p !== ".") + .join("/"); +} + +function dirnameOf(path: string): string { + const parts = path.split("/"); + parts.pop(); + return normalizeFolderPath(parts.join("/")); +} + +/** The directory URL of a manifest URL, with a trailing slash */ +function baseDirectoryUrl(manifestUrl: string): string { + const url = new URL(manifestUrl); + url.search = ""; + url.hash = ""; + url.pathname = url.pathname.replace(/[^/]*$/, ""); + return url.toString(); +} + +/** + * Parse and validate a manifest. Throws an Error with a user-facing message when it is not usable. + * Unknown keys are ignored, so the manifest can carry extra data for other tools. + */ +export function parseHttpWorkspaceManifest(json: unknown, manifestUrl: string): ParsedHttpWorkspace { + if (!isRecord(json)) throw new Error("workspace.json must contain a JSON object"); + if (!Array.isArray(json.queries)) { + throw new Error('workspace.json must have a "queries" list with the URLs of the query files'); + } + + const base = baseDirectoryUrl(manifestUrl); + + const endpoints: Record = {}; + if (isRecord(json.endpoints)) { + for (const [alias, url] of Object.entries(json.endpoints)) { + if (typeof url === "string" && url.trim()) endpoints[alias] = url.trim(); + } + } + + const folders: Record = {}; + if (isRecord(json.folders)) { + for (const [path, value] of Object.entries(json.folders)) { + const key = normalizeFolderPath(path); + if (!key || !isRecord(value)) continue; + folders[key] = { + label: optionalString(value.label), + description: optionalString(value.description), + endpoint: optionalString(value.endpoint), + }; + } + } + + const queries: HttpWorkspaceQueryEntry[] = []; + const seen = new Set(); + json.queries.forEach((raw, index) => { + const item: Partial | undefined = + typeof raw === "string" ? { url: raw } : isRecord(raw) ? (raw as Partial) : undefined; + const href = optionalString(item?.url); + if (!item || !href) throw new Error(`Query ${index + 1} in workspace.json has no "url"`); + + let absolute: URL; + try { + absolute = new URL(href, base); + } catch { + throw new Error(`Query ${index + 1} in workspace.json has an invalid url: ${href}`); + } + if (absolute.protocol !== "https:" && absolute.protocol !== "http:") { + throw new Error(`Query ${index + 1} in workspace.json must be an http(s) URL: ${href}`); + } + const url = absolute.toString(); + const id = url.startsWith(base) ? decodeURIComponent(url.slice(base.length)) : url; + if (seen.has(id)) return; + seen.add(id); + + const folder = + item.folder !== undefined + ? normalizeFolderPath(String(item.folder)) + : id === url + ? "" // A file outside the manifest's directory goes in the root + : dirnameOf(id); + + queries.push({ + id, + url, + folder, + name: optionalString(item.name), + description: optionalString(item.description), + endpoint: optionalString(item.endpoint), + }); + }); + + return { + label: optionalString(json.label), + description: optionalString(json.description), + endpoints, + folders, + queries, + }; +} + +export interface ParsedQueryFile { + /** Header values (keys in lower case) */ + meta: Record; + /** Query text without the header lines */ + queryText: string; +} + +const HEADER_RE = /^#\+\s*([A-Za-z_-]+)\s*:\s*(.*?)\s*$/; + +/** + * Split the leading `#+ key: value` lines (the convention grlc uses) from a query file. + * Only the lines at the very top count; the rest of the file is the query. + */ +export function parseQueryFile(text: string): ParsedQueryFile { + const lines = text.replace(/\r\n?/g, "\n").split("\n"); + const meta: Record = {}; + let start = 0; + while (start < lines.length) { + const match = HEADER_RE.exec(lines[start]); + if (!match) break; + meta[match[1].toLowerCase()] = match[2]; + start++; + } + if (start === 0) return { meta, queryText: text }; + const queryText = lines.slice(start).join("\n").replace(/^\n+/, ""); + return { meta, queryText }; +} + +/** Resolve an endpoint alias of the manifest; anything else is returned as is */ +export function resolveEndpointAlias( + endpoint: string | undefined, + endpoints: Record, +): string | undefined { + const value = endpoint?.trim(); + if (!value) return undefined; + return endpoints[value] || value; +} + +/** The default endpoint of a folder: its own, else the nearest parent's */ +export function folderDefaultEndpoint( + folder: string, + folders: Record, +): string | undefined { + const parts = normalizeFolderPath(folder).split("/").filter(Boolean); + while (parts.length) { + const endpoint = folders[parts.join("/")]?.endpoint; + if (endpoint) return endpoint; + parts.pop(); + } + return undefined; +} + +/** A readable name for a query file without a name: "operational-points.rq" → "operational points" */ +export function nameFromFileUrl(url: string): string { + const last = url.split(/[?#]/)[0].split("/").filter(Boolean).pop() || url; + let decoded = last; + try { + decoded = decodeURIComponent(last); + } catch { + // keep as is + } + return ( + decoded + .replace(/\.(rq|sparql|txt)$/i, "") + .replace(/[-_]+/g, " ") + .trim() || decoded + ); +} diff --git a/packages/yasgui/src/queryManagement/index.ts b/packages/yasgui/src/queryManagement/index.ts index a33ff4f3..94315631 100644 --- a/packages/yasgui/src/queryManagement/index.ts +++ b/packages/yasgui/src/queryManagement/index.ts @@ -16,3 +16,6 @@ export { default as InMemoryWorkspaceBackend } from "./backends/InMemoryWorkspac export { default as GitWorkspaceBackend } from "./backends/GitWorkspaceBackend"; export { default as SparqlWorkspaceBackend } from "./backends/SparqlWorkspaceBackend"; export { default as ReadOnlyWorkspaceBackend } from "./backends/ReadOnlyWorkspaceBackend"; +export { default as HttpWorkspaceBackend, clearHttpWorkspaceCache } from "./backends/HttpWorkspaceBackend"; +export * from "./backends/httpWorkspaceManifest"; +export * from "./queryRefs"; diff --git a/packages/yasgui/src/queryManagement/queryRefs.ts b/packages/yasgui/src/queryManagement/queryRefs.ts new file mode 100644 index 00000000..419e2869 --- /dev/null +++ b/packages/yasgui/src/queryManagement/queryRefs.ts @@ -0,0 +1,33 @@ +import type { BackendType, ManagedTabMetadata, QueryRef, VersionRef } from "./types"; + +/** + * The reference that is stored in a tab for a query of a workspace. + * Git and web workspaces identify a query by its path, SPARQL workspaces by the managed query IRI. + */ +export function queryRefFromId(backendType: BackendType, queryId: string): QueryRef { + if (backendType === "sparql") return { managedQueryIri: queryId }; + return { path: queryId }; +} + +/** The query id (path or managed query IRI) stored in a tab's metadata */ +export function queryIdFromMetadata(meta: ManagedTabMetadata): string | undefined { + const ref = meta.queryRef as any; + if (meta.backendType === "sparql") return ref?.managedQueryIri as string | undefined; + return ref?.path as string | undefined; +} + +/** The version reference for a version tag returned by a backend. Web workspaces have no versions. */ +export function versionRefFromTag(backendType: BackendType, versionTag: string | undefined): VersionRef | undefined { + if (!versionTag) return undefined; + if (backendType === "git") return { commitSha: versionTag }; + if (backendType === "sparql") return { managedQueryVersionIri: versionTag }; + return undefined; +} + +/** The version tag of a stored version reference, used for optimistic concurrency on save */ +export function versionTagFromRef(backendType: BackendType, versionRef: VersionRef | undefined): string | undefined { + if (!versionRef) return undefined; + if (backendType === "git") return (versionRef as any)?.commitSha as string | undefined; + if (backendType === "sparql") return (versionRef as any)?.managedQueryVersionIri as string | undefined; + return undefined; +} diff --git a/packages/yasgui/src/queryManagement/readOnlyWorkspace.ts b/packages/yasgui/src/queryManagement/readOnlyWorkspace.ts index beedd9a9..6ed93250 100644 --- a/packages/yasgui/src/queryManagement/readOnlyWorkspace.ts +++ b/packages/yasgui/src/queryManagement/readOnlyWorkspace.ts @@ -1,7 +1,7 @@ import type { WorkspaceConfig } from "./types"; export function isWorkspaceReadOnly(workspace: WorkspaceConfig | undefined): boolean { - return !!workspace?.readOnly; + return !!workspace?.readOnly || workspace?.type === "http"; } export function getWritableWorkspaces(workspaces: WorkspaceConfig[]): WorkspaceConfig[] { diff --git a/packages/yasgui/src/queryManagement/saveManagedQuery.ts b/packages/yasgui/src/queryManagement/saveManagedQuery.ts index f2a03b84..b18b41c2 100644 --- a/packages/yasgui/src/queryManagement/saveManagedQuery.ts +++ b/packages/yasgui/src/queryManagement/saveManagedQuery.ts @@ -1,5 +1,6 @@ import type { WorkspaceBackend } from "./backends/WorkspaceBackend"; -import type { BackendType, ManagedTabMetadata, VersionRef } from "./types"; +import type { BackendType, ManagedTabMetadata } from "./types"; +import { queryRefFromId, versionRefFromTag } from "./queryRefs"; import { asWorkspaceBackendError } from "./backends/errors"; import { hashQueryText } from "./textHash"; import { normalizeQueryFilename } from "./normalizeQueryFilename"; @@ -59,14 +60,11 @@ function mintSparqlManagedQueryIri(workspaceIri: string): string { return `${workspaceIri.trim()}_mq_${uuidV4()}`; } -function versionRefFromVersionTag(backendType: BackendType, versionTag: string | undefined): VersionRef | undefined { - if (!versionTag) return undefined; - if (backendType === "git") return { commitSha: versionTag }; - return { managedQueryVersionIri: versionTag }; -} - export async function saveManagedQuery(input: SaveManagedQueryInput): Promise { const folderId = normalizeFolderPath(input.folderPath); + if (input.backendType === "http") { + throw new Error("Web workspaces are read-only"); + } const queryId = input.backendType === "git" ? buildQueryId(input.folderPath, input.filename) : ""; if (input.backendType === "sparql" && !input.workspaceIri?.trim()) { @@ -126,8 +124,8 @@ export async function saveManagedQuery(input: SaveManagedQueryInput): Promise` */ +export const WORKSPACE_LINK_PARAM = "workspace"; + +/** + * The web workspace URL a page was opened with, from its query string or its hash. + * Only absolute http(s) URLs count. + */ +export function getWorkspaceUrlFromLink(href: string): string | undefined { + let url: URL; + try { + url = new URL(href); + } catch { + return undefined; + } + const fromHash = url.hash.length > 1 ? new URLSearchParams(url.hash.slice(1)).get(WORKSPACE_LINK_PARAM) : null; + const value = (url.searchParams.get(WORKSPACE_LINK_PARAM) || fromHash || "").trim(); + if (!value) return undefined; + try { + const target = new URL(value); + if (target.protocol !== "https:" && target.protocol !== "http:") return undefined; + return target.toString(); + } catch { + return undefined; + } +} + +/** The page URL without the workspace parameter, so a reload does not ask again */ +export function removeWorkspaceParamFromLink(href: string): string { + const url = new URL(href); + url.searchParams.delete(WORKSPACE_LINK_PARAM); + if (url.hash.length > 1) { + const hashParams = new URLSearchParams(url.hash.slice(1)); + if (hashParams.has(WORKSPACE_LINK_PARAM)) { + hashParams.delete(WORKSPACE_LINK_PARAM); + const rest = hashParams.toString(); + url.hash = rest ? `#${rest}` : ""; + } + } + return url.toString(); +} + +/** A link to this page that adds the web workspace for whoever opens it */ +export function createWorkspaceLink(pageHref: string, manifestUrl: string): string { + const url = new URL(pageHref); + url.hash = ""; + url.searchParams.set(WORKSPACE_LINK_PARAM, manifestUrl); + return url.toString(); +} + +/** The configured web workspace for a workspace.json URL, if there is one */ +export function findWebWorkspace(workspaces: WorkspaceConfig[], manifestUrl: string): HttpWorkspaceConfig | undefined { + const normalize = (u: string) => { + try { + return new URL(u.trim()).toString(); + } catch { + return u.trim(); + } + }; + const target = normalize(manifestUrl); + return workspaces.find((w): w is HttpWorkspaceConfig => w.type === "http" && normalize(w.manifestUrl) === target); +} diff --git a/test/run.ts b/test/run.ts index 17d7d38f..2602ab4f 100644 --- a/test/run.ts +++ b/test/run.ts @@ -16,6 +16,7 @@ import "./ai-assistant-browser.js"; import "./sparql-anything-browser.js"; import "./whats-new-browser.js"; import "./share-link-browser.js"; +import "./web-workspace-browser.js"; declare var window: Window & { Yasqe: typeof Yasqe; diff --git a/test/unit/query-management-web-workspace-test.ts b/test/unit/query-management-web-workspace-test.ts new file mode 100644 index 00000000..3835eee5 --- /dev/null +++ b/test/unit/query-management-web-workspace-test.ts @@ -0,0 +1,274 @@ +import * as chai from "chai"; +import { describe, it, beforeEach, afterEach } from "mocha"; + +import HttpWorkspaceBackend, { + clearHttpWorkspaceCache, +} from "../../packages/yasgui/src/queryManagement/backends/HttpWorkspaceBackend.js"; +import { + folderDefaultEndpoint, + nameFromFileUrl, + parseHttpWorkspaceManifest, + parseQueryFile, +} from "../../packages/yasgui/src/queryManagement/backends/httpWorkspaceManifest.js"; +import { getWorkspaceBackend } from "../../packages/yasgui/src/queryManagement/backends/getWorkspaceBackend.js"; +import { isWorkspaceReadOnly } from "../../packages/yasgui/src/queryManagement/readOnlyWorkspace.js"; +import { validateWorkspaceConfig } from "../../packages/yasgui/src/queryManagement/validateWorkspaceConfig.js"; +import { getEndpointToAutoSwitch } from "../../packages/yasgui/src/queryManagement/openManagedQuery.js"; +import { + queryIdFromMetadata, + queryRefFromId, + versionRefFromTag, + versionTagFromRef, +} from "../../packages/yasgui/src/queryManagement/queryRefs.js"; +import { + createWorkspaceLink, + findWebWorkspace, + getWorkspaceUrlFromLink, + removeWorkspaceParamFromLink, +} from "../../packages/yasgui/src/queryManagement/workspaceLink.js"; +import type { HttpWorkspaceConfig, WorkspaceConfig } from "../../packages/yasgui/src/queryManagement/types.js"; + +const expect = chai.expect; + +const MANIFEST_URL = "https://example.org/queries/workspace.json"; + +function webWorkspace(manifestUrl = MANIFEST_URL): HttpWorkspaceConfig { + return { id: "web", label: "Web", type: "http", manifestUrl }; +} + +describe("Query management - web workspaces", () => { + describe("manifest", () => { + it("resolves query URLs relative to the manifest and derives folders from their paths", () => { + const parsed = parseHttpWorkspaceManifest( + { + label: "Example queries", + endpoints: { era: "https://era.example/sparql" }, + folders: { "era/maps": { label: "Maps" }, era: { label: "ERA", endpoint: "era" } }, + queries: [ + "./era/op.rq", + "era/maps/lines.rq", + { url: "top.rq", name: "Top", endpoint: "https://other.example/sparql" }, + { url: "https://elsewhere.example/q.rq" }, + { url: "./misc/x.rq", folder: "/era/" }, + "./era/op.rq", + ], + workspaceIri: "ignored", + }, + `${MANIFEST_URL}?v=2`, + ); + + expect(parsed.label).to.equal("Example queries"); + expect(parsed.queries.map((q) => [q.id, q.url, q.folder])).to.deep.equal([ + ["era/op.rq", "https://example.org/queries/era/op.rq", "era"], + ["era/maps/lines.rq", "https://example.org/queries/era/maps/lines.rq", "era/maps"], + ["top.rq", "https://example.org/queries/top.rq", ""], + ["https://elsewhere.example/q.rq", "https://elsewhere.example/q.rq", ""], + ["misc/x.rq", "https://example.org/queries/misc/x.rq", "era"], + ]); + expect(parsed.queries[2]).to.include({ name: "Top", endpoint: "https://other.example/sparql" }); + expect(folderDefaultEndpoint("era/maps", parsed.folders)).to.equal("era"); + expect(folderDefaultEndpoint("other", parsed.folders)).to.equal(undefined); + }); + + it("rejects a manifest without a queries list or with an unusable entry", () => { + expect(() => parseHttpWorkspaceManifest([], MANIFEST_URL)).to.throw("JSON object"); + expect(() => parseHttpWorkspaceManifest({ label: "x" }, MANIFEST_URL)).to.throw('"queries"'); + expect(() => parseHttpWorkspaceManifest({ queries: [{ name: "x" }] }, MANIFEST_URL)).to.throw('no "url"'); + expect(() => parseHttpWorkspaceManifest({ queries: ["javascript:alert(1)"] }, MANIFEST_URL)).to.throw("http(s)"); + }); + + it("splits the #+ header from the query text", () => { + const parsed = parseQueryFile( + "#+ name: Operational points\r\n#+ Description: Per country\n#+ endpoint: era\n\nSELECT * WHERE { ?s ?p ?o }\n# not a header\n", + ); + expect(parsed.meta).to.deep.equal({ name: "Operational points", description: "Per country", endpoint: "era" }); + expect(parsed.queryText).to.equal("SELECT * WHERE { ?s ?p ?o }\n# not a header\n"); + + const plain = "# A comment\nSELECT 1 WHERE {}"; + expect(parseQueryFile(plain)).to.deep.equal({ meta: {}, queryText: plain }); + }); + + it("makes a readable name from a file name", () => { + expect(nameFromFileUrl("https://x/y/operational-points_per%20country.rq")).to.equal( + "operational points per country", + ); + expect(nameFromFileUrl("https://x/y/q.sparql?raw=1")).to.equal("q"); + }); + }); + + describe("backend", () => { + const files: Record = {}; + const requested: string[] = []; + let originalFetch: typeof fetch; + + beforeEach(() => { + clearHttpWorkspaceCache(); + requested.length = 0; + for (const key of Object.keys(files)) delete files[key]; + originalFetch = globalThis.fetch; + globalThis.fetch = (async (url: string, init?: RequestInit) => { + requested.push(url); + expect(init?.credentials).to.equal("omit"); + if (!(url in files)) return new Response("not found", { status: 404 }); + return new Response(files[url], { status: 200 }); + }) as typeof fetch; + }); + + afterEach(() => { + globalThis.fetch = originalFetch; + clearHttpWorkspaceCache(); + }); + + function publish() { + files[MANIFEST_URL] = JSON.stringify({ + label: "Example queries", + description: "Published as files", + endpoints: { era: "https://era.example/sparql" }, + folders: { era: { label: "ERA", endpoint: "era" }, empty: { label: "Nothing here" } }, + queries: [ + "era/op.rq", + "era/maps/lines.rq", + { url: "hello.rq", name: "Hello from the manifest", description: "Manifest description" }, + "missing.rq", + ], + }); + files["https://example.org/queries/era/op.rq"] = + "#+ name: Operational points\n#+ description: Per country\nSELECT * WHERE { ?op a ?type }\n"; + files["https://example.org/queries/era/maps/lines.rq"] = + "#+ name: Lines\n#+ endpoint: https://maps.example/sparql\nSELECT ?line WHERE { ?line ?p ?o }\n"; + files["https://example.org/queries/hello.rq"] = "#+ name: Hello\n#+ endpoint: era\nSELECT ('hello' AS ?x) {}\n"; + } + + it("lists folders and queries with names from the manifest, the file header or the file name", async () => { + publish(); + const backend = new HttpWorkspaceBackend(webWorkspace()); + + const root = await backend.listFolder(); + expect(root.map((e) => [e.kind, e.id, e.label])).to.deep.equal([ + ["folder", "era", "ERA"], + ["query", "hello.rq", "Hello from the manifest"], + ["query", "missing.rq", "missing"], + ]); + + const era = await backend.listFolder("era"); + expect(era.map((e) => [e.kind, e.id, e.label, e.parentId])).to.deep.equal([ + ["folder", "era/maps", "maps", "era"], + ["query", "era/op.rq", "Operational points", "era"], + ]); + + const found = await backend.searchByName("LINE"); + expect(found.map((e) => e.id)).to.deep.equal(["era/maps/lines.rq"]); + }); + + it("reads a query without its header, with the endpoint from the entry, the header or the folder", async () => { + publish(); + const backend = new HttpWorkspaceBackend(webWorkspace()); + + const op = await backend.readQuery("era/op.rq"); + expect(op).to.deep.equal({ + queryText: "SELECT * WHERE { ?op a ?type }\n", + description: "Per country", + associatedEndpoint: "https://era.example/sparql", + }); + expect((await backend.readQuery("era/maps/lines.rq")).associatedEndpoint).to.equal("https://maps.example/sparql"); + const hello = await backend.readQuery("hello.rq"); + expect(hello).to.include({ + description: "Manifest description", + associatedEndpoint: "https://era.example/sparql", + }); + + expect(getEndpointToAutoSwitch("http", op)).to.equal("https://era.example/sparql"); + expect(backend.getQueryUri("era/op.rq")).to.equal("https://example.org/queries/era/op.rq"); + expect(await backend.listVersions("era/op.rq")).to.deep.equal([]); + }); + + it("fetches the manifest and the files once, and reports what is missing", async () => { + publish(); + const backend = new HttpWorkspaceBackend(webWorkspace()); + await backend.listFolder(); + await new HttpWorkspaceBackend(webWorkspace()).listFolder(); + expect(requested.filter((u) => u === MANIFEST_URL)).to.have.length(1); + + const missing = await backend.readQuery("missing.rq").catch((e) => e); + expect(missing).to.include({ name: "WorkspaceBackendError", code: "NOT_FOUND" }); + const unknown = await backend.readQuery("nope.rq").catch((e) => e); + expect(unknown).to.include({ code: "NOT_FOUND" }); + + const noManifest = await new HttpWorkspaceBackend(webWorkspace("https://example.org/none/workspace.json")) + .validateAccess() + .catch((e) => e); + expect(noManifest).to.include({ code: "NOT_FOUND" }); + + files["https://example.org/bad/workspace.json"] = "{ not json"; + const bad = await new HttpWorkspaceBackend(webWorkspace("https://example.org/bad/workspace.json")) + .validateAccess() + .catch((e) => e); + expect(bad.message).to.contain("not valid JSON"); + }); + + it("is always read-only", async () => { + publish(); + const config = webWorkspace(); + expect(isWorkspaceReadOnly(config)).to.equal(true); + const backend = getWorkspaceBackend(config); + expect(backend.type).to.equal("http"); + expect(backend.renameQuery).to.equal(undefined); + expect(backend.deleteQuery).to.equal(undefined); + const err = await backend.writeQuery("era/op.rq", "SELECT 1 {}").catch((e) => e); + expect(err).to.include({ code: "FORBIDDEN" }); + expect((await backend.readQuery("era/op.rq")).queryText).to.contain("?op a ?type"); + }); + }); + + it("validates the workspace.json URL", () => { + expect(validateWorkspaceConfig(webWorkspace()).valid).to.equal(true); + expect(validateWorkspaceConfig(webWorkspace("")).errors).to.deep.equal([ + "Web workspace URL (workspace.json) is required", + ]); + expect(validateWorkspaceConfig(webWorkspace("ftp://x/workspace.json")).valid).to.equal(false); + expect(validateWorkspaceConfig(webWorkspace("not a url")).valid).to.equal(false); + }); + + it("stores a path reference without versions for web workspace queries", () => { + const ref = queryRefFromId("http", "era/op.rq"); + expect(ref).to.deep.equal({ path: "era/op.rq" }); + expect(queryIdFromMetadata({ workspaceId: "web", backendType: "http", queryRef: ref })).to.equal("era/op.rq"); + expect(versionRefFromTag("http", "abc")).to.equal(undefined); + expect(queryRefFromId("sparql", "urn:q")).to.deep.equal({ managedQueryIri: "urn:q" }); + expect(versionRefFromTag("git", "sha")).to.deep.equal({ commitSha: "sha" }); + expect(versionTagFromRef("sparql", { managedQueryVersionIri: "urn:v" })).to.equal("urn:v"); + }); + + describe("workspace links", () => { + it("reads the workspace parameter from the query string or the hash", () => { + expect(getWorkspaceUrlFromLink(`https://matgui.example/?workspace=${encodeURIComponent(MANIFEST_URL)}`)).to.equal( + MANIFEST_URL, + ); + expect( + getWorkspaceUrlFromLink(`https://matgui.example/#query=x&workspace=${encodeURIComponent(MANIFEST_URL)}`), + ).to.equal(MANIFEST_URL); + expect(getWorkspaceUrlFromLink("https://matgui.example/?workspace=javascript:alert(1)")).to.equal(undefined); + expect(getWorkspaceUrlFromLink("https://matgui.example/?workspace=relative/workspace.json")).to.equal(undefined); + expect(getWorkspaceUrlFromLink("https://matgui.example/")).to.equal(undefined); + }); + + it("removes the parameter and builds links", () => { + const link = createWorkspaceLink("https://matgui.example/app/?lang=en#query=x", MANIFEST_URL); + expect(link).to.equal(`https://matgui.example/app/?lang=en&workspace=${encodeURIComponent(MANIFEST_URL)}`); + expect(getWorkspaceUrlFromLink(link)).to.equal(MANIFEST_URL); + expect(removeWorkspaceParamFromLink(link)).to.equal("https://matgui.example/app/?lang=en"); + expect(removeWorkspaceParamFromLink("https://m.example/#workspace=x&query=y")).to.equal( + "https://m.example/#query=y", + ); + }); + + it("finds a configured web workspace by its URL", () => { + const workspaces: WorkspaceConfig[] = [ + { id: "s", label: "S", type: "sparql", endpoint: "https://e/sparql", workspaceIri: "urn:s" }, + webWorkspace(), + ]; + expect(findWebWorkspace(workspaces, MANIFEST_URL)?.id).to.equal("web"); + expect(findWebWorkspace(workspaces, "https://example.org/other/workspace.json")).to.equal(undefined); + }); + }); +}); diff --git a/test/web-workspace-browser.ts b/test/web-workspace-browser.ts new file mode 100644 index 00000000..cbac6270 --- /dev/null +++ b/test/web-workspace-browser.ts @@ -0,0 +1,152 @@ +import * as path from "path"; +import * as http from "http"; +import * as chai from "chai"; +import * as puppeteer from "puppeteer"; +import { it, describe, before, beforeEach, after, afterEach } from "mocha"; +import { setup, destroy, closePage, getPage, TEST_ORIGIN } from "./utils.js"; + +const expect = chai.expect; + +// Served by request interception below, as a static web server on another origin would +const PUBLISHED = "https://queries.test/demo"; +const MANIFEST_URL = `${PUBLISHED}/workspace.json`; +const FILES: { [url: string]: string } = { + [MANIFEST_URL]: JSON.stringify({ + label: "Published queries", + description: "Example queries served as plain files", + endpoints: { demo: `${TEST_ORIGIN}/web-workspace-endpoint` }, + folders: { basics: { label: "Basics", endpoint: "demo" } }, + queries: ["basics/hello.rq", { url: "count.rq", name: "Count everything" }], + }), + [`${PUBLISHED}/basics/hello.rq`]: "#+ name: Hello world\n#+ description: Says hello\nSELECT ('hello' AS ?x) {}\n", + [`${PUBLISHED}/count.rq`]: "SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }\n", +}; + +describe("Web workspaces", function () { + let browser: puppeteer.Browser; + let page: puppeteer.Page; + let server: http.Server | undefined; + + before(async function () { + const refs = await setup(this, path.resolve("./build")); + browser = refs.browser; + server = refs.server; + }); + + beforeEach(async function () { + this.timeout(30000); + page = await getPage(browser, "yasgui.html"); + await page.evaluate(() => localStorage.clear()); + await page.setRequestInterception(true); + page.on("request", (request) => { + const url = request.url(); + if (!url.startsWith(PUBLISHED)) { + void request.continue(); + return; + } + const body = FILES[url]; + void request.respond({ + status: body === undefined ? 404 : 200, + headers: { "Access-Control-Allow-Origin": "*" }, + contentType: url.endsWith(".json") ? "application/json" : "application/sparql-query", + body: body ?? "not found", + }); + }); + }); + + afterEach(async () => { + await closePage(this, page); + }); + + after(async function () { + return destroy(browser, server); + }); + + async function openWithLink() { + await page.goto("about:blank"); + await page.goto( + `${TEST_ORIGIN}/yasgui.html?noExternalServices=true&workspace=${encodeURIComponent(MANIFEST_URL)}`, + { waitUntil: "networkidle2" }, + ); + await page.waitForFunction(() => "yasgui" in window); + } + + function workspaces() { + return page.evaluate(() => (window as any).yasgui.persistentConfig.getWorkspaces()); + } + + it("adds a workspace from a link after the user confirms, and opens its queries", async function () { + await openWithLink(); + await page.waitForSelector(".yasgui-dialog__message"); + const message = await page.$eval(".yasgui-dialog__message", (el) => el.textContent); + expect(message).to.contain('"Published queries" (2 queries)'); + expect(message).to.contain(MANIFEST_URL); + // The parameter is gone, so a reload doesn't ask again + expect(page.url()).to.not.contain("workspace="); + + await page.click(".yasgui-dialog__button--primary"); + await page.waitForFunction(() => (window as any).yasgui.persistentConfig.getWorkspaces().length === 1); + const [workspace] = await workspaces(); + expect(workspace).to.include({ + type: "http", + label: "Published queries", + manifestUrl: MANIFEST_URL, + readOnly: true, + }); + + // The query browser shows it + await page.waitForFunction(() => + Array.from(document.querySelectorAll(".yasgui-query-browser__tree-label")).some( + (el) => el.textContent === "Count everything", + ), + ); + const labels = await page.$$eval(".yasgui-query-browser__tree-label", (els) => els.map((el) => el.textContent)); + expect(labels).to.include.members(["Basics", "Count everything"]); + + // Opening a query: header stripped, folder endpoint applied, read-only + // No async functions inside evaluate: the test build would transpile them into helpers the page lacks + const opened = await page.evaluate((id) => { + const yasgui = (window as any).yasgui; + return yasgui.queryBrowser.openQuery(id, "basics/hello.rq", "Hello world").then(() => { + const tab = yasgui.getTab(); + return { + query: tab.getQuery(), + endpoint: tab.getEndpoint(), + name: tab.getName(), + meta: tab.getManagedQueryMetadata(), + crumbs: tab.getBreadcrumbs(), + }; + }); + }, workspace.id); + expect(opened.query).to.equal("SELECT ('hello' AS ?x) {}\n"); + expect(opened.endpoint).to.equal(`${TEST_ORIGIN}/web-workspace-endpoint`); + expect(opened.name).to.equal("Hello world"); + expect(opened.meta).to.deep.include({ backendType: "http", queryRef: { path: "basics/hello.rq" } }); + expect(opened.crumbs).to.deep.equal(["Published queries", "basics"]); + + // Opening the same link again shows the workspace without asking + await openWithLink(); + await new Promise((resolve) => setTimeout(resolve, 800)); + expect(await page.$(".yasgui-dialog__message")).to.equal(null); + expect(await workspaces()).to.have.length(1); + }); + + it("adds nothing when the user declines or the workspace cannot be loaded", async function () { + await openWithLink(); + await page.waitForSelector(".yasgui-dialog__message"); + await page.click(".yasgui-dialog__button--secondary"); + await new Promise((resolve) => setTimeout(resolve, 200)); + expect(await workspaces()).to.have.length(0); + + await page.evaluate((url) => { + (window as any).addResult = (window as any).yasgui.addWebWorkspace(url); + }, `${PUBLISHED}/missing/workspace.json`); + await page.waitForSelector(".yasgui-dialog__message"); + const message = await page.$eval(".yasgui-dialog__message", (el) => el.textContent); + await page.click(".yasgui-dialog__button--primary"); + const result = { message, id: await page.evaluate(() => (window as any).addResult) }; + expect(result.message).to.contain("could not be loaded"); + expect(result.id).to.equal(undefined); + expect(await workspaces()).to.have.length(0); + }); +});