From 60aea080c281a985e64760376dcad24e4541fbe9 Mon Sep 17 00:00:00 2001 From: "Eric Chen (from Dev Box)" Date: Thu, 14 May 2026 01:45:01 -0500 Subject: [PATCH 01/19] Article-mode oEmbed extraction for video pages + V1 payload parity V1's article-mode flow on video pages (YouTube, Vimeo) produced a save payload with an embedded video iframe, citation, title/author caption, and `` / `` tags that OneNote's page renderer uses to recognize the result as an article-style clip with playable embeds. V2 shipped without any of that machinery -- the article mode just ran Readability over the YouTube DOM, which strips iframes and produces a text-only result with no player and no description. Users reported the regression on YouTube specifically; the same gap applied to Vimeo as well. oEmbed standard provides exactly the shape we need (iframe `html`, title, author_name, thumbnail_url, dimensions) without any provider-specific scraping. Both YouTube and Vimeo publish CORS-enabled oEmbed endpoints that the chrome-extension origin can fetch directly under our existing `` host_permissions. Changes: - New `src/scripts/contentCapture/oembedExtractor.ts` -- thin module with a provider table (YouTube + Vimeo only, matching V1's SupportedVideoDomains), hostname-pattern matching, fetch + JSON parse, and a small `sanitizeProviderHtml` helper that strips script-execution surfaces from provider-supplied HTML. - `extractArticle` in renderer now tries oEmbed first; on no-match or fetch failure it falls through to the existing Readability path with zero behavior change. - Preview vs save are decoupled: - Preview shows the `thumbnail_url` at the same 600x338 (16:9) box the saved iframe uses, with title / "author . provider" attribution, page description (og:description fallback chain same as bookmark mode), and a CSS-only play-glyph overlay when `type === "video"`. No iframe in preview because the renderer's `preview-frame` is sandboxed (allow-same-origin) and the YouTube/Vimeo player can't run JS inside it -- which is why earlier attempts produced a broken "Unable to execute JavaScript" placeholder. - Save uses the provider's iframe HTML (sanitized), with `data-original-src=` injected and dimensions normalized to 600x338 -- the marker OneNote's renderer uses to recognize and render the embedded player on the saved page, matching V1's YoutubeVideoExtractor behavior exactly. - PageMetadata plumbing: renderer threads a `pageMetadata` map through the save port message; worker's `buildPage` iterates and emits `` for each entry. Mirrors V1's `OneNoteApi.OneNotePage.getPageMetadataAsHtml` behavior. Article mode (both oEmbed and Readability paths) populates `AutoPageTagsCodes=Article`, `AutoPageTags=Article`, plus title/author/siteName (oEmbed) or title/description/author/siteName/publishedTime (Readability, matching V1 augmentationHelper). - `buildPage` HTML output realigned to V1 `OneNoteApi.OneNotePage.getEntireOnml` shape: no ``, `>` (no quotes around lang -- matches V1 output literally), locale via `chrome.i18n.getUILanguage()`. Same change applied to the parallel `distHtml` builder for distributed-PDF saves so all save paths emit the same shape. - Bookmark thumbnail size fallback restored: `imageToDataUrl` initial-encode is PNG (good for icons/logos), with iterative JPEG-quality step-down when the encoded data URL exceeds the OneNote API per-MIME-part limit (~2MB minus padding). Matches V1's deleted `DomUtils.adjustImageQualityIfNecessary` behavior including the 0.1 step size. Surfaced because the user was hitting "400 Maximum request size exceeded" on bookmark-mode saves of YouTube pages whose 1280x720 og:image PNG-encoded to ~2.5MB. Provider scope is intentionally narrow (YouTube + Vimeo only) to match V1's effective surface and avoid accidentally enabling capture on sites V1 never supported. V1 also handled Khan Academy via regex scrape for embedded YouTube IDs in lesson-page HTML; that markup likely no longer matches modern Khan Academy pages and is skipped here per maintainer direction. Verified manually: YouTube watch page and Vimeo video page produce saved OneNote pages with the embedded player, title/author caption, and og:description text below; non-matching domains fall through to Readability with no regression; bookmark mode on YouTube saves successfully without the 400 limit error. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/scripts/contentCapture/oembedExtractor.ts | 145 ++++++++++++ .../webExtensionBase/webExtensionWorker.ts | 28 ++- src/scripts/renderer.ts | 224 +++++++++++++++++- 3 files changed, 380 insertions(+), 17 deletions(-) create mode 100644 src/scripts/contentCapture/oembedExtractor.ts diff --git a/src/scripts/contentCapture/oembedExtractor.ts b/src/scripts/contentCapture/oembedExtractor.ts new file mode 100644 index 00000000..5c49655d --- /dev/null +++ b/src/scripts/contentCapture/oembedExtractor.ts @@ -0,0 +1,145 @@ +/** + * oEmbed-based article extraction for rich-media pages (video, slideshare, + * soundcloud, etc.). When a page URL matches a known oEmbed provider, fetch + * the provider's structured embed payload. Returns the raw response data so + * the renderer can compose distinct HTML for preview (clean static thumbnail) + * and save (iframe embed picked up by OneNote's page renderer). + * + * Provider list mirrors the canonical OneNote-supported set. Each entry is + * { name, endpoint, hostPattern } where hostPattern is either a bare + * hostname (matched as suffix), a "host/path" prefix, or a partial hostname + * ending in "." (matched as prefix). + * + * Returns null on no-match or fetch failure; callers should fall back to + * Readability. + */ + +interface OEmbedProvider { + name: string; + endpoint: string; + hostPattern: string; +} + +export interface OEmbedData { + type: string; // "video" | "photo" | "link" | "rich" + html?: string; // present for video / rich + url?: string; // present for photo + width?: number; + height?: number; + title?: string; + author_name?: string; + thumbnail_url?: string; + provider_name?: string; + pageUrl: string; // echo of the page URL we matched against +} + +// Provider set matches V1's video extractor support (YouTube + Vimeo). +// V1 also had KhanAcademy in its SupportedVideoDomains, but Khan Academy +// doesn't publish an oEmbed endpoint -- their V1 extractor was just +// scanning Khan Academy pages for embedded YouTube iframes, which our +// YouTube provider already covers when those iframes are present. +const PROVIDERS: OEmbedProvider[] = [ + { name: "YouTube", endpoint: "https://www.youtube.com/oembed", hostPattern: "youtube.com" }, + { name: "YouTube", endpoint: "https://www.youtube.com/oembed", hostPattern: "youtu.be" }, + { name: "Vimeo", endpoint: "https://vimeo.com/api/oembed.json", hostPattern: "vimeo.com" }, +]; + +function matchProvider(url: string): OEmbedProvider | null { + let parsed: URL; + try { + parsed = new URL(url); + } catch (e) { + return null; + } + const host = parsed.hostname.toLowerCase(); + const hostAndPath = (host + parsed.pathname).toLowerCase(); + + for (const provider of PROVIDERS) { + const pattern = provider.hostPattern.toLowerCase(); + + if (pattern.indexOf("/") !== -1) { + if (hostAndPath === pattern + || hostAndPath.indexOf(pattern) === 0 + || hostAndPath.indexOf("." + pattern) !== -1) { + return provider; + } + } else if (pattern.charAt(pattern.length - 1) === ".") { + if (host.indexOf(pattern) === 0) { + return provider; + } + } else { + if (host === pattern || host.indexOf("." + pattern) === host.length - pattern.length - 1) { + return provider; + } + } + } + return null; +} + +/** + * Strip executable surfaces from provider-supplied HTML while preserving the + * iframes/anchors/images that carry the actual embed. Belt-and-suspenders: + * the renderer's preview iframe is sandboxed (allow-same-origin), and + * OneNote sanitizes server-side on save. + */ +export function sanitizeProviderHtml(html: string): string { + const doc = new DOMParser().parseFromString(html, "text/html"); + + const removable = doc.querySelectorAll("script, object, embed, link, style, meta"); + for (let i = removable.length - 1; i >= 0; i--) { + const el = removable[i]; + if (el.parentNode) { el.parentNode.removeChild(el); } + } + + const all = doc.querySelectorAll("*"); + for (let i = 0; i < all.length; i++) { + const el = all[i] as HTMLElement; + const attrs = el.attributes; + for (let j = attrs.length - 1; j >= 0; j--) { + const name = attrs[j].name.toLowerCase(); + const value = attrs[j].value; + if (name.indexOf("on") === 0) { + el.removeAttribute(attrs[j].name); + } else if ((name === "href" || name === "src") && /^\s*javascript:/i.test(value)) { + el.removeAttribute(attrs[j].name); + } + } + } + + return doc.body ? doc.body.innerHTML : ""; +} + +/** + * Entry point. Returns raw oEmbed response data on success, null on + * no-match or any failure (caller should fall back to Readability). + */ +export async function tryOEmbed(pageUrl: string): Promise { + if (!pageUrl) { return null; } + + const provider = matchProvider(pageUrl); + if (!provider) { return null; } + + const endpoint = provider.endpoint + + "?url=" + encodeURIComponent(pageUrl) + + "&format=json&maxwidth=600"; + + try { + const resp = await fetch(endpoint); + if (!resp.ok) { return null; } + const data = await resp.json() as Partial; + // Only video / rich / photo types produce embeddable content. + // "link" type carries metadata only; let Readability handle the page text. + if (data.type !== "video" && data.type !== "rich" && data.type !== "photo") { + return null; + } + // Ensure provider_name is set even when the response omits it -- some + // providers leave it blank but our match guarantees we know who it is. + if (!data.provider_name) { + data.provider_name = provider.name; + } + data.pageUrl = pageUrl; + return data as OEmbedData; + } catch (e) { + return null; + } +} diff --git a/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts b/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts index ef11141f..64d437ea 100644 --- a/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts +++ b/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts @@ -679,6 +679,7 @@ export class WebExtensionWorker extends ExtensionWorkerBase { let saveAnnotation = msg.annotation || ""; let saveSectionId = msg.sectionId || ""; let saveUrl = msg.url || ""; + let savePageMetadata: { [key: string]: string } | undefined = msg.pageMetadata; // Ensure fresh token before save (matches old clipper.tsx ensureFreshUserBeforeClip) workerSelf.auth.updateUserInfoData(workerSelf.clientInfo.get().clipperId, UpdateReason.TokenRefreshForPendingClip).then(() => { @@ -697,7 +698,10 @@ export class WebExtensionWorker extends ExtensionWorkerBase { return; } - // Build OneNote page content based on mode + // Build OneNote page content based on mode. Output shape mirrors V1 + // OneNotePage.getEntireOnml: `` (no DOCTYPE, no + // quotes around lang), `` with title + created meta + one + // `` per PageMetadata entry. let buildPage = (bodyOnml: string, imageParts: { name: string; blob: Blob; type: string }[]) => { let boundary = "OneNoteRendererBoundary" + Date.now(); let now = new Date(); @@ -710,9 +714,21 @@ export class WebExtensionWorker extends ExtensionWorkerBase { if (parseInt(offsetMins, 10) < 10) { offsetMins = "0" + offsetMins; } let createdTime = offsetSign + offsetHours + ":" + offsetMins; let fontStyle = "font-size: 16px; font-family: Verdana;"; - let presentationHtml = "" - + "" + saveTitle.replace(/</g, "<").replace(/>/g, ">") + "" + let locale = (typeof chrome !== "undefined" && chrome.i18n && chrome.i18n.getUILanguage) ? chrome.i18n.getUILanguage() : "en"; + let metaTags = ""; + if (savePageMetadata) { + for (let key in savePageMetadata) { + if (Object.prototype.hasOwnProperty.call(savePageMetadata, key)) { + metaTags += ""; + } + } + } + let presentationHtml = "" + + "" + + "" + escapeHtml(saveTitle) + "" + "" + + metaTags + ""; if (saveAnnotation) { let escaped = saveAnnotation.replace(/&/g, "&").replace(//g, ">"); @@ -861,8 +877,10 @@ export class WebExtensionWorker extends ExtensionWorkerBase { if (parseInt(oM, 10) < 10) { oM = "0" + oM; } let ct = offsetSign2 + oH + ":" + oM; let fStyle = "font-size: 16px; font-family: Verdana;"; - let distHtml = "" - + "" + pageTitle.replace(/</g, "<").replace(/>/g, ">") + "" + let distLocale = (typeof chrome !== "undefined" && chrome.i18n && chrome.i18n.getUILanguage) ? chrome.i18n.getUILanguage() : "en"; + let distHtml = "" + + "" + + "" + escapeHtml(pageTitle) + "" + "" + ""; if (pageIdx === 0 && saveAnnotation) { diff --git a/src/scripts/renderer.ts b/src/scripts/renderer.ts index 0c932142..c935667e 100644 --- a/src/scripts/renderer.ts +++ b/src/scripts/renderer.ts @@ -7,6 +7,8 @@ import {PropertyName} from "./logging/submodules/propertyName"; import {Session} from "./logging/submodules/session"; import {Status} from "./logging/submodules/status"; +import {tryOEmbed, sanitizeProviderHtml, OEmbedData} from "./contentCapture/oembedExtractor"; + // Renderer page script - connects to service worker via port // and handles scroll/capture commands. Content HTML arrives inline // on the loadContent port message; images sent to worker via chunked @@ -134,7 +136,10 @@ let fullPageComplete = false; let fullPageDataUrl = ""; // Cached full-page screenshot data URL for mode switching let saveDone = false; let articleLoaded = false; -let cachedArticleHtml = ""; // Readability result, extracted lazily from content-frame DOM +let cachedArticleHtml = ""; // Article preview HTML (Readability or oEmbed-preview shape) +let cachedOEmbedData: OEmbedData | null = null; // Raw oEmbed payload; if present, save path builds iframe-based HTML from this instead of cachedArticleHtml +let cachedOEmbedDescription = ""; // og:description (or fallback chain) from page DOM; oEmbed responses typically don't carry the full description +let cachedPageMetadata: { [key: string]: string } | null = null; // PageMetadata sent in save msg; worker emits each entry as a tag (matches V1 OneNotePage) let cachedBookmarkHtml = ""; // Bookmark card HTML, extracted lazily from content-frame DOM let bookmarkLoaded = false; let contentDocReady = false; // true once content-frame has loaded HTML @@ -1053,6 +1058,170 @@ function loadArticleContent() { } function extractArticle() { + // For known oEmbed providers (YouTube, Vimeo, Slideshare, etc.), prefer the + // provider's structured embed payload over Readability's text extraction -- + // Readability strips iframes, so video pages would otherwise lose the player. + // Falls back to Readability on no-match or fetch failure. + let pageUrl = sourceUrlText.textContent || ""; + tryOEmbed(pageUrl).then(function(data) { + if (data) { + cachedOEmbedData = data; + // Description from page DOM -- oEmbed responses typically don't include + // the long description (YouTube's e.g. carries only title/author). Same + // og:description / description / twitter:description fallback chain + // bookmark mode uses. + let iframeDoc = iframe.contentDocument; + let description = ""; + if (iframeDoc) { + description = getMetaContent(iframeDoc, "og:description", "property") + || getMetaContent(iframeDoc, "description", "name") + || getMetaContent(iframeDoc, "twitter:description", "name") + || ""; + } + cachedOEmbedDescription = description; + cachedArticleHtml = composeOEmbedForPreview(data, description); + cachedPageMetadata = buildPageMetadataForOEmbed(data, description); + renderArticleHtml(cachedArticleHtml); + articleLoaded = true; + if (currentMode === "article") { + saveBtn.disabled = false; + saveBtn.textContent = strings.saveToOneNote; + } + return; + } + extractArticleViaReadability(); + }); +} + +// Preview rendering for oEmbed responses: a static thumbnail with title / +// author / provider attribution and the page description. We avoid +// rendering the provider's iframe here because the sandboxed preview-frame +// blocks the embedded player's JS, which would surface a broken "Unable +// to execute JavaScript" UI. The iframe still flows through to save -- +// OneNote isn't sandboxed. +function composeOEmbedForPreview(data: OEmbedData, pageDescription: string): string { + let html = "
"; + if (data.thumbnail_url) { + // For video/rich types the save path emits a 600x338 (16:9) iframe; + // lock the preview thumbnail to the same frame so the visual size + // matches what the user will see on the saved OneNote page. For + // photo type the photo itself is the content, so use natural aspect. + let isFramed = data.type === "video" || data.type === "rich"; + let containerStyle = isFramed + ? "position:relative; display:block; width:100%; max-width:600px; aspect-ratio:600/338; background:#000;" + : "position:relative; display:inline-block; max-width:100%;"; + let imgStyle = isFramed + ? "display:block; width:100%; height:100%; object-fit:cover;" + : "display:block; max-width:600px; width:100%; height:auto;"; + html += "
"; + html += "\"""; + if (data.type === "video") { + // Play-glyph overlay -- pure CSS, no text, so no i18n surface + html += "
▶
"; + } + html += "
"; + } + if (data.title) { + html += "

" + escapeHtml(data.title) + "

"; + } + let metaLine: string[] = []; + if (data.author_name) { metaLine.push(escapeHtml(data.author_name)); } + if (data.provider_name) { metaLine.push(escapeHtml(data.provider_name)); } + if (metaLine.length) { + html += "
" + metaLine.join(" · ") + "
"; + } + if (pageDescription) { + html += "
" + + escapeHtml(pageDescription) + "
"; + } + html += "
"; + return html; +} + +// Save-side composition: emit the actual provider iframe (sanitized) so +// OneNote's page renderer recognizes and embeds the video player. Title, +// author and description render below the iframe as a simple caption. +// Adds `data-original-src` to every iframe (the marker OneNote's renderer +// uses to recognize video embeds, matching V1's YouTubeVideoExtractor +// behavior) and normalizes iframe dimensions to V1's 600x338 for visual +// parity with the legacy clipper output. +function composeOEmbedForSave(data: OEmbedData, pageDescription: string): string { + let body = ""; + if (data.type === "photo" && data.url) { + body = ""; + } else if (data.html) { + body = normalizeProviderIframe(sanitizeProviderHtml(data.html), data.pageUrl); + } else { + return ""; + } + let caption = ""; + if (data.title) { caption += "

" + escapeHtml(data.title) + "

"; } + if (data.author_name) { caption += "
" + escapeHtml(data.author_name) + "
"; } + if (pageDescription) { caption += "
" + escapeHtml(pageDescription) + "
"; } + return "
" + + body + + (caption ? "
" + caption : "") + + "
"; +} + +function normalizeProviderIframe(html: string, pageUrl: string): string { + let parsed = new DOMParser().parseFromString(html, "text/html"); + let iframes = parsed.getElementsByTagName("iframe"); + for (let i = 0; i < iframes.length; i++) { + // data-original-src is the marker OneNote's renderer looks for to + // recognize and render a video embed on the saved page. + iframes[i].setAttribute("data-original-src", pageUrl); + // Match V1 dimensions for consistent presentation in OneNote. + iframes[i].setAttribute("width", "600"); + iframes[i].setAttribute("height", "338"); + } + return parsed.body ? parsed.body.innerHTML : html; +} + +// PageMetadata for oEmbed-extracted pages -- mirrors V1 server-side +// `PageMetadata.AutoPageTags*` plus oEmbed-sourced descriptive fields. +// Worker iterates the map and emits one per entry (V1 OneNotePage +// behavior). +function buildPageMetadataForOEmbed(data: OEmbedData, pageDescription: string): { [key: string]: string } { + let meta: { [key: string]: string } = { + AutoPageTagsCodes: "Article", + AutoPageTags: "Article" + }; + if (data.title) { meta.title = data.title; } + if (data.author_name) { meta.author = data.author_name; } + if (data.provider_name) { meta.siteName = data.provider_name; } + if (pageDescription) { meta.description = pageDescription; } + return meta; +} + +// PageMetadata for Readability-extracted articles -- matches V1 +// augmentationHelper's local metadata population (title/excerpt/byline/ +// siteName/publishedTime) plus the AutoPageTags markers. +function buildPageMetadataForReadability(article: any): { [key: string]: string } { + let meta: { [key: string]: string } = { + AutoPageTagsCodes: "Article", + AutoPageTags: "Article" + }; + if (article.title) { meta.title = article.title; } + if (article.excerpt) { meta.description = article.excerpt; } + if (article.byline) { meta.author = article.byline; } + if (article.siteName) { meta.siteName = article.siteName; } + if (article.publishedTime) { meta.publishedTime = article.publishedTime; } + return meta; +} + +function extractArticleViaReadability() { // Clone content-frame document — Readability mutates the DOM let iframeDoc = iframe.contentDocument; if (!iframeDoc || !iframeDoc.body) { @@ -1067,6 +1236,8 @@ function extractArticle() { if (article && article.content) { cachedArticleHtml = cleanArticleHtml(article.content); + cachedOEmbedData = null; + cachedPageMetadata = buildPageMetadataForReadability(article); renderArticleHtml(cachedArticleHtml); articleLoaded = true; if (currentMode === "article") { @@ -1477,7 +1648,13 @@ function extractBookmark() { } // Fetch an image URL and convert to base64 data URL via canvas -// (OneNote API can't fetch external URLs — matches legacy DomUtils.getImageDataUrl) +// (OneNote API can't fetch external URLs — matches legacy DomUtils.getImageDataUrl). +// Initial encode is PNG (lossless, ideal for icons/logos). For oversized +// photos (e.g. YouTube's 1280x720 og:image at maxresdefault.jpg) PNG can +// exceed the OneNote API per-MIME-part limit and the save POST returns 400 +// "Maximum request size exceeded" -- fall back to JPEG and step quality +// down until the encoded size fits. Matches legacy +// DomUtils.adjustImageQualityIfNecessary behavior. function imageToDataUrl(url: string, callback: (dataUrl: string) => void) { let img = new Image(); img.crossOrigin = "anonymous"; @@ -1487,7 +1664,8 @@ function imageToDataUrl(url: string, callback: (dataUrl: string) => void) { canvas.width = img.naturalWidth; canvas.height = img.naturalHeight; (canvas.getContext("2d") as CanvasRenderingContext2D).drawImage(img, 0, 0); - callback(canvas.toDataURL("image/png")); + let dataUrl = canvas.toDataURL("image/png"); + callback(adjustImageQualityIfNecessary(canvas, dataUrl)); } catch (e) { callback(""); // tainted canvas or other error — fall back } @@ -1496,6 +1674,20 @@ function imageToDataUrl(url: string, callback: (dataUrl: string) => void) { img.src = url; } +// OneNote API per-MIME-part limit (matches legacy +// Settings.Instance.Apis_MediaTypesHandledInMemoryMaxRequestLength) minus a +// small padding for the request envelope. +const MAX_BYTES_FOR_MEDIA_TYPES = 2097152 - 500; + +function adjustImageQualityIfNecessary(canvas: HTMLCanvasElement, dataUrl: string): string { + let quality = 1.0; + while (quality > 0 && dataUrl.length > MAX_BYTES_FOR_MEDIA_TYPES) { + dataUrl = canvas.toDataURL("image/jpeg", quality); + quality -= 0.1; + } + return dataUrl; +} + function getMetaContent(doc: Document, value: string, attr: string): string { let el = doc.querySelector("meta[" + attr + "=\"" + value + "\"]"); return el ? el.getAttribute("content") || "" : ""; @@ -3032,21 +3224,29 @@ saveBtn.addEventListener("click", () => { safeSend({ action: "saveImage", index: i, dataUrl: regionImages[i] }); } } else if (currentMode === "article") { - let pDoc = previewFrame.contentDocument; let articleBody = ""; - if (pDoc && pDoc.body && pDoc.body.querySelector(".highlighted")) { - let clone = pDoc.body.cloneNode(true) as HTMLElement; - let delBtns = clone.querySelectorAll(".delete-highlight"); - for (let i = delBtns.length - 1; i >= 0; i--) { - if (delBtns[i].parentNode) { delBtns[i].parentNode.removeChild(delBtns[i]); } - } - articleBody = clone.innerHTML; + let oembedSnap = cachedOEmbedData; + if (oembedSnap) { + // oEmbed-source: save uses provider iframe (preview only showed + // thumbnail since sandboxed iframes can't run the player). + articleBody = composeOEmbedForSave(oembedSnap, cachedOEmbedDescription); } else { - articleBody = cachedArticleHtml; + let pDoc = previewFrame.contentDocument; + if (pDoc && pDoc.body && pDoc.body.querySelector(".highlighted")) { + let clone = pDoc.body.cloneNode(true) as HTMLElement; + let delBtns = clone.querySelectorAll(".delete-highlight"); + for (let i = delBtns.length - 1; i >= 0; i--) { + if (delBtns[i].parentNode) { delBtns[i].parentNode.removeChild(delBtns[i]); } + } + articleBody = clone.innerHTML; + } else { + articleBody = cachedArticleHtml; + } } let fontFamily = articleSerif ? strings.fontFamilySerif : strings.fontFamilySansSerif; let fontStyle = "font-size: " + articleFontSize + "px; font-family: " + fontFamily + ";"; saveMsg.contentHtml = "
" + articleBody + "
"; + if (cachedPageMetadata) { saveMsg.pageMetadata = cachedPageMetadata; } saveMsg.saveImageCount = 0; safeSend(saveMsg); } else if (currentMode === "bookmark") { From 9bc0549c4e3ad562e3ad0c4d140d0fa223df7d10 Mon Sep 17 00:00:00 2001 From: "Eric Chen (from Dev Box)" Date: Thu, 14 May 2026 03:54:15 -0500 Subject: [PATCH 02/19] Fix article-mode highlighter under sandboxed preview iframe The article-mode highlighter has been non-functional since commit 16f62c0 (Security hardening) added `sandbox="allow-same-origin"` to the preview-frame iframe. The sandbox blocks script execution inside the iframe document, so `setupHighlighter`'s runtime injection of ` + diff --git a/src/scripts/highlighting/textHighlighter.js b/src/scripts/highlighting/textHighlighter.js index e3b8e393..4fb14f62 100644 --- a/src/scripts/highlighting/textHighlighter.js +++ b/src/scripts/highlighting/textHighlighter.js @@ -504,21 +504,28 @@ THE SOFTWARE. }; TextHighlighter.prototype.bindEvents = function (el) { + // Bind selection-completion events to el's own window so this library works + // when constructed from a parent window operating on a same-origin sandboxed + // iframe (where bare `window` would resolve to the parent and miss events + // that fire inside the iframe). All other DOM access already routes through + // dom(el).get{Window,Document,Selection}(), this is the only outlier. + var win = el.ownerDocument.defaultView; el.addEventListener('keydown', this.boundStartDragHandler); el.addEventListener('mousedown', this.boundStartDragHandler); el.addEventListener('touchstart', this.boundStartDragHandler); - window.addEventListener('keyup', this.boundHighlightHandler); - window.addEventListener('mouseup', this.boundHighlightHandler); - window.addEventListener('touchend', this.boundHighlightHandler); + win.addEventListener('keyup', this.boundHighlightHandler); + win.addEventListener('mouseup', this.boundHighlightHandler); + win.addEventListener('touchend', this.boundHighlightHandler); } TextHighlighter.prototype.unbindEvents = function (el) { + var win = el.ownerDocument.defaultView; el.removeEventListener('keydown', this.boundStartDragHandler); el.removeEventListener('mousedown', this.boundStartDragHandler); el.removeEventListener('touchstart', this.boundStartDragHandler); - window.removeEventListener('keyup', this.boundHighlightHandler); - window.removeEventListener('mouseup', this.boundHighlightHandler); - window.removeEventListener('touchend', this.boundHighlightHandler); + win.removeEventListener('keyup', this.boundHighlightHandler); + win.removeEventListener('mouseup', this.boundHighlightHandler); + win.removeEventListener('touchend', this.boundHighlightHandler); } /** diff --git a/src/scripts/renderer.ts b/src/scripts/renderer.ts index c935667e..f453fa5e 100644 --- a/src/scripts/renderer.ts +++ b/src/scripts/renderer.ts @@ -1351,28 +1351,25 @@ function applyArticleFontSize() { } function initHighlighter() { - let pDoc = previewFrame.contentDocument; - let pWin = previewFrame.contentWindow as any; - if (!pDoc || !pDoc.body || !pWin) { return; } - - // Inject TextHighlighter script if not already present - if (!pWin.TextHighlighter) { - let script = pDoc.createElement("script"); - script.src = "textHighlighter.js"; - script.onload = function() { - createHighlighterInstance(); - }; - pDoc.head.appendChild(script); - } else { - createHighlighterInstance(); - } + // TextHighlighter loads as a regular parent-window script (renderer.html), + // not injected into preview-frame -- the iframe is sandboxed with + // allow-same-origin only (no allow-scripts), so script execution inside + // it is blocked. Same-origin permits the parent's TextHighlighter to + // operate directly on the iframe's body/selection/events, which is + // how pdf.js also runs in this codebase (parent context, manipulating + // child DOM). The library is constructed against `pDoc.body` and uses + // `el.ownerDocument.defaultView` internally for selection-completion + // event binding (small patch applied to the vendored textHighlighter.js + // to route those bindings through the el-relative window helper instead + // of bare `window`). + createHighlighterInstance(); } function createHighlighterInstance() { - let pWin = previewFrame.contentWindow as any; let pDoc = previewFrame.contentDocument; - if (!pWin || !pWin.TextHighlighter || !pDoc || !pDoc.body) { return; } - textHighlighterInstance = new pWin.TextHighlighter(pDoc.body, { + let highlighterCtor = (window as any).TextHighlighter; + if (!highlighterCtor || !pDoc || !pDoc.body) { return; } + textHighlighterInstance = new highlighterCtor(pDoc.body, { color: "#fefe56", highlightedClass: "highlighted", enabled: true, From ca6ac23a07ec1d0b3509af6b2b1a20a9f12b332a Mon Sep 17 00:00:00 2001 From: "Eric Chen (from Dev Box)" Date: Thu, 14 May 2026 04:26:21 -0500 Subject: [PATCH 03/19] Roomier sidebar group spacing; bump window height cap to fit Per designer: more breathing room around the 1px Fluent dividers between sidebar groups (mode buttons, meta fields, section picker). Bumped `.sidebar-group` padding from 8px/8px to 12px/12px so each gap goes from 8+8=16px around the divider to 12+12=24px. Designer asked for 16/16 originally; 12/12 was the landing point after checking PDF mode -- 16/16 pushed the "View in OneNote" success banner against the window bottom and required scrolling. 12/12 is still a clear improvement over 8/8 while leaving the post-clip button in view. Bumped the window-height cap from 900px to 980px to absorb the +24px sidebar growth and keep the PDF-mode success banner comfortably visible. Floor of 600px unchanged; smaller browsers still shrink to fit. 980 stays well clear of typical 1080p displays. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../extensions/webExtensionBase/webExtensionWorker.ts | 8 ++++++-- src/styles/renderer.less | 8 ++++++-- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts b/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts index 64d437ea..f358906e 100644 --- a/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts +++ b/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts @@ -437,10 +437,14 @@ export class WebExtensionWorker extends ExtensionWorkerBase { contentWidth = renderWidth - sidebarWidth; } - // Height: cap at 900px so the popup doesn't fill near-full-height on + // Height: cap at 980px so the popup doesn't fill near-full-height on // 1080p+ monitors. Floor at 600 so capture progress UI fits comfortably. // On smaller browsers we still shrink to fit (browserHeight - margin). - let renderHeight = Math.max(Math.min(browserHeight - screenMargin, 900), 600); + // 980 accommodates the 12px-above/12px-below divider spacing on the + // sidebar groups (+24px vs the original 8/8 spacing) and keeps the + // "View in OneNote" success-banner button in view in PDF mode without + // requiring the user to scroll. + let renderHeight = Math.max(Math.min(browserHeight - screenMargin, 980), 600); // Position: align with browser window's top-left let renderLeft = browserLeft; diff --git a/src/styles/renderer.less b/src/styles/renderer.less index 188f84e7..429e38da 100644 --- a/src/styles/renderer.less +++ b/src/styles/renderer.less @@ -240,9 +240,13 @@ iframe#preview-frame { } // Sidebar groups — Fluent design uses 1px neutral dividers between sections. -// 8px padding above + 8px padding below the divider matches Figma `py-[8px]`. +// 12px padding above + 12px padding below the divider. Designer asked for +// 16/16 but that crowded the PDF-mode success banner against the window +// bottom even with the bumped height cap; 12/12 keeps the spacing +// generous vs the original 8/8 while leaving room to land the +// "View in OneNote" button without scrolling. .sidebar-group { - padding: 8px 0; + padding: 12px 0; &:first-child { padding-top: 4px; } & + .sidebar-group { border-top: 1px solid @colorNeutralStroke2; From a29292648b75ca68da22dc17702419fb7134965a Mon Sep 17 00:00:00 2001 From: "Eric Chen (from Dev Box)" Date: Thu, 14 May 2026 04:55:09 -0500 Subject: [PATCH 04/19] Revert window-height cap bump (keep sidebar spacing change) Owning team prefers to keep the renderer-window height cap at the original 900px and let the UX consequence of the roomier divider spacing surface (PDF-mode "View in OneNote" success-banner button sits below the fold and requires scrolling on caps-bound displays). Sidebar group padding stays at 12px/12px from the prior commit so the design ask is still in place; only the height-cap accommodation is reverted. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../extensions/webExtensionBase/webExtensionWorker.ts | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts b/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts index f358906e..64d437ea 100644 --- a/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts +++ b/src/scripts/extensions/webExtensionBase/webExtensionWorker.ts @@ -437,14 +437,10 @@ export class WebExtensionWorker extends ExtensionWorkerBase { contentWidth = renderWidth - sidebarWidth; } - // Height: cap at 980px so the popup doesn't fill near-full-height on + // Height: cap at 900px so the popup doesn't fill near-full-height on // 1080p+ monitors. Floor at 600 so capture progress UI fits comfortably. // On smaller browsers we still shrink to fit (browserHeight - margin). - // 980 accommodates the 12px-above/12px-below divider spacing on the - // sidebar groups (+24px vs the original 8/8 spacing) and keeps the - // "View in OneNote" success-banner button in view in PDF mode without - // requiring the user to scroll. - let renderHeight = Math.max(Math.min(browserHeight - screenMargin, 980), 600); + let renderHeight = Math.max(Math.min(browserHeight - screenMargin, 900), 600); // Position: align with browser window's top-left let renderLeft = browserLeft; From fe40b3de7bbd21b91d55fce76ae5aaa1a98770e0 Mon Sep 17 00:00:00 2001 From: "Eric Chen (from Dev Box)" Date: Thu, 14 May 2026 05:36:20 -0500 Subject: [PATCH 05/19] Restore Segoe UI @font-face for Mac/Linux users V1 had an @font-face declaration that loaded Segoe UI from OneNote's own CDN, explicitly so Apple/Android users without local Segoe UI would still render the intended Fluent typography. V2 dropped that declaration and never replaced it -- Mac users have been falling through the font-family stack to -apple-system (San Francisco) since V2 launch, which renders noticeably differently from the intended look. Designer flagged it. Re-added the declaration verbatim from V1 (same www.onenote.com/styles/segoeui.{eot,woff} URLs V1 used; verified currently serving 200). Local() cascade hits first on Windows (no network), remote fallback only on machines that don't have Segoe UI installed. V1 also declared @font-face for `Segoe UI Light`; V2's CSS never references the Light variant -- skipped here as dead code. V1 referenced `Segoe UI Semibold` as a font-family value without ever declaring an @font-face for it; V2 instead uses `font-weight: 600` on the Regular face throughout, which the browser synthesizes (matches V1's effective behavior on Mac since Mac doesn't have Segoe UI Semibold locally either). No manifest CSP change: the prior CSP omitted font-src entirely, so font loading was already unrestricted -- the @font-face just works under existing policy. Article-preview pages with their own @font-face declarations (e.g. FontAwesome on a Gentoo page) continue to render their icon fonts in the sandboxed preview iframe as before. Verified on Windows: with local() in place, no network request (uses installed Segoe UI). Stripping local() temporarily forced the remote fetch path; renderer DevTools' "Rendered Font: Network resource" confirmed the CDN-served woff was used. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/styles/renderer.less | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/styles/renderer.less b/src/styles/renderer.less index 429e38da..8554a4d9 100644 --- a/src/styles/renderer.less +++ b/src/styles/renderer.less @@ -1,6 +1,24 @@ // Renderer window styles — sidebar panel for capture progress and preview // Color values follow Fluent 2 design tokens (white theme). +// CSS declaration for (Apple & Android) users who don't have Segoe UI +// installed locally. Without this, Mac/Linux users fall through the +// font-family stack to -apple-system (San Francisco) which renders +// noticeably different from the intended Fluent Segoe UI typography. +// Loaded from OneNote's own CDN, same source V1 used. +// Semibold rendering uses `font-weight: 600` on this Regular face; +// browser synthesizes the bolder weight (V1 behavior). If pixel-perfect +// Mac/Windows parity on semibold is ever needed, add a second +// @font-face for `Segoe UI Semibold` pointing at segoeuisb.woff. +@font-face { + font-family: 'Segoe UI'; + src: url('segoeui.eot'); + src: local("Segoe UI"), local("Segoe"), local("Segoe WP"), + url('https://www.onenote.com/styles/segoeui.eot?#iefix') format('embedded-opentype'), + url('https://www.onenote.com/styles/segoeui.woff') format('woff'); + font-weight: normal; +} + // --- Fluent 2 neutral tokens (light theme) --- @colorNeutralBackground1: #ffffff; // Sidebar / surface @colorNeutralBackground2: #fafafa; // Subtle hover From be5b15cd90f9436b23e20e021e518096bc8d5878 Mon Sep 17 00:00:00 2001 From: "Eric Chen (from Dev Box)" Date: Sun, 17 May 2026 02:56:07 -0500 Subject: [PATCH 06/19] Selection mode + ContextImage parity + cross-mode highlight preservation Reproduces V1's context-menu features that were either no-ops or silently broken in V2's unified-window architecture, plus a latent listener bug that caused right-click invocations to crash. 1. Context-menu listener registered once, not N times chrome.contextMenus.onClicked.addListener was being called from inside the for-loop that creates the three menu items, so one right-click fired the handler 3x -> 3 parallel invokeClipperInTab -> 3 captureVisibleTab racing the per-second quota -> disconnected port + "image readback failed" + the popup blinking and closing. Hoisted the addListener call out of the loop; types narrowed to match chrome.contextMenus.OnClickData signature (tab?: Tab). 2. "Clip Image to OneNote" (ContextImage) -- V1 parity The menu item dispatched InvokeMode.ContextImage with the image srcUrl, but the V2 renderer ignored it and just opened in Full Page mode. Worker now captures options.invokeDataForMode into a new pendingInvokeData field, forwarded as `invokeData` on the loadContent port message. Renderer fetches the URL via imageToDataUrl (same path bookmark mode uses for og:image), pushes into regionImages, auto-clicks Region. switchToRegion already does the right thing on a non-empty regionImages -- skips overlay launch, renders thumbnails, enables Save. Tainted/CORS-failed fetch falls back to Full Page focus so the user is never stranded. 3. "Clip Selection to OneNote" -- V1 parity A 6th conditional mode button revealed only when invoked via the right-click "Clip Selection to OneNote" item AND when a non-empty selection was captured. Auto-engages after the screenshot finishes, leaving other modes (Region/Bookmark/Article/PDF) reachable. Button uses Fluent UI System Icons "Copy Select" 20-Regular SVG (matches the four existing Fluent-style mode icons in renderer.html). Selection HTML capture (contentCaptureInject.ts): builds a "selection-substituted doc" -- cloneDocument(document), replace body with the cloned selection fragment, then run the doc-only steps of the existing pipeline (addBaseTagIfNecessary, addImageSize, removeUnwantedItems). Parallel-walk steps (inlineHiddenElements, neutralizePositioning, flattenShadowDomSlots, convertCanvasElementsToImages) are screenshot-stitching concerns and don't apply to static save HTML -- correctly skipped. To preserve URL resolution after the is dropped (we return body.innerHTML which loses head), img.src + a.href are materialized from the resolved properties back into attributes. resolveLazyImages applied on the resulting string, same as full-DOM capture. Renderer (renderer.ts): selection HTML piped through the existing cleanArticleHtml for ONML-equivalent strip. Added `iframe` to the strip list -- safe because the oEmbed article path (composeOEmbedForPreview/composeOEmbedForSave) intentionally bypasses cleanArticleHtml, so the video iframe for known providers still flows through; and Readability already drops iframes upstream. Net effect: selection iframes get stripped, matching V1's tagsNotSupportedInOnml. cleanArticleHtml also gained blank-image removal (V1's removeBlankImages parity). Save dispatch: article + selection share one branch; selection uses cachedSelectionHtml as source and writes AutoPageTagsCodes/AutoPageTags=Article in PageMetadata (V1 parity). Localization: button label + tooltip wired via existing V1 i18n keys WebClipper.ClipType.Selection.Button and .Tooltip -- no new strings. 4. Cross-mode highlight preservation Pre-existing latent bug: switching away from article mode and back preserved highlights via articleWorkingHtml, but article <-> other <-> article via the article->selection or selection->article path would have lost article's highlights (those switch functions never called saveArticleWorkingState). Same gap for selection mode -- selectionWorkingHtml didn't exist; switch-away-then-back rendered the pristine cachedSelectionHtml every time. Generalized saveArticleWorkingState -> saveWorkingState dispatched by currentMode (article/selection slots). Added the save call at the top of switchToArticle and switchToSelection so the outgoing mode's preview-frame body is captured on every transition. switchToSelection now renders selectionWorkingHtml || cachedSelectionHtml, mirroring article. Also added articleWorkingHtml + selectionWorkingHtml to the sign-in/sign-out fresh-capture reset blocks (article's reset was also missing). V1 wrote on every highlight add/delete via handleBodyChange (eager); V2 writes on switch-out (lazy). Functionally equivalent because the save path already clones the live preview-frame DOM when highlights are present. 5. Highlighter line-jump in code blocks .highlight-anchor was display:inline-block, making the wrapped text an atomic layout unit and defeating word-break:break-word inside 
 blocks -- highlighting a long URL forced the entire span to
   its own line. Changed to display:inline; position:relative on an
   inline still works as the containing block for the absolutely-
   positioned delete button (CSS spec: any positioned ancestor, inline
   or block, serves as a containing block).

Co-Authored-By: Claude Opus 4.7 (1M context) 
---
 src/renderer.html                             |   3 +
 .../extensions/contentCaptureInject.ts        |  60 +++++-
 .../webExtensionBase/webExtension.ts          |  76 ++++---
 .../webExtensionBase/webExtensionWorker.ts    |  33 +++-
 src/scripts/renderer.ts                       | 187 ++++++++++++++++--
 5 files changed, 302 insertions(+), 57 deletions(-)

diff --git a/src/renderer.html b/src/renderer.html
index e6cea475..d929416a 100644
--- a/src/renderer.html
+++ b/src/renderer.html
@@ -72,6 +72,9 @@ 
 				
+