From aa537be5235078b0c678fbb07681d08f58bd80e6 Mon Sep 17 00:00:00 2001 From: piquark6046 Date: Wed, 18 Nov 2026 12:39:42 +0000 Subject: [PATCH 1/2] feat: preserve matching domains in wildcard groups --- README.ja.md | 10 ++- README.ko.md | 12 ++- README.md | 12 ++- builder/source/banner/index.ts | 6 +- builder/source/build-grouped.ts | 42 +++++------ libs/source/references/index.ts | 54 ++++++-------- .../utils/discard-resolved-dup-wildcard.ts | 26 +++---- .../utils/regroup-domain-tldlevel.ts | 8 +- .../utils/wildcard-suffix-converter.ts | 12 --- libs/test/references/banner.test.ts | 29 ++++++++ .../discard-resolved-dup-wildcard.test.ts | 4 +- libs/test/references/domain-views.test.ts | 32 ++++++++ libs/test/references/group-metadata.test.ts | 73 +++++++++++++++++++ 13 files changed, 219 insertions(+), 101 deletions(-) delete mode 100644 libs/source/references/utils/wildcard-suffix-converter.ts create mode 100644 libs/test/references/banner.test.ts create mode 100644 libs/test/references/domain-views.test.ts create mode 100644 libs/test/references/group-metadata.test.ts diff --git a/README.ja.md b/README.ja.md index c5301e9..2a63949 100644 --- a/README.ja.md +++ b/README.ja.md @@ -18,6 +18,8 @@ tinyShieldスクリプトは、Adblockコミュニティによって管理され ### クイックスタート このユーザースクリプトを検出するには、次のURLをクリックしてください。 +生成されたスクリプトには正規表現の `@include` に対応した実行環境が必要です。Violentmonkey は対応を明記していますが、AdGuard のユーザースクリプト機能と wBlock での互換性は未確認です。 + https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.user.js ### ドメイングループ購読ガイド (Apple Safari Extension ユーザー専用) @@ -32,13 +34,13 @@ https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.use 1. `https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/grouped/` を開きます。 2. 対象グループのファイル名先頭文字に対応する `/` サブディレクトリを開きます。 3. 利用しているサイトに対応する `tinyShield-.user.js` を1つ選びます。 - 4. そのファイルを一度開き、メタデータ `@match` に対象ドメインが含まれていることを確認します。 + 4. そのファイルを一度開き、メタデータ `@include` に対象ドメインが含まれていることを確認します。 5. Safari のユーザースクリプト拡張には、そのファイルURLをそのまま購読URLとして登録します。 このドメイングループ分割は、https://github.com/FilteringDev/tinyShield/issues/38 の影響範囲を可能な限り最小化することを目的としています。 ### インストールのやり方 -Firefox および Chromium ベースのブラウザーでは **Violentmonkey** を、macOS・iOS・iPadOS の Safari では **wBlock** を推奨します。**Tampermonkey は推奨しません。** +Firefox および Chromium ベースのブラウザーでは **Violentmonkey** を推奨します。Safari の **wBlock** の手順は以下に残していますが、正規表現の `@include` との互換性は未確認です。**Tampermonkey は推奨しません。** - [Violentmonkey](https://violentmonkey.github.io/get-it/) (推奨) - Firefox および Chromium ベースのブラウザー拡張 1. `ダッシュボード`を開きます。 @@ -88,7 +90,7 @@ Firefox および Chromium ベースのブラウザーでは **Violentmonkey** 6. Ad-Shieldで保護されたWebサイトを開いているタブに戻り、再読み込みボタンをタッチでタブを再読み込みします。 -- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (推奨) - macOS・iOS・iPadOS の Safari +- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (未確認) - macOS・iOS・iPadOS の Safari
Safari で wBlock を使ってインストールする @@ -98,7 +100,7 @@ Firefox および Chromium ベースのブラウザーでは **Violentmonkey** - macOS: `Safari` > `設定` > `機能拡張` - iOS・iPadOS 18 以降: `設定` > `アプリ` > `Safari` > `機能拡張` - iOS・iPadOS 17 以前: `設定` > `Safari` > `機能拡張` - 3. 上記のドメイングループ購読ガイドに従ってグループスクリプトURL (`tinyShield-.user.js`) を選択し、`@match` を確認します。 + 3. 上記のドメイングループ購読ガイドに従ってグループスクリプトURL (`tinyShield-.user.js`) を選択し、`@include` を確認します。 4. wBlock アプリの `Userscripts` 画面で、標準搭載の tinyShield を無効にしてグループスクリプトとの重複実行を防ぎます。 5. 同じ画面の追加メニュー (`+`) > `Userscript` > `URL` > `Single URL` を選択し、グループスクリプトURLを入力します。クイックスタートの全体スクリプトURLは使用しないでください。 6. `Next` を選択してメタデータを確認し、`Add` を選択してグループスクリプトを追加・有効化します。 diff --git a/README.ko.md b/README.ko.md index 874af75..0080f0a 100644 --- a/README.ko.md +++ b/README.ko.md @@ -27,6 +27,8 @@ tinyShield 유저스크립트는 애드블록 커뮤니티에 의해 관리되 ### 빠른 시작 아래 URL를 클릭하여 설치해주세요: +생성된 스크립트에는 정규식 `@include`를 지원하는 실행기가 필요합니다. Violentmonkey는 이 기능을 문서화하고 있으며, AdGuard 유저스크립트 기능과 wBlock에서의 호환성은 아직 확인되지 않았습니다. + https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.user.js ### 도메인 그룹 구독 가이드 (Apple Safari Extension 사용자 전용) @@ -41,13 +43,15 @@ https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.use 1. `https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/grouped/`를 엽니다. 2. 대상 그룹 파일명 첫 글자와 같은 `/` 하위 디렉터리로 이동합니다. 3. 사용하는 웹 사이트에 맞는 `tinyShield-.user.js` 파일 하나를 선택합니다. - 4. 해당 파일을 한 번 열어 메타데이터 `@match`에 대상 도메인이 포함되어 있는지 확인합니다. + 4. 해당 파일을 한 번 열어 메타데이터 `@include`에 대상 도메인이 포함되어 있는지 확인합니다. 5. Safari 유저스크립트 확장에 그 파일 URL을 그대로 구독 URL로 등록합니다. 이 도메인 그룹 분리는 https://github.com/FilteringDev/tinyShield/issues/38 의 영향 범위를 가능한 한 최소화하기 위한 목적입니다. +`metadata.json`은 수집된 도메인과 원본 접미사 와일드카드를 그룹 파일명에 연결합니다. 소비자는 `.*`로 끝나는 키를 처리해야 하며, `example.tld` 같은 그룹 이름이 모든 접미사의 실행을 허용하는 것은 아닙니다. 도메인 그룹이 바뀌면 그룹 파일명도 바뀔 수 있습니다. `SHA386`은 대문자 SHA-384 해시입니다. + ### 설치하는 법 -Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를, macOS·iOS·iPadOS의 Safari에서는 **wBlock**을 권장합니다. **Tampermonkey는 권장하지 않습니다.** +Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를 권장합니다. Safari의 **wBlock** 설치 안내는 아래에 유지하지만 정규식 `@include` 호환성은 확인되지 않았습니다. **Tampermonkey는 권장하지 않습니다.** - [Violentmonkey](https://violentmonkey.github.io/get-it/) (권장) - Firefox 및 Chromium 기반 브라우저 확장 1. `Dashboard`를 엽니다. @@ -67,7 +71,7 @@ Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를, macOS·i - AdGuard for Android 1. 빠른 시작 세션에 있는 URL를 클릭하여 설치하십시오. -- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (권장) - macOS·iOS·iPadOS Safari +- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (미검증) - macOS·iOS·iPadOS Safari
Safari에서 wBlock으로 설치하기 @@ -77,7 +81,7 @@ Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를, macOS·i - macOS: `Safari` > `설정` > `확장 프로그램` - iOS·iPadOS 18 이상: `설정` > `앱` > `Safari` > `확장 프로그램` - iOS·iPadOS 17 이하: `설정` > `Safari` > `확장 프로그램` - 3. 위 도메인 그룹 구독 가이드에 따라 대상 도메인의 그룹 스크립트 URL(`tinyShield-.user.js`)을 선택하고 `@match`를 확인하십시오. + 3. 위 도메인 그룹 구독 가이드에 따라 대상 도메인의 그룹 스크립트 URL(`tinyShield-.user.js`)을 선택하고 `@include`를 확인하십시오. 4. wBlock 앱의 `Userscripts` 화면에서 기본 제공되는 tinyShield 항목을 비활성화하여 그룹 스크립트와 중복 실행되지 않게 하십시오. 5. 같은 화면의 추가 메뉴(`+`) > `Userscript` > `URL` > `Single URL`을 선택하고 그룹 스크립트 URL을 입력하십시오. 빠른 시작의 전체 스크립트 URL은 사용하지 마십시오. 6. `Next`를 누르고 메타데이터를 확인한 뒤 `Add`를 눌러 그룹 스크립트를 추가하고 활성화하십시오. diff --git a/README.md b/README.md index 9caa0f5..ba8c9c6 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,8 @@ If you believe this userscript is causing issues while in use, please use GitHub ### Quick Start Just click the following URL to detect this userscript. +The generated scripts require regex `@include` support. Violentmonkey documents this support; compatibility with AdGuard's userscript manager and wBlock has not been verified for these rules. + https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.user.js ### Domain Group Subscription Guide (Apple Safari Extension Only) @@ -41,13 +43,15 @@ https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.use 1. Open `https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/grouped/`. 2. Open the `/` subdirectory that matches the first character of your target grouped file name. 3. Select one `tinyShield-.user.js` file for the website you use. - 4. Open that file once and confirm its metadata `@match` includes your target domain. + 4. Open that file once and confirm its metadata `@include` includes your target domain. 5. Subscribe to that exact grouped file URL in your Safari userscript extension. This domain-group split is intended to reduce the impact scope of https://github.com/FilteringDev/tinyShield/issues/38 as much as possible. +`metadata.json` maps collected domains and original suffix-wildcard patterns to group filenames. Consumers must recognize keys ending in `.*`; generated group names such as `example.tld` do not themselves authorize every suffix. Group filenames may change when the domain groups change. `SHA386` contains the uppercase SHA-384 digest. + ### How to install -**Violentmonkey** is recommended for Firefox and Chromium-based browsers. **wBlock** is recommended for Safari on macOS, iOS, and iPadOS. **Tampermonkey is not recommended.** +**Violentmonkey** is recommended for Firefox and Chromium-based browsers. The instructions for **wBlock** on Safari are retained below, but regex `@include` compatibility is unverified. **Tampermonkey is not recommended.** - [Violentmonkey](https://violentmonkey.github.io/get-it/) (Recommended) - Firefox and Chromium-based browser extension 1. Open `Dashboard`. @@ -97,7 +101,7 @@ This domain-group split is intended to reduce the impact scope of https://github 6. Return to a tab opening the website protected by Ad-Shield and reload the tab by touching the reload button. -- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (Recommended) - Safari on macOS, iOS, and iPadOS +- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (Unverified) - Safari on macOS, iOS, and iPadOS
Installing with wBlock in Safari @@ -107,7 +111,7 @@ This domain-group split is intended to reduce the impact scope of https://github - macOS: `Safari` > `Settings` > `Extensions` - iOS/iPadOS 18+: `Settings` > `Apps` > `Safari` > `Extensions` - iOS/iPadOS 17 and older: `Settings` > `Safari` > `Extensions` - 3. Follow the domain group subscription guide above to choose your grouped script URL (`tinyShield-.user.js`) and check its `@match` metadata. + 3. Follow the domain group subscription guide above to choose your grouped script URL (`tinyShield-.user.js`) and check its `@include` metadata. 4. In the wBlock app's `Userscripts` screen, disable the built-in tinyShield entry to avoid running it alongside the grouped script. 5. On the same screen, open the add menu (`+`) > `Userscript` > `URL` > `Single URL` and enter the grouped script URL. Do not use the full script URL from Quick Start. 6. Select `Next`, review the metadata, then select `Add` to add and enable the grouped script. diff --git a/builder/source/banner/index.ts b/builder/source/banner/index.ts index 89cc27a..c0fbdd9 100644 --- a/builder/source/banner/index.ts +++ b/builder/source/banner/index.ts @@ -39,8 +39,10 @@ export function CreateBanner(Options: BannerOptions): string { BannerString += '//\n' for (const Domain of Options.Domains) { - BannerString += `// @match *://${Domain}/*\n` - BannerString += `// @match *://*.${Domain}/*\n` + const WildcardSuffix = Domain.endsWith('.*') + const Host = (WildcardSuffix ? Domain.slice(0, -2) : Domain).replaceAll(/[.*+?^${}()|[\]\\/]/g, '\\$&') + const Suffix = WildcardSuffix ? '(?:\\.[^.\\/:?#@]+)+' : '' + BannerString += `// @include /^https?:\\/\\/(?:[^.\\/:?#@]+\\.)*${Host}${Suffix}\\.?(?::[0-9]+)?(?:[\\/?#]|$)/\n` } BannerString += '// ==/UserScript==\n\n' return BannerString diff --git a/builder/source/build-grouped.ts b/builder/source/build-grouped.ts index 1e9d5c7..bf54756 100644 --- a/builder/source/build-grouped.ts +++ b/builder/source/build-grouped.ts @@ -17,16 +17,13 @@ export class GroupedBuild extends Build { super(FromOrOption) } - private TransformDomainSetToFileName(DomainSet: Set>): Set> { - const Result = new Set>() - for (const Value of DomainSet) { - const FileNameSet = new Set() - for (const Domain of Value) { - FileNameSet.add(Domain.endsWith('.*') ? Domain.replaceAll(/\.\*$/g, '.tld') : Domain) - } - Result.add(FileNameSet) - } - return Result + private GetGroups(): Map, Set> { + const Groups = this.FetchedDomains.get('Groups') + if (!Groups) throw new Error('Domain groups are not initialized.') + return new Map([...Groups].map(([Names, Domains]) => [ + new Set([...Names].map(Domain => Domain.endsWith('.*') ? Domain.slice(0, -2) + '.tld' : Domain)), + Domains + ])) } async MakeGroupMetadata() { @@ -38,12 +35,8 @@ export class GroupedBuild extends Build { FileMapping: {} } - const FileNameList: Set> = this.TransformDomainSetToFileName(new Set>(this.FetchedDomains.get('EachDomain'))) - const MatchingDomains: Set> = new Set>(this.FetchedDomains.get('EachDomainFull')) - - for (let I = 0; I < FileNameList.size; I++) { - const FileName = [...[...FileNameList][I]][0] - const Domains = [...MatchingDomains][I] + for (const [Names, Domains] of this.GetGroups()) { + const FileName = Names.values().next().value! const FilePath = `${this.ProjectRoot}/dist/grouped/${FileName[0]}/tinyShield-${FileName}.user.js` const FileContent = await Fs.promises.readFile(FilePath) const SHA386Hash = Crypto.createHash('sha384').update(FileContent).digest('hex').toUpperCase() @@ -67,19 +60,20 @@ export class GroupedBuild extends Build { this.Options.Version ??= (await PackageJson.load(this.ProjectRoot)).content.version ?? '0.0.0' - const FileNameList: Set> = this.TransformDomainSetToFileName(new Set>(this.FetchedDomains.get('EachDomain'))) - const MatchingDomains: Set> = new Set>(this.FetchedDomains.get('EachDomainFull')) - const Workerpool = new Piscina.Piscina({ filename: Path.resolve(this.ProjectRoot, 'builder/source/build-grouped-worker.ts'), - execArgv: [...Process.execArgv, '--import=tsx'], + execArgv: [...Process.execArgv, `--import=${import.meta.resolve('tsx')}`], workerData: { Config: { ...this.Options, SubscriptionUrl: this.Options.SubscriptionUrl.href }, ProjectRoot: this.ProjectRoot } }) - const WorkerResults: Promise[] = [] - for (let I = 0; I < FileNameList.size; I++) { - WorkerResults.push(Workerpool.run({ FileName: [...FileNameList][I], Domains: [...MatchingDomains][I] })) + try { + const WorkerResults: Promise[] = [] + for (const [FileName, Domains] of this.GetGroups()) { + WorkerResults.push(Workerpool.run({ FileName, Domains })) + } + await Promise.all(WorkerResults) + } finally { + await Workerpool.destroy() } - await Promise.all(WorkerResults) } } \ No newline at end of file diff --git a/libs/source/references/index.ts b/libs/source/references/index.ts index cd921ac..a115647 100644 --- a/libs/source/references/index.ts +++ b/libs/source/references/index.ts @@ -3,8 +3,7 @@ import { CustomDefinedMatches, CustomExcludeMatches } from './custom-defined.js' import { FetchAdShieldDomainsFromFiltersLists } from './filterslists.js' import { FetchIABSellersJsonData } from './iabsellers.js' import { DiscardResolvedDupWildcard } from './utils/discard-resolved-dup-wildcard.js' -import { RegroupDomainTldLevel } from './utils/regroup-domain-tldlevel.js' -import { ConvertWildcardSuffixToRegexPattern } from './utils/wildcard-suffix-converter.js' +import { GetDomainRootLabel, RegroupDomainTldLevel } from './utils/regroup-domain-tldlevel.js' export { CustomDefinedMatches, CustomExcludeMatches } from './custom-defined.js' export { FetchAdShieldDomainsFromFiltersLists } from './filterslists.js' @@ -14,31 +13,35 @@ export { IndexAdShieldDomainsFromUBO } from './filterslists/uBO.js' export { AdShieldCDNDomains, IsAdShieldCDNDomain } from './filterslists/keywords.js' export { DiscardResolvedDupWildcard } from './utils/discard-resolved-dup-wildcard.js' export { RegroupDomainTldLevel } from './utils/regroup-domain-tldlevel.js' -export { ConvertWildcardSuffixToRegexPattern, PublicSuffixList } from './utils/wildcard-suffix-converter.js' export type TASDomainContainer = Map<'Normal', Set> & Map<'Full', Set> & Map<'EachDomain', Set>> & - Map<'EachDomainFull', Set>> + Map<'EachDomainFull', Set>> & + Map<'Groups', Map, Set>> -function ConvertToFlatFullDomains(Origin: Set): Set { - return new Set([...Origin].flatMap(Domain => ConvertWildcardSuffixToRegexPattern(Domain))) -} - -function IsGroupedDomains(Origin: Set | Set>): Origin is Set> { - const FirstValue = Origin.values().next().value - return typeof FirstValue !== 'string' && typeof FirstValue !== 'undefined' -} - -function ConvertToFullDomains(Origin: Set): Set -function ConvertToFullDomains(Origin: Set>): Set> -function ConvertToFullDomains(Origin: Set | Set>): Set | Set> { - if (IsGroupedDomains(Origin)) { - return new Set>([...Origin].map(DomainGroup => ConvertToFlatFullDomains(DomainGroup))) +export function CreateDomainViews(Domains: Set): TASDomainContainer { + const FullDomains = new Set([...Domains].sort()) + const NormalDomains = DiscardResolvedDupWildcard(FullDomains) + const EachDomain = RegroupDomainTldLevel(NormalDomains) + const MatchingGroups = new Map>() + for (const Domain of FullDomains) { + const Root = GetDomainRootLabel(Domain) + const Group = MatchingGroups.get(Root) ?? new Set() + Group.add(Domain) + MatchingGroups.set(Root, Group) } - return ConvertToFlatFullDomains(Origin) + const Groups = new Map([...EachDomain].map(Group => [Group, MatchingGroups.get(GetDomainRootLabel(Group.values().next().value!))!] as const)) + const EachDomainFull = new Set(Groups.values()) + const Result: TASDomainContainer = new Map() + Result.set('Normal', NormalDomains) + Result.set('Full', FullDomains) + Result.set('EachDomain', EachDomain) + Result.set('EachDomainFull', EachDomainFull) + Result.set('Groups', Groups) + return Result } export async function FetchAdShieldDomains(): Promise { @@ -54,16 +57,5 @@ export async function FetchAdShieldDomains(): Promise { }) CustomDefinedMatches.forEach(Match => CombinedDomains.add(Match)) CustomExcludeMatches.forEach(Match => CombinedDomains.delete(Match)) - const Result: TASDomainContainer = new Map() - const NormalDomains = DiscardResolvedDupWildcard(CombinedDomains) - const FullDomains = ConvertToFullDomains(NormalDomains) - const EachDomain = RegroupDomainTldLevel(NormalDomains) - const EachDomainFull = ConvertToFullDomains(EachDomain) - - Result.set('Normal', NormalDomains) - Result.set('Full', FullDomains) - Result.set('EachDomain', EachDomain) - Result.set('EachDomainFull', EachDomainFull) - - return Result + return CreateDomainViews(CombinedDomains) } diff --git a/libs/source/references/utils/discard-resolved-dup-wildcard.ts b/libs/source/references/utils/discard-resolved-dup-wildcard.ts index 2595835..82ee6ff 100644 --- a/libs/source/references/utils/discard-resolved-dup-wildcard.ts +++ b/libs/source/references/utils/discard-resolved-dup-wildcard.ts @@ -1,5 +1,4 @@ import * as TLDTS from 'tldts' -import { PublicSuffixList } from './wildcard-suffix-converter.js' type ParsedEntry = { Entry: string @@ -23,9 +22,6 @@ function ParseEntry(Entry: string): ParsedEntry { } const Parsed = TLDTS.parse(Entry) - if (Parsed.publicSuffix) { - PublicSuffixList.add(Parsed.publicSuffix) - } if (Parsed.hostname && Parsed.publicSuffix && Parsed.hostname.endsWith(`.${Parsed.publicSuffix}`)) { return { @@ -48,7 +44,10 @@ function ParseEntry(Entry: string): ParsedEntry { } function IsCoveredByParent(Child: ParsedEntry, Parent: ParsedEntry): boolean { - if (Child.Stem === Parent.Stem || !Child.Stem.endsWith(`.${Parent.Stem}`)) { + if (Child.Stem === Parent.Stem) { + return Parent.WildcardSuffix && !Child.WildcardSuffix + } + if (!Child.Stem.endsWith(`.${Parent.Stem}`)) { return false } @@ -61,23 +60,22 @@ function IsCoveredByParent(Child: ParsedEntry, Parent: ParsedEntry): boolean { export function DiscardResolvedDupWildcard(OriginSet: Set): Set { const ParsedEntries = [...OriginSet].map(ParseEntry) - const RootsWithWildcard = new Set( - ParsedEntries - .filter(Entry => Entry.WildcardSuffix) - .map(Entry => Entry.RootLabel) - ) - const ConcreteStemCounts = new Map() + const ConcreteSuffixes = new Map>() ParsedEntries - .filter(Entry => !Entry.WildcardSuffix) - .forEach(Entry => ConcreteStemCounts.set(Entry.Stem, (ConcreteStemCounts.get(Entry.Stem) ?? 0) + 1)) + .filter(Entry => !Entry.WildcardSuffix && Entry.PublicSuffix) + .forEach(Entry => { + const Suffixes = ConcreteSuffixes.get(Entry.Stem) ?? new Set() + Suffixes.add(Entry.PublicSuffix!) + ConcreteSuffixes.set(Entry.Stem, Suffixes) + }) const NormalizedEntries = ParsedEntries.map(Entry => { if (Entry.WildcardSuffix) { return Entry } - if (RootsWithWildcard.has(Entry.RootLabel) || (ConcreteStemCounts.get(Entry.Stem) ?? 0) >= 2) { + if ((ConcreteSuffixes.get(Entry.Stem)?.size ?? 0) >= 2) { return ParseEntry(`${Entry.Stem}.*`) } diff --git a/libs/source/references/utils/regroup-domain-tldlevel.ts b/libs/source/references/utils/regroup-domain-tldlevel.ts index e6be615..ca0784e 100644 --- a/libs/source/references/utils/regroup-domain-tldlevel.ts +++ b/libs/source/references/utils/regroup-domain-tldlevel.ts @@ -1,5 +1,4 @@ import * as TLDTS from 'tldts' -import { PublicSuffixList } from './wildcard-suffix-converter.js' type ParsedEntry = { Entry: string @@ -23,9 +22,6 @@ function ParseEntry(Entry: string): ParsedEntry { } const Parsed = TLDTS.parse(Entry) - if (Parsed.publicSuffix) { - PublicSuffixList.add(Parsed.publicSuffix) - } if (Parsed.hostname && Parsed.publicSuffix && Parsed.hostname.endsWith(`.${Parsed.publicSuffix}`)) { return { @@ -47,6 +43,10 @@ function ParseEntry(Entry: string): ParsedEntry { } } +export function GetDomainRootLabel(Entry: string): string { + return ParseEntry(Entry).RootLabel +} + function IsCoveredByParent(Child: ParsedEntry, Parent: ParsedEntry): boolean { if (Child.Stem === Parent.Stem || !Child.Stem.endsWith(`.${Parent.Stem}`)) { return false diff --git a/libs/source/references/utils/wildcard-suffix-converter.ts b/libs/source/references/utils/wildcard-suffix-converter.ts deleted file mode 100644 index 0261193..0000000 --- a/libs/source/references/utils/wildcard-suffix-converter.ts +++ /dev/null @@ -1,12 +0,0 @@ -export const PublicSuffixList = new Set([ - 'com', 'org', 'co', 'de', 'ru', 'fr', 'me', 'it', 'nl', 'io', 'cc', 'in', 'pl', 'xyz', 'es', 'se', 'uk', 'tv', 'info', - 'site', 'us', 'online', 'ch', 'at', 'eu', 'top', 'be', 'cz', 'app', 'ca', 'to', 'jp', 'dev', 'kr' -]) - -export function ConvertWildcardSuffixToRegexPattern(Domain: string): string[] { - const Result: string[] = [] - PublicSuffixList.forEach(Suffix => { - Result.push(Domain.replaceAll(/\.\*$/g, '.' + Suffix)) - }) - return Result -} diff --git a/libs/test/references/banner.test.ts b/libs/test/references/banner.test.ts new file mode 100644 index 0000000..7a03a82 --- /dev/null +++ b/libs/test/references/banner.test.ts @@ -0,0 +1,29 @@ +import { expect, test } from 'vitest' +import { Script } from 'node:vm' +import { CreateBanner } from '../../../builder/source/banner/index.js' + +test('banner includes only collected domains and explicit wildcard suffixes', () => { + const Url = new URL('https://example.com/') + const Banner = CreateBanner({ + Version: '1', BuildType: 'production', Domains: new Set(['example.com', 'example.net', 'wild.*']), + Author: 'author', Name: 'name', Namespace: 'namespace', HomepageURL: Url, SupportURL: Url, + UpdateURL: Url, DownloadURL: Url, License: 'license', Description: { en: 'description' } + }) + const Patterns = [...Banner.matchAll(/^\/\/ @include\s+\/(.*)\/$/gm)].map(Match => new RegExp(Match[1])) + for (const Match of Banner.matchAll(/^\/\/ @include\s+(\/.*\/)$/gm)) { + expect(() => new Script(Match[1])).not.toThrow() + } + const Matches = (Input: string) => Patterns.some(Pattern => Pattern.test(Input)) + + expect(Patterns).toHaveLength(3) + expect(Matches('https://sub.example.com:8443/path')).toBe(true) + expect(Matches('http://example.net/?query')).toBe(true) + expect(Matches('https://example.org/')).toBe(false) + expect(Matches('https://notexample.com/')).toBe(false) + expect(Matches('https://example.com.evil/')).toBe(false) + expect(Matches('https://wild.com@evil.test/')).toBe(false) + expect(Matches('https://example.com./')).toBe(true) + expect(Matches('ftp://example.com/')).toBe(false) + expect(Matches('https://sub.wild.co.uk/')).toBe(true) + expect(Matches('https://wild.com.evil/')).toBe(true) +}) \ No newline at end of file diff --git a/libs/test/references/discard-resolved-dup-wildcard.test.ts b/libs/test/references/discard-resolved-dup-wildcard.test.ts index c032cd3..414acd6 100644 --- a/libs/test/references/discard-resolved-dup-wildcard.test.ts +++ b/libs/test/references/discard-resolved-dup-wildcard.test.ts @@ -30,14 +30,14 @@ test('DiscardResolvedDupWildcard removes resolved duplicate wildcards with multi test('DiscardResolvedDupWildcard handles nested wildcard scenarios', () => { const Input = new Set(['token.google.*', 'access.google.*', 'tools.google.com', 'google.google.co.kr']) - const Expected = new Set(['token.google.*', 'access.google.*', 'tools.google.*', 'google.google.*']) + const Expected = Input return expect(DiscardResolvedDupWildcard(Input)).toEqual(Expected) }) test('DiscardResolvedDupWildcard handles complex wildcard scenarios', () => { const Input = new Set(['token.google.*', 'access.google.*', 'tools.google.com', 'google.google.co.kr', 'example.*', 'example.com', 'rust-lang.org']) - const Expected = new Set(['token.google.*', 'access.google.*', 'tools.google.*', 'google.google.*', 'example.*', 'rust-lang.org']) + const Expected = new Set(['token.google.*', 'access.google.*', 'tools.google.com', 'google.google.co.kr', 'example.*', 'rust-lang.org']) return expect(DiscardResolvedDupWildcard(Input)).toEqual(Expected) }) diff --git a/libs/test/references/domain-views.test.ts b/libs/test/references/domain-views.test.ts new file mode 100644 index 0000000..00d5de5 --- /dev/null +++ b/libs/test/references/domain-views.test.ts @@ -0,0 +1,32 @@ +import { expect, test } from 'vitest' +import { CreateDomainViews } from '@filteringdev/tinyshield-lib/references' + +test('preserves collected domains and explicit wildcards in matching groups', () => { + const Domains = new Set(['example.com', 'other.org', 'example.net', 'sub.example.com', 'wild.*']) + const Views = CreateDomainViews(Domains) + + expect(Views.get('Normal')).toEqual(new Set(['example.*', 'other.org', 'wild.*'])) + expect(Views.get('Full')).toEqual(Domains) + expect([...Views.get('EachDomain')!]).toEqual([ + new Set(['example.*']), new Set(['other.org']), new Set(['wild.*']) + ]) + expect([...Views.get('EachDomainFull')!]).toEqual([ + new Set(['example.com', 'example.net', 'sub.example.com']), new Set(['other.org']), new Set(['wild.*']) + ]) +}) + +test('explicit wildcards and concrete domains remain one group', () => { + const Domains = new Set(['example.*', 'example.com', 'sub.example.net']) + const Views = CreateDomainViews(Domains) + expect(Views.get('Normal')).toEqual(new Set(['example.*'])) + expect([...Views.get('EachDomainFull')!]).toEqual([Domains]) +}) + +test('domain views and group names do not depend on input order', () => { + const Domains = ['z.example.com', 'a.example.com', 'example.net', 'example.com', 'wild.*'] + const Forward = CreateDomainViews(new Set(Domains)) + const Reverse = CreateDomainViews(new Set([...Domains].reverse())) + expect([...Forward.get('Normal')!]).toEqual([...Reverse.get('Normal')!]) + expect([...Forward.get('Groups')!].map(([Names, Matches]) => [[...Names], [...Matches]])) + .toEqual([...Reverse.get('Groups')!].map(([Names, Matches]) => [[...Names], [...Matches]])) +}) \ No newline at end of file diff --git a/libs/test/references/group-metadata.test.ts b/libs/test/references/group-metadata.test.ts new file mode 100644 index 0000000..fb928f5 --- /dev/null +++ b/libs/test/references/group-metadata.test.ts @@ -0,0 +1,73 @@ +import { createHash } from 'node:crypto' +import { mkdtemp, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { expect, test } from 'vitest' +import { CreateDomainViews } from '@filteringdev/tinyshield-lib/references' +import { GroupedBuild } from '../../../builder/source/build-grouped.js' +import { StandardBuild } from '../../../builder/source/build.js' + +class TestGroupedBuild extends GroupedBuild { + constructor(Root: string, Domains: Set) { + super({ Minify: false, BuildType: 'development', SubscriptionUrl: 'https://example.com/tinyShield.user.js' }) + this.ProjectRoot = Root + this.FetchedDomains = CreateDomainViews(Domains) + } +} + +class TestStandardBuild extends StandardBuild { + constructor(Root: string, Domains: Set) { + super({ Minify: false, BuildType: 'development', SubscriptionUrl: 'https://example.com/tinyShield.user.js', Version: '1' }) + this.ProjectRoot = Root + this.FetchedDomains = CreateDomainViews(Domains) + } +} + +test('group metadata maps collected domains without expanding wildcard suffixes', async () => { + const Root = await mkdtemp(join(tmpdir(), 'tinyshield-group-')) + try { + const Directory = join(Root, 'dist/grouped/e') + await mkdir(Directory, { recursive: true }) + const Content = 'grouped userscript' + await writeFile(join(Directory, 'tinyShield-example.tld.user.js'), Content) + const Builder = new TestGroupedBuild(Root, new Set(['example.com', 'example.net'])) + await Builder.MakeGroupMetadata() + const Metadata: unknown = JSON.parse(await readFile(join(Root, 'dist/grouped/metadata.json'), 'utf8')) + expect(Metadata).toEqual({ + DomainMapping: { 'example.com': 'example.tld', 'example.net': 'example.tld' }, + FileMapping: { + 'example.tld': { + SHA386: createHash('sha384').update(Content).digest('hex').toUpperCase(), + FileName: 'tinyShield-example.tld.user.js' + } + } + }) + } finally { + await rm(Root, { recursive: true, force: true }) + } +}) + +test('standard and grouped workers build with collected matching rules', async () => { + const Root = await mkdtemp(join(tmpdir(), 'tinyshield-build-')) + try { + await symlink(new URL('../../../builder/', import.meta.url).pathname, join(Root, 'builder'), 'dir') + await symlink(new URL('../../../userscript/', import.meta.url).pathname, join(Root, 'userscript'), 'dir') + await symlink(new URL('../../../package.json', import.meta.url).pathname, join(Root, 'package.json')) + const Domains = new Set(['example.com', 'example.net']) + await new TestStandardBuild(Root, Domains).Build() + const Builder = new TestGroupedBuild(Root, Domains) + await Builder.Build() + await Builder.MakeGroupMetadata() + const Standard = await readFile(join(Root, 'dist/tinyShield.dev.user.js'), 'utf8') + const Grouped = await readFile(join(Root, 'dist/grouped/e/tinyShield-example.tld.user.js'), 'utf8') + for (const Output of [Standard, Grouped]) { + expect(Output).toContain('@include /^https?:') + expect(Output).toContain('example\\.com') + expect(Output).toContain('example\\.net') + expect(Output).not.toContain('example\\.org') + expect(Output).not.toContain('@match') + } + } finally { + await rm(Root, { recursive: true, force: true }) + } +}) \ No newline at end of file From 0f22a6f55d3faa75182c73f2eb091227bea8e0c8 Mon Sep 17 00:00:00 2001 From: piquark6046 Date: Wed, 18 Nov 2026 13:02:10 +0000 Subject: [PATCH 2/2] fix: use standard match rules for domain groups --- README.ja.md | 10 +++--- README.ko.md | 12 +++---- README.md | 12 +++---- builder/source/banner/index.ts | 11 +++--- builder/source/build-grouped.ts | 15 +++++--- libs/test/references/banner.test.ts | 38 +++++++++++---------- libs/test/references/group-metadata.test.ts | 17 ++++----- 7 files changed, 63 insertions(+), 52 deletions(-) diff --git a/README.ja.md b/README.ja.md index 2a63949..3caa0a2 100644 --- a/README.ja.md +++ b/README.ja.md @@ -18,7 +18,7 @@ tinyShieldスクリプトは、Adblockコミュニティによって管理され ### クイックスタート このユーザースクリプトを検出するには、次のURLをクリックしてください。 -生成されたスクリプトには正規表現の `@include` に対応した実行環境が必要です。Violentmonkey は対応を明記していますが、AdGuard のユーザースクリプト機能と wBlock での互換性は未確認です。 +生成されたスクリプトは標準の `@match` を使用します。元データの `.*` パターンは、具体的なドメインも収集されない限り除外されます。 https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.user.js @@ -34,13 +34,13 @@ https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.use 1. `https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/grouped/` を開きます。 2. 対象グループのファイル名先頭文字に対応する `/` サブディレクトリを開きます。 3. 利用しているサイトに対応する `tinyShield-.user.js` を1つ選びます。 - 4. そのファイルを一度開き、メタデータ `@include` に対象ドメインが含まれていることを確認します。 + 4. そのファイルを一度開き、メタデータ `@match` に対象ドメインが含まれていることを確認します。 5. Safari のユーザースクリプト拡張には、そのファイルURLをそのまま購読URLとして登録します。 このドメイングループ分割は、https://github.com/FilteringDev/tinyShield/issues/38 の影響範囲を可能な限り最小化することを目的としています。 ### インストールのやり方 -Firefox および Chromium ベースのブラウザーでは **Violentmonkey** を推奨します。Safari の **wBlock** の手順は以下に残していますが、正規表現の `@include` との互換性は未確認です。**Tampermonkey は推奨しません。** +Firefox および Chromium ベースのブラウザーでは **Violentmonkey** を、macOS・iOS・iPadOS の Safari では **wBlock** を推奨します。**Tampermonkey は推奨しません。** - [Violentmonkey](https://violentmonkey.github.io/get-it/) (推奨) - Firefox および Chromium ベースのブラウザー拡張 1. `ダッシュボード`を開きます。 @@ -90,7 +90,7 @@ Firefox および Chromium ベースのブラウザーでは **Violentmonkey** 6. Ad-Shieldで保護されたWebサイトを開いているタブに戻り、再読み込みボタンをタッチでタブを再読み込みします。 -- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (未確認) - macOS・iOS・iPadOS の Safari +- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (推奨) - macOS・iOS・iPadOS の Safari
Safari で wBlock を使ってインストールする @@ -100,7 +100,7 @@ Firefox および Chromium ベースのブラウザーでは **Violentmonkey** - macOS: `Safari` > `設定` > `機能拡張` - iOS・iPadOS 18 以降: `設定` > `アプリ` > `Safari` > `機能拡張` - iOS・iPadOS 17 以前: `設定` > `Safari` > `機能拡張` - 3. 上記のドメイングループ購読ガイドに従ってグループスクリプトURL (`tinyShield-.user.js`) を選択し、`@include` を確認します。 + 3. 上記のドメイングループ購読ガイドに従ってグループスクリプトURL (`tinyShield-.user.js`) を選択し、`@match` を確認します。 4. wBlock アプリの `Userscripts` 画面で、標準搭載の tinyShield を無効にしてグループスクリプトとの重複実行を防ぎます。 5. 同じ画面の追加メニュー (`+`) > `Userscript` > `URL` > `Single URL` を選択し、グループスクリプトURLを入力します。クイックスタートの全体スクリプトURLは使用しないでください。 6. `Next` を選択してメタデータを確認し、`Add` を選択してグループスクリプトを追加・有効化します。 diff --git a/README.ko.md b/README.ko.md index 0080f0a..c6e6a4e 100644 --- a/README.ko.md +++ b/README.ko.md @@ -27,7 +27,7 @@ tinyShield 유저스크립트는 애드블록 커뮤니티에 의해 관리되 ### 빠른 시작 아래 URL를 클릭하여 설치해주세요: -생성된 스크립트에는 정규식 `@include`를 지원하는 실행기가 필요합니다. Violentmonkey는 이 기능을 문서화하고 있으며, AdGuard 유저스크립트 기능과 wBlock에서의 호환성은 아직 확인되지 않았습니다. +생성된 스크립트는 표준 `@match`를 사용합니다. 원본의 `.*` 패턴은 구체적인 도메인이 함께 수집되지 않으면 제외됩니다. https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.user.js @@ -43,15 +43,15 @@ https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.use 1. `https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/grouped/`를 엽니다. 2. 대상 그룹 파일명 첫 글자와 같은 `/` 하위 디렉터리로 이동합니다. 3. 사용하는 웹 사이트에 맞는 `tinyShield-.user.js` 파일 하나를 선택합니다. - 4. 해당 파일을 한 번 열어 메타데이터 `@include`에 대상 도메인이 포함되어 있는지 확인합니다. + 4. 해당 파일을 한 번 열어 메타데이터 `@match`에 대상 도메인이 포함되어 있는지 확인합니다. 5. Safari 유저스크립트 확장에 그 파일 URL을 그대로 구독 URL로 등록합니다. 이 도메인 그룹 분리는 https://github.com/FilteringDev/tinyShield/issues/38 의 영향 범위를 가능한 한 최소화하기 위한 목적입니다. -`metadata.json`은 수집된 도메인과 원본 접미사 와일드카드를 그룹 파일명에 연결합니다. 소비자는 `.*`로 끝나는 키를 처리해야 하며, `example.tld` 같은 그룹 이름이 모든 접미사의 실행을 허용하는 것은 아닙니다. 도메인 그룹이 바뀌면 그룹 파일명도 바뀔 수 있습니다. `SHA386`은 대문자 SHA-384 해시입니다. +`metadata.json`은 수집된 구체 도메인을 그룹 파일명에 연결합니다. `example.tld` 같은 그룹 이름이 모든 접미사의 실행을 허용하는 것은 아닙니다. 도메인 그룹이 바뀌면 그룹 파일명도 바뀔 수 있습니다. `SHA386`은 대문자 SHA-384 해시입니다. ### 설치하는 법 -Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를 권장합니다. Safari의 **wBlock** 설치 안내는 아래에 유지하지만 정규식 `@include` 호환성은 확인되지 않았습니다. **Tampermonkey는 권장하지 않습니다.** +Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를, macOS·iOS·iPadOS의 Safari에서는 **wBlock**을 권장합니다. **Tampermonkey는 권장하지 않습니다.** - [Violentmonkey](https://violentmonkey.github.io/get-it/) (권장) - Firefox 및 Chromium 기반 브라우저 확장 1. `Dashboard`를 엽니다. @@ -71,7 +71,7 @@ Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를 권장합 - AdGuard for Android 1. 빠른 시작 세션에 있는 URL를 클릭하여 설치하십시오. -- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (미검증) - macOS·iOS·iPadOS Safari +- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (권장) - macOS·iOS·iPadOS Safari
Safari에서 wBlock으로 설치하기 @@ -81,7 +81,7 @@ Firefox 및 Chromium 기반 브라우저에서는 **Violentmonkey**를 권장합 - macOS: `Safari` > `설정` > `확장 프로그램` - iOS·iPadOS 18 이상: `설정` > `앱` > `Safari` > `확장 프로그램` - iOS·iPadOS 17 이하: `설정` > `Safari` > `확장 프로그램` - 3. 위 도메인 그룹 구독 가이드에 따라 대상 도메인의 그룹 스크립트 URL(`tinyShield-.user.js`)을 선택하고 `@include`를 확인하십시오. + 3. 위 도메인 그룹 구독 가이드에 따라 대상 도메인의 그룹 스크립트 URL(`tinyShield-.user.js`)을 선택하고 `@match`를 확인하십시오. 4. wBlock 앱의 `Userscripts` 화면에서 기본 제공되는 tinyShield 항목을 비활성화하여 그룹 스크립트와 중복 실행되지 않게 하십시오. 5. 같은 화면의 추가 메뉴(`+`) > `Userscript` > `URL` > `Single URL`을 선택하고 그룹 스크립트 URL을 입력하십시오. 빠른 시작의 전체 스크립트 URL은 사용하지 마십시오. 6. `Next`를 누르고 메타데이터를 확인한 뒤 `Add`를 눌러 그룹 스크립트를 추가하고 활성화하십시오. diff --git a/README.md b/README.md index ba8c9c6..23c6ef0 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ If you believe this userscript is causing issues while in use, please use GitHub ### Quick Start Just click the following URL to detect this userscript. -The generated scripts require regex `@include` support. Violentmonkey documents this support; compatibility with AdGuard's userscript manager and wBlock has not been verified for these rules. +The generated scripts use standard `@match` rules. Source patterns ending in `.*` are excluded unless concrete domains are also collected. https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.user.js @@ -43,15 +43,15 @@ https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/tinyShield.use 1. Open `https://cdn.jsdelivr.net/npm/@filteringdev/tinyshield@latest/dist/grouped/`. 2. Open the `/` subdirectory that matches the first character of your target grouped file name. 3. Select one `tinyShield-.user.js` file for the website you use. - 4. Open that file once and confirm its metadata `@include` includes your target domain. + 4. Open that file once and confirm its metadata `@match` includes your target domain. 5. Subscribe to that exact grouped file URL in your Safari userscript extension. This domain-group split is intended to reduce the impact scope of https://github.com/FilteringDev/tinyShield/issues/38 as much as possible. -`metadata.json` maps collected domains and original suffix-wildcard patterns to group filenames. Consumers must recognize keys ending in `.*`; generated group names such as `example.tld` do not themselves authorize every suffix. Group filenames may change when the domain groups change. `SHA386` contains the uppercase SHA-384 digest. +`metadata.json` maps collected concrete domains to group filenames. Generated group names such as `example.tld` do not themselves authorize every suffix. Group filenames may change when the domain groups change. `SHA386` contains the uppercase SHA-384 digest. ### How to install -**Violentmonkey** is recommended for Firefox and Chromium-based browsers. The instructions for **wBlock** on Safari are retained below, but regex `@include` compatibility is unverified. **Tampermonkey is not recommended.** +**Violentmonkey** is recommended for Firefox and Chromium-based browsers. **wBlock** is recommended for Safari on macOS, iOS, and iPadOS. **Tampermonkey is not recommended.** - [Violentmonkey](https://violentmonkey.github.io/get-it/) (Recommended) - Firefox and Chromium-based browser extension 1. Open `Dashboard`. @@ -101,7 +101,7 @@ This domain-group split is intended to reduce the impact scope of https://github 6. Return to a tab opening the website protected by Ad-Shield and reload the tab by touching the reload button. -- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (Unverified) - Safari on macOS, iOS, and iPadOS +- [wBlock](https://apps.apple.com/app/wblock/id6746388723) (Recommended) - Safari on macOS, iOS, and iPadOS
Installing with wBlock in Safari @@ -111,7 +111,7 @@ This domain-group split is intended to reduce the impact scope of https://github - macOS: `Safari` > `Settings` > `Extensions` - iOS/iPadOS 18+: `Settings` > `Apps` > `Safari` > `Extensions` - iOS/iPadOS 17 and older: `Settings` > `Safari` > `Extensions` - 3. Follow the domain group subscription guide above to choose your grouped script URL (`tinyShield-.user.js`) and check its `@include` metadata. + 3. Follow the domain group subscription guide above to choose your grouped script URL (`tinyShield-.user.js`) and check its `@match` metadata. 4. In the wBlock app's `Userscripts` screen, disable the built-in tinyShield entry to avoid running it alongside the grouped script. 5. On the same screen, open the add menu (`+`) > `Userscript` > `URL` > `Single URL` and enter the grouped script URL. Do not use the full script URL from Quick Start. 6. Select `Next`, review the metadata, then select `Add` to add and enable the grouped script. diff --git a/builder/source/banner/index.ts b/builder/source/banner/index.ts index c0fbdd9..c4822b0 100644 --- a/builder/source/banner/index.ts +++ b/builder/source/banner/index.ts @@ -14,6 +14,9 @@ export interface BannerOptions { } export function CreateBanner(Options: BannerOptions): string { + const Domains = [...Options.Domains].filter(Domain => !Domain.endsWith('.*')) + if (Domains.length === 0) throw new Error('No concrete domains available for userscript matching.') + let BannerString: string = '// ==UserScript==\n' BannerString += `// @name ${Options.BuildType === 'production' ? Options.Name : Options.Name + ' (Development)'}\n` BannerString += '//\n' @@ -38,11 +41,9 @@ export function CreateBanner(Options: BannerOptions): string { } BannerString += '//\n' - for (const Domain of Options.Domains) { - const WildcardSuffix = Domain.endsWith('.*') - const Host = (WildcardSuffix ? Domain.slice(0, -2) : Domain).replaceAll(/[.*+?^${}()|[\]\\/]/g, '\\$&') - const Suffix = WildcardSuffix ? '(?:\\.[^.\\/:?#@]+)+' : '' - BannerString += `// @include /^https?:\\/\\/(?:[^.\\/:?#@]+\\.)*${Host}${Suffix}\\.?(?::[0-9]+)?(?:[\\/?#]|$)/\n` + for (const Domain of Domains) { + BannerString += `// @match *://${Domain}/*\n` + BannerString += `// @match *://*.${Domain}/*\n` } BannerString += '// ==/UserScript==\n\n' return BannerString diff --git a/builder/source/build-grouped.ts b/builder/source/build-grouped.ts index bf54756..44896ec 100644 --- a/builder/source/build-grouped.ts +++ b/builder/source/build-grouped.ts @@ -20,10 +20,16 @@ export class GroupedBuild extends Build { private GetGroups(): Map, Set> { const Groups = this.FetchedDomains.get('Groups') if (!Groups) throw new Error('Domain groups are not initialized.') - return new Map([...Groups].map(([Names, Domains]) => [ - new Set([...Names].map(Domain => Domain.endsWith('.*') ? Domain.slice(0, -2) + '.tld' : Domain)), - Domains - ])) + const Result = new Map, Set>() + for (const [Names, Domains] of Groups) { + const MatchingDomains = new Set([...Domains].filter(Domain => !Domain.endsWith('.*'))) + if (MatchingDomains.size === 0) continue + Result.set( + new Set([...Names].map(Domain => Domain.endsWith('.*') ? Domain.slice(0, -2) + '.tld' : Domain)), + MatchingDomains + ) + } + return Result } async MakeGroupMetadata() { @@ -50,6 +56,7 @@ export class GroupedBuild extends Build { } } + await Fs.promises.mkdir(`${this.ProjectRoot}/dist/grouped`, { recursive: true }) await Fs.promises.writeFile(`${this.ProjectRoot}/dist/grouped/metadata.json`, JSON.stringify(MetadataObj, null, 2), 'utf-8') } diff --git a/libs/test/references/banner.test.ts b/libs/test/references/banner.test.ts index 7a03a82..f34810b 100644 --- a/libs/test/references/banner.test.ts +++ b/libs/test/references/banner.test.ts @@ -1,29 +1,31 @@ import { expect, test } from 'vitest' -import { Script } from 'node:vm' import { CreateBanner } from '../../../builder/source/banner/index.js' -test('banner includes only collected domains and explicit wildcard suffixes', () => { +test('banner uses match rules only for collected concrete domains', () => { const Url = new URL('https://example.com/') const Banner = CreateBanner({ Version: '1', BuildType: 'production', Domains: new Set(['example.com', 'example.net', 'wild.*']), Author: 'author', Name: 'name', Namespace: 'namespace', HomepageURL: Url, SupportURL: Url, UpdateURL: Url, DownloadURL: Url, License: 'license', Description: { en: 'description' } }) - const Patterns = [...Banner.matchAll(/^\/\/ @include\s+\/(.*)\/$/gm)].map(Match => new RegExp(Match[1])) - for (const Match of Banner.matchAll(/^\/\/ @include\s+(\/.*\/)$/gm)) { - expect(() => new Script(Match[1])).not.toThrow() - } - const Matches = (Input: string) => Patterns.some(Pattern => Pattern.test(Input)) + expect(Banner.match(/^\/\/ @match\s+.*$/gm)).toEqual([ + '// @match *://example.com/*', + '// @match *://*.example.com/*', + '// @match *://example.net/*', + '// @match *://*.example.net/*' + ]) + expect(Banner).not.toContain('@include') + expect(Banner).not.toContain('wild.*') +}) - expect(Patterns).toHaveLength(3) - expect(Matches('https://sub.example.com:8443/path')).toBe(true) - expect(Matches('http://example.net/?query')).toBe(true) - expect(Matches('https://example.org/')).toBe(false) - expect(Matches('https://notexample.com/')).toBe(false) - expect(Matches('https://example.com.evil/')).toBe(false) - expect(Matches('https://wild.com@evil.test/')).toBe(false) - expect(Matches('https://example.com./')).toBe(true) - expect(Matches('ftp://example.com/')).toBe(false) - expect(Matches('https://sub.wild.co.uk/')).toBe(true) - expect(Matches('https://wild.com.evil/')).toBe(true) +test('banner rejects an empty or wildcard-only matching set', () => { + const Url = new URL('https://example.com/') + const Options = { + Version: '1', BuildType: 'production' as const, Domains: new Set(), + Author: 'author', Name: 'name', Namespace: 'namespace', HomepageURL: Url, SupportURL: Url, + UpdateURL: Url, DownloadURL: Url, License: 'license', Description: { en: 'description' } + } + expect(() => CreateBanner(Options)).toThrow('No concrete domains') + Options.Domains.add('wild.*') + expect(() => CreateBanner(Options)).toThrow('No concrete domains') }) \ No newline at end of file diff --git a/libs/test/references/group-metadata.test.ts b/libs/test/references/group-metadata.test.ts index fb928f5..cf3abda 100644 --- a/libs/test/references/group-metadata.test.ts +++ b/libs/test/references/group-metadata.test.ts @@ -1,5 +1,5 @@ import { createHash } from 'node:crypto' -import { mkdtemp, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises' +import { access, mkdtemp, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { expect, test } from 'vitest' @@ -30,7 +30,7 @@ test('group metadata maps collected domains without expanding wildcard suffixes' await mkdir(Directory, { recursive: true }) const Content = 'grouped userscript' await writeFile(join(Directory, 'tinyShield-example.tld.user.js'), Content) - const Builder = new TestGroupedBuild(Root, new Set(['example.com', 'example.net'])) + const Builder = new TestGroupedBuild(Root, new Set(['example.*', 'example.com', 'example.net', 'wild.*'])) await Builder.MakeGroupMetadata() const Metadata: unknown = JSON.parse(await readFile(join(Root, 'dist/grouped/metadata.json'), 'utf8')) expect(Metadata).toEqual({ @@ -53,19 +53,20 @@ test('standard and grouped workers build with collected matching rules', async ( await symlink(new URL('../../../builder/', import.meta.url).pathname, join(Root, 'builder'), 'dir') await symlink(new URL('../../../userscript/', import.meta.url).pathname, join(Root, 'userscript'), 'dir') await symlink(new URL('../../../package.json', import.meta.url).pathname, join(Root, 'package.json')) - const Domains = new Set(['example.com', 'example.net']) + const Domains = new Set(['example.*', 'example.com', 'example.net', 'wild.*']) await new TestStandardBuild(Root, Domains).Build() const Builder = new TestGroupedBuild(Root, Domains) await Builder.Build() await Builder.MakeGroupMetadata() const Standard = await readFile(join(Root, 'dist/tinyShield.dev.user.js'), 'utf8') const Grouped = await readFile(join(Root, 'dist/grouped/e/tinyShield-example.tld.user.js'), 'utf8') + await expect(access(join(Root, 'dist/grouped/w/tinyShield-wild.tld.user.js'))).rejects.toThrow() for (const Output of [Standard, Grouped]) { - expect(Output).toContain('@include /^https?:') - expect(Output).toContain('example\\.com') - expect(Output).toContain('example\\.net') - expect(Output).not.toContain('example\\.org') - expect(Output).not.toContain('@match') + expect(Output).toContain('@match *://*.example.com/*') + expect(Output).toContain('@match *://*.example.net/*') + expect(Output).not.toContain('example.org') + expect(Output).not.toContain('wild.*') + expect(Output).not.toContain('@include') } } finally { await rm(Root, { recursive: true, force: true })