Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: release

# Un tag vX.Y.Z publie la release GitHub (roue et archive), puis le paquet sur pypi.org par Trusted Publishing, sans
# jeton : le mainteneur déclare ce workflow comme éditeur de confiance du projet accesspath (environnement « pypi »)
# et pose la variable PYPI_PUBLISH à true.
on:
push:
tags: ["v*.*.*"]

permissions:
contents: read

jobs:
construire:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: "3.13"
- name: la version du tag est celle du paquet
run: |
v=$(python -c "import tomllib;print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
test "v$v" = "$GITHUB_REF_NAME" || { echo "tag $GITHUB_REF_NAME, paquet $v"; exit 1; }
- run: python -m pip install build && python -m build --outdir dist
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: paquet
path: dist/

release:
needs: construire
runs-on: ubuntu-24.04
permissions:
contents: write
steps:
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: paquet
path: dist/
- env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: gh release create "$GITHUB_REF_NAME" dist/* -R "$GITHUB_REPOSITORY" --title "accesspath ${GITHUB_REF_NAME#v}" --generate-notes

pypi:
needs: release
if: vars.PYPI_PUBLISH == 'true'
runs-on: ubuntu-24.04
environment: pypi
permissions:
id-token: write
steps:
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: paquet
path: dist/
- uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3
66 changes: 66 additions & 0 deletions .github/workflows/testpypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: testpypi

# Répétition de la publication sur test.pypi.org, lancée à la main. Le paquet porte la version X.Y.Z.dev<numéro du
# run> : un numéro ne sert qu'une fois sur un index. Puis il s'installe depuis TestPyPI (ses dépendances depuis PyPI)
# et sa commande répond.
on:
workflow_dispatch:

permissions:
contents: read

jobs:
construire:
runs-on: ubuntu-24.04
outputs:
version: ${{ steps.v.outputs.version }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: "3.13"
- name: version de répétition
id: v
run: |
v=$(python -c "import tomllib;print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])").dev${{ github.run_id }}
sed -i "s/^version = .*/version = \"$v\"/" pyproject.toml
echo "version=$v" >> "$GITHUB_OUTPUT"
- run: python -m pip install build && python -m build --outdir dist
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: paquet
path: dist/

publier:
needs: construire
runs-on: ubuntu-24.04
environment: testpypi
permissions:
id-token: write
steps:
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: paquet
path: dist/
- uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3
with:
repository-url: https://test.pypi.org/legacy/

essayer:
needs: [construire, publier]
runs-on: ubuntu-24.04
steps:
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: "3.13"
- name: installer depuis TestPyPI, la commande répond
run: |
python -m venv /tmp/v
for i in 1 2 3 4 5 6; do
/tmp/v/bin/pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ \
"accesspath[tls]==${{ needs.construire.outputs.version }}" && break
sleep 30
done
/tmp/v/bin/accesspath sap --help
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# accesspath

[![CI](https://github.com/lfoletti/accesspath/actions/workflows/ci.yml/badge.svg)](https://github.com/lfoletti/accesspath/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/github/license/lfoletti/accesspath)](LICENSE)
[![License: MIT](https://img.shields.io/github/license/lfoletti/accesspath)](https://github.com/lfoletti/accesspath/blob/main/LICENSE)
[![Release](https://img.shields.io/github/v/release/lfoletti/accesspath)](https://github.com/lfoletti/accesspath/releases)

accesspath audits a website for two readers. For a **collector**, it finds the simplest way to read the content: a feed, an API, a sitemap, static HTML, or a browser as the last step. For the **site owner**, it shows what the site exposes to machines and how it is protected, with recommendations. It is the engine of the Claude Code skill `/sap`.
Expand All @@ -12,7 +12,7 @@ Use accesspath before you write a collector, or when a source stops you: a 403,

accesspath tries the tiers from the base to the top and keeps the first tier that works. Tier 1 is an API or a feed: RSS, Atom, JSON, GraphQL, sitemap. Tier 4 is a single-page application that needs Chrome. Tier 5 is a site that stops Chrome too: a person, an archive copy or the owner must help.

![The five access tiers and what each tier means for a collector](docs/img/tiers.svg)
![The five access tiers and what each tier means for a collector](https://raw.githubusercontent.com/lfoletti/accesspath/main/docs/img/tiers.svg)

## Owner report

Expand Down Expand Up @@ -52,7 +52,7 @@ python -m accesspath sap owner https://www.example.org # owner report: expos
python -m accesspath sap probe https://www.example.org --save --skeleton # keep the profile, write a collector skeleton
```

Without the skill, install with `pip install -e ".[inspect,tls]"`; the command `accesspath` is the same as `python -m accesspath`.
Without the skill, install from PyPI with `pip install "accesspath[inspect,tls]"` (or `pip install -e ".[inspect,tls]"` in a clone); the command `accesspath` is the same as `python -m accesspath`.

You need Python 3.11 or later. `inspect` needs Google Chrome.

Expand Down Expand Up @@ -80,7 +80,7 @@ Without `--save`, accesspath writes nothing. Use this mode from other projects.
- **Third-party copies.** When no door opens, `probe` asks the Wayback Machine and Common Crawl for their copies of the site.
- **Human step.** Some sites need a person: a challenge, a login, a consent page. `inspect` shows "✋ Geste humain requis" and the command to use. With `--save`, the step goes into a queue.
- **Ghost mode.** Record a browser session as HAR. `ghost --har` finds the data calls and their pagination. `ghost --replay` sends these calls again without a browser. Secrets come from environment variables and accesspath never writes them to disk.
- **Patterns and templates.** Collector projects and studies leave reusable knowledge: content URL families, CMS profiles, ranked templates (the winning combinations of choices, with their variants), known platforms (commercial registers, commune CMS, site CMS), and patterns learned after each campaign. See [docs/patterns.md](docs/patterns.md).
- **Patterns and templates.** Collector projects and studies leave reusable knowledge: content URL families, CMS profiles, ranked templates (the winning combinations of choices, with their variants), known platforms (commercial registers, commune CMS, site CMS), and patterns learned after each campaign. See [docs/patterns.md](https://github.com/lfoletti/accesspath/blob/main/docs/patterns.md).
- **Disputed access.** accesspath does not refuse a disputed access, for example the replay of a logged-in session. The report shows a warning.

## Safety
Expand All @@ -104,11 +104,11 @@ The tests do not use the internet. They use local fixtures and servers on 127.0.

## More information

- [docs/REFERENCE.md](docs/REFERENCE.md): installation from the lock files, configuration, data, collector projects, CI.
- [docs/patterns.md](docs/patterns.md): patterns, ranked templates, known platforms, learned patterns, and how the project uses them.
- [docs/VEILLE.md](docs/VEILLE.md): how the technology watch changes the code.
- [skill/SKILL.md](skill/SKILL.md): the `/sap` skill.
- [docs/REFERENCE.md](https://github.com/lfoletti/accesspath/blob/main/docs/REFERENCE.md): installation from the lock files, configuration, data, collector projects, CI.
- [docs/patterns.md](https://github.com/lfoletti/accesspath/blob/main/docs/patterns.md): patterns, ranked templates, known platforms, learned patterns, and how the project uses them.
- [docs/VEILLE.md](https://github.com/lfoletti/accesspath/blob/main/docs/VEILLE.md): how the technology watch changes the code.
- [skill/SKILL.md](https://github.com/lfoletti/accesspath/blob/main/skill/SKILL.md): the `/sap` skill.

## License

MIT. See [LICENSE](LICENSE). Contributions: [CONTRIBUTING.md](CONTRIBUTING.md). Security: [SECURITY.md](SECURITY.md).
MIT. See [LICENSE](https://github.com/lfoletti/accesspath/blob/main/LICENSE). Contributions: [CONTRIBUTING.md](https://github.com/lfoletti/accesspath/blob/main/CONTRIBUTING.md). Security: [SECURITY.md](https://github.com/lfoletti/accesspath/blob/main/SECURITY.md).
9 changes: 7 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ build-backend = "hatchling.build"

[project]
name = "accesspath"
version = "0.1.0"
version = "0.1.1"
description = "Website access audit (Site Architecture Profile): the simplest way that works to read a site, and what the site exposes to machines."
readme = "README.md"
requires-python = ">=3.11"
authors = [{ name = "Lauro Foletti" }]
authors = [{ name = "The accesspath maintainers" }]
license = "MIT"
license-files = ["LICENSE"]
dependencies = [
Expand Down Expand Up @@ -43,6 +43,11 @@ dev = [
"pip-audit",
]

[project.urls]
Source = "https://github.com/lfoletti/accesspath"
Issues = "https://github.com/lfoletti/accesspath/issues"
Releases = "https://github.com/lfoletti/accesspath/releases"

[project.scripts]
accesspath = "accesspath.cli:main"

Expand Down
Loading