[jwrosewell]

The problem

The cache header detail of the web integration specification lists the headers the client-side JavaScript endpoint (/51Degrees.core.js) and the JSON endpoint (/json) MUST send, being Cache-Control: max-age=1800 private, Vary, ETag, Content-Type and Content-Length, and says a request whose If-None-Match matches the ETag is not sent the content again.

Python has no web integration package. On rc the client-side endpoints are written in each Flask example, and none of them sets any cache header, so a browser fetches the script and the JSON in full on every page. The places that build them are these.

• /json and /51Degrees.core.js in device-detection/.../cloud/gettingstarted_web/app.py and in the on-premise copy beside it.
• /51Degrees.core.js and /json in the pipeline's mixed cloud example, .../cloud/mixed/gettingstarted_web/app.py.
• /json in location/examples/cloud/web.py and in pipeline/fiftyone_pipeline_core/examples/client_side_evidence_custom_flow_element.py.

The one shared module is fiftyone_pipeline_core/web.py, which the examples already use for webevidence and set_response_header.

What the specification and RFC 9110 require

RFC 9110 (HTTP Semantics, STD 97) is current, as its RFC Editor record lists nothing that obsoletes or updates it.

1. The ETag is weak. It is a hash of the Evidence alone, so a data file update can change the JSON or the JavaScript for the same Evidence while the ETag stays the same. RFC 9110 section 8.8.1 defines a strong validator as one that "changes value whenever a change occurs to the representation data that would be observable in the content of a 200 (OK) response to GET", and section 8.8.3 says an origin server whose entity tag does not satisfy that "MUST mark the entity tag as weak by prefixing its opaque value with "W/" (case-sensitive)". The open specification draft specifications#115 writes this into the header table as W/"<hash of ALL the Evidence values in Evidence filter key>".
2. The hash covers only the Evidence the request supplied, in the keys the Pipeline's elements accept, before any Flow Element adds a value. The Sequence Element gives a request with no session id a new one while processing, so a hash taken after processing differs on every request and a revalidation never matches. The open specification draft specifications#121 says so.
3. The same Evidence gives the same ETag in every process, as the specification requires, because a browser revalidates against whichever server answers. Python seeds hash() of a string afresh in each process, so it is not used.
4. If-None-Match is compared weakly. RFC 9110 section 13.1.2 says "A recipient MUST use the weak comparison function when comparing entity tags for If-None-Match", and the weak comparison of section 8.8.3.2 matches two tags "if their opaque-tags match character-by-character, regardless of either or both being tagged as "weak"". The field is "*" / #entity-tag, and step 3 of the evaluation in section 13.1.2 says that a value which is neither "*" nor a list of entity tags leaves the condition true, so it matches nothing. A list allows optional spaces and tabs around its commas and empty elements (section 5.6.1.2), the opaque tag holds no space or double quote (etagc in section 8.8.3), and W/ is case sensitive. Section 13.1.2 also says a list holding "*" and other values "is syntactically invalid".
5. A match gets 304 to GET or HEAD and 412 to any other method. Section 13.1.2 says the server "MUST NOT perform the requested method if the condition evaluates to false" and must answer "the 304 (Not Modified) status code if the request method is GET or HEAD or b) the 412 (Precondition Failed) status code for all other request methods". Section 15.4.5 says a 304 MUST carry the Cache-Control, ETag and Vary a 200 would carry. The specification gives the same answer, and the open draft specifications#81 adds the 15.4.5 headers.
6. A failed response carries Cache-Control: private, no-cache and no ETag, when processing recorded errors in the Flow Data errors collection (specification lines 248 to 261), so a browser has nothing to revalidate the faulty response with and asks again in full once processing works.
7. The max age SHOULD be configurable (line 278).

The change

fiftyone_pipeline_core.web (web.py) gains javascript_response(pipeline, request, max_age_seconds=1800) and json_response(pipeline, request, max_age_seconds=1800), which process the request and answer with the JavaScriptBuilder's JavaScript or the JSONBundler's JSON, and the constant CLIENT_SIDE_MAX_AGE_SECONDS. Everything else is private to the module.

• The ETag is W/"<32 hex digits>", the first 128 bits of a SHA-256 hash of the name and value of each piece of the request's Evidence that one of the pipeline's elements accepts, in name order. It is worked out from what webevidence() read from the request, before processing.
• The headers a browser adds when it revalidates or reloads (If-None-Match, If-Modified-Since, If-Match, If-Unmodified-Since, If-Range, Cache-Control, Pragma) and the web request value server.http-request are left out of the ETag and of Vary. FlowElement's default filter accepts every key, so an element that does not set its own filter would otherwise put If-None-Match into the ETag, and a revalidation would never match.
• If-None-Match is read by a small parser that follows the grammar above, and matches when the field is "*" or when one tag in a valid list has the ETag's opaque tag.
• A matching request is not processed. A GET or HEAD gets a 304 with Cache-Control, Vary and ETag and no Content-Type, and any other method, being the POST to the JSON endpoint, gets 412 Precondition Failed with no content and no ETag. So a browser holding a good copy keeps it while processing fails. Python's endpoints set no CORS headers on a 200, so the 304 has none either.
• Otherwise the response carries Content-Type (application/x-javascript or application/json), Content-Length from Flask, and Cache-Control: max-age=1800, private with the ETag, or Cache-Control: private, no-cache and no ETag when flowdata.errors is not empty.
• Vary names the request's headers that one of the pipeline's elements accepts, for example Host, User-Agent.
• The flow data is closed when the response is built.

The five example routes call these two functions, and their comments say what the call does.

The other languages follow the same rules. The PHP draft 51degrees-php#48 adds the same functions to Utils, and Node on rc sends the weak ETag from helpers.js.

For the release notes

• The client-side JavaScript and JSON responses of the Python examples now carry Cache-Control, Vary, a weak ETag, Content-Type and Content-Length, so a browser keeps the script for 30 minutes and revalidates it with a 304 rather than fetching it in full on every page.
• fiftyone_pipeline_core.web gains javascript_response() and json_response(), which answer the client-side endpoints of a Flask application with these headers. The max age is the optional max_age_seconds argument.
• A request whose If-None-Match matches gets a 304 to GET or HEAD and a 412 to any other method, as RFC 9110 section 13.1.2 requires. A response to a request whose processing recorded errors carries Cache-Control: private, no-cache and no ETag.

Tests

pipeline/fiftyone_pipeline_core/tests/test_web_client_side.py has 16 tests and pipeline/fiftyone_pipeline_cloudrequestengine/examples/tests/test_cloud_mixed_web_offline.py runs the mixed example against a stand-in for the cloud service. Neither needs a key or the network. Between them they cover the following.

• The weak tag format, W/"<32 hex digits>", on the JavaScript and the JSON.
• The same tag from two pipelines, and from two separate Python processes started with PYTHONHASHSEED set to 1 and to 2.
• The same tag for two requests whose scripts carry different session ids from the Sequence Element.
• 26 If-None-Match values on a GET and on a POST, being the tag as sent, its strong form, *, lists, a comma inside a tag, a tab or no space around a comma, empty list elements, a tag then a non-tag, a non-tag then a tag, two tags with no comma, a space inside the quotes, w/ in lower case, W/ followed by a space, * inside a list, an unquoted tag and an unterminated one, and two If-None-Match fields read as one list.
• 304 to GET and HEAD with the ETag, Cache-Control and Vary of the good response, and 412 to POST with no content and no ETag.
• A failed response with private, no-cache and no ETag on both endpoints, a matching tag answered while processing fails (304 to the script, 412 to the JSON), and a failed response replaced once processing works.
• The max_age_seconds argument on both endpoints.

The tests were committed before each change, so each pair shows red then green. e2cabee holds the weak tag and If-None-Match tests and a7e05a4 the change, then 83c4722 the max age test and bef273b the change. rc at a474f04 was merged in first, at ccd87d7. e0799ee then makes the session id test read the minified script.

Test results

The suites have not yet run on this head, because the machine this was written on had too little free memory to run them safely, so the results of the CI build on this head are to be added here. These commands run the tests above from a clone of this branch.

git submodule update --init pipeline/fiftyone_pipeline_core/src/fiftyone_pipeline_core/js_templates
python -m venv venv
venv/bin/pip install pytest flask parameterized requests_mock -e pipeline/fiftyone_pipeline_core -e pipeline/fiftyone_pipeline_engines -e pipeline/fiftyone_pipeline_cloudrequestengine -e pipeline/fiftyone_pipeline_cloudrequestengine/examples
cd pipeline/fiftyone_pipeline_core
../../venv/bin/python -m pytest tests/test_web_client_side.py tests/test_web.py tests/test_javascriptbuilder.py
cd ../fiftyone_pipeline_cloudrequestengine/examples
../../../venv/bin/python -m pytest tests/test_cloud_mixed_web_offline.py

Earlier runs

These ran on older heads of this branch, before the weak ETag and the stricter If-None-Match reading.

• Red run 42394701136 on rc at 0507eef with only the first tests added, and green run 42395023618 at b1a6754, 24 passed in the core tests and 18 in the examples on Ubuntu and Windows with Python 3.12.
• Red run 42416498061 and green run 42416617410 for the 412 to a POST, 25 and 18 passed.
• Red run 42418120887 and green run 42418163568 for leaving the web request value out of the ETag, 26 and 18 passed.

Full build

The full build and test matrix passed every job other than macOS on head e0799ee in run 42695092282 on 9 November 2026, 11 jobs succeeded, run as a dry run so nothing merged. macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.