TASK-047: fire request_received and body_chunk hooks (pre-handler short-circuit)

Wires the two pre-routing, pre-handler hook phases that observe the
inbound request. Both support short-circuit via
hook_action::respond_with(...): a non-pass return populates
mr->response_, sets the new mr->skip_handler flag, and routes through
finalize_answer's new skip-handler branch. The body_chunk short-circuit
also destroys any in-flight MHD_PostProcessor so its 32 KB buffer is
freed (ASan-verified).

Closes #273 (early 413 on oversize body) — demonstrated by
examples/early_413.cpp. Partially addresses #272 (observation half of
delayed body processing).

Acceptance tests:
- hooks_body_chunk_ctx_shape: compile-time pin for the TASK-045 ctx
  shapes (mutable http_request*, std::span<const std::byte> chunk,
  std::uint64_t offset, bool is_final).
- hooks_request_received_short_circuit: 413 hook aborts the upload
  before any body bytes flow; downstream body_chunk hook never fires.
  Second sub-test pins the non-short-circuit path through to 200.
- hooks_body_chunk_observes_progress: accumulated bytes equal body
  size; offsets are monotonic and start at zero.
- hooks_body_chunk_short_circuit_no_leak: form-urlencoded POST forces
  MHD_PostProcessor allocation; first-chunk short-circuit must free
  it (sentinel for ASan).

Implementation:
- New fire_short_circuit_hooks_for_phase template in hook_handle.cpp
  (sibling to TASK-046's fire_hooks_for_phase) returns
  std::optional<http_response>; same shared_lock-snapshot-then-iterate
  pattern, throwing hook treated as pass per DR-009 §5.2.
- New any_hooks_-gated firing sites in
  webserver_impl::requests_answer_first_step (request_received) and
  requests_answer_second_step (body_chunk).
- modded_request gains a skip_handler bool; finalize_answer gains an
  early-exit branch that routes directly to
  materialize_and_queue_response when set. Both pipeline functions
  also re-check the flag so a request_received short-circuit
  suppresses body_chunk firings on subsequent MHD callbacks.
- File-size mitigation: the two firing-site insertions pushed
  src/detail/webserver_request.cpp over the 500-LOC ceiling; mirrored
  the TASK-046 split pattern by carving
  src/detail/webserver_body_pipeline.cpp out (hosts both pipeline
  functions plus two anon-ns helpers that keep
  requests_answer_second_step at CCN <= 10).
- hook_context.hpp doc comments now warn that body_chunk fires from
  arbitrary MHD worker threads at arbitrary granularity (no I/O, no
  per-chunk allocation; the chunk span aliases MHD-owned memory).
- hooks_no_firing narrowed to drop request_received and body_chunk
  from its "must observe zero" set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: etr

examples/Makefile.am

@@ -19,7 +19,7 @@
 LDADD = $(top_builddir)/src/libhttpserver.la
 AM_CPPFLAGS = -I$(top_srcdir)/src -I$(top_srcdir)/src/httpserver/
 METASOURCES = AUTO
-noinst_PROGRAMS = hello_world shared_state service custom_error allowing_disallowing_methods handlers hello_with_get_arg args_processing setting_headers custom_access_log minimal_https minimal_file_response minimal_deferred url_registration minimal_ip_ban banned_ip_log benchmark_select benchmark_threads benchmark_nodelay deferred_with_accumulator file_upload file_upload_with_callback empty_response_example iovec_response_example pipe_response_example daemon_info external_event_loop turbo_mode binary_buffer_response
+noinst_PROGRAMS = hello_world shared_state service custom_error allowing_disallowing_methods handlers hello_with_get_arg args_processing setting_headers custom_access_log minimal_https minimal_file_response minimal_deferred url_registration minimal_ip_ban banned_ip_log early_413 benchmark_select benchmark_threads benchmark_nodelay deferred_with_accumulator file_upload file_upload_with_callback empty_response_example iovec_response_example pipe_response_example daemon_info external_event_loop turbo_mode binary_buffer_response
 
 hello_world_SOURCES = hello_world.cpp
 shared_state_SOURCES = shared_state.cpp
@@ -38,6 +38,7 @@ deferred_with_accumulator_SOURCES = deferred_with_accumulator.cpp
 url_registration_SOURCES = url_registration.cpp
 minimal_ip_ban_SOURCES = minimal_ip_ban.cpp
 banned_ip_log_SOURCES = banned_ip_log.cpp
+early_413_SOURCES = early_413.cpp
 benchmark_select_SOURCES = benchmark_select.cpp
 benchmark_threads_SOURCES = benchmark_threads.cpp
 benchmark_nodelay_SOURCES = benchmark_nodelay.cpp

examples/early_413.cpp

@@ -0,0 +1,54 @@
+// Demonstrates the solution to issue #273: short-circuit large uploads
+// with a 413 BEFORE any body bytes are consumed.
+//
+// Register a `request_received` hook that inspects Content-Length; if
+// it exceeds the configured cap, return a respond_with(413) action.
+// libhttpserver aborts the upload -- the resource handler is not
+// invoked and the body bytes never cross the daemon's I/O boundary.
+
+#include <cstddef>
+#include <functional>
+#include <memory>
+#include <string>
+
+#include <httpserver.hpp>
+
+namespace hs = httpserver;
+
+namespace {
+constexpr std::size_t kMaxUploadBytes = 1 * 1024 * 1024;   // 1 MB
+}  // namespace
+
+class upload_resource : public hs::http_resource {
+ public:
+    hs::http_response render_post(const hs::http_request&) override {
+        return hs::http_response::string("UPLOAD OK");
+    }
+};
+
+int main() {
+    hs::webserver ws{hs::create_webserver(8080)};
+
+    auto h = ws.add_hook(hs::hook_phase::request_received,
+        std::function<hs::hook_action(hs::request_received_ctx&)>(
+            [](hs::request_received_ctx& ctx) {
+                std::string cl{ctx.request->get_header("Content-Length")};
+                if (cl.empty()) return hs::hook_action::pass();
+                try {
+                    if (std::stoull(cl) > kMaxUploadBytes) {
+                        return hs::hook_action::respond_with(
+                            hs::http_response::empty().with_status(413));
+                    }
+                } catch (...) {
+                    // Malformed Content-Length: let the normal pipeline
+                    // produce a 400 elsewhere.
+                }
+                return hs::hook_action::pass();
+            }));
+
+    auto resource = std::make_shared<upload_resource>();
+    ws.register_path("/upload", resource);
+
+    ws.start(true);   // blocking
+    return 0;
+}

specs/tasks/M5-routing-lifecycle/TASK-047.md

@@ -8,11 +8,11 @@
 Wire the two pre-routing, pre-handler phases that observe the inbound request. Both support short-circuit (`hook_action::respond_with(...)`). Closes feature request #273 (early 413 on oversize body); partially addresses #272 (delayed body processing — the `body_chunk` half).
 
 **Action Items:**
-- [ ] Fire `request_received` from `webserver_impl::requests_answer_first_step` (`webserver.cpp:2010`) — after `mr->dhr.reset(new http_request(...))` so the `http_request` is populated, before the body read decision. Context: `request_received_ctx { http_request& req }`. (Mutable ref so a hook can adjust per-request state like content-size limit if a follow-up adds that knob.)
-- [ ] Fire `body_chunk` from `webserver_impl::requests_answer_second_step` (`webserver.cpp:2034`) — once per chunk delivered by MHD, before the chunk is appended to `mr->dhr` content / post-processor. Context: `body_chunk_ctx { http_request& req; std::span<const char> bytes; std::size_t total_bytes_received; }`.
-- [ ] Short-circuit handling: if any hook returns `hook_action::respond_with(r)`, stash `r` into `mr->response_`, mark `mr` as "skip-to-finalize", and let the existing finalize path queue it. The resource handler is not invoked. For `body_chunk` specifically, also drop any in-flight post-processor (`MHD_destroy_post_processor`) to free its buffer.
-- [ ] Document in the public header that `body_chunk` fires from MHD worker threads at arbitrary granularity — chunks may be byte-level on slow networks. Hooks must be cheap (no I/O, no allocation in the hot path) or risk back-pressuring the connection.
-- [ ] Same `fire_*` pattern as TASK-046 (shared_lock → copy → release → iterate).
+- [x] Fire `request_received` from `webserver_impl::requests_answer_first_step` (relocated to `src/detail/webserver_body_pipeline.cpp` by the TASK-047 split) — after `mr->dhr.reset(new http_request(...))` so the `http_request` is populated, before the body read decision. Context: `request_received_ctx { http_request* request; steady_clock::time_point received_at; }` (the TASK-045 shape).
+- [x] Fire `body_chunk` from `webserver_impl::requests_answer_second_step` (same file) — once per chunk delivered by MHD, before the chunk is appended to `mr->dhr` content / post-processor. Context: `body_chunk_ctx { http_request* request; std::span<const std::byte> chunk; std::uint64_t offset; bool is_final; }` (the TASK-045 shape; `offset` carries the running total of bytes already buffered).
+- [x] Short-circuit handling: a non-pass hook return stashes the response into `mr->response_`, sets `mr->skip_handler = true`, and lets the existing finalize path queue it (new `if (mr->skip_handler)` early-return in `finalize_answer`). The resource handler is not invoked. The `body_chunk` short-circuit also calls `MHD_destroy_post_processor(mr->pp)` to free the 32 KB buffer so ASan stays clean.
+- [x] Documented in the public header (`src/httpserver/hook_context.hpp`) that `body_chunk` fires from MHD worker threads at arbitrary granularity and hooks must be cheap (no blocking I/O, no per-chunk heap allocation).
+- [x] Same `fire_*` pattern as TASK-046 (shared_lock → reserve(8) → copy → release → iterate → try/catch → `log_dispatch_error`). The `hook_action`-returning variant `fire_short_circuit_hooks_for_phase` lives in `hook_handle.cpp` next to the void variant.
 
 **Dependencies:**
 - Blocked by: TASK-045
@@ -29,4 +29,4 @@ Wire the two pre-routing, pre-handler phases that observe the inbound request. B
 **Related Decisions:** DR-012, §4.10
 **Resolves:** Issue #273. Partially addresses issue #272 (observation half only — body-buffer steal remains a v2.1 candidate).
 
-**Status:** Not Started
+**Status:** Done

src/Makefile.am

@@ -25,7 +25,7 @@ lib_LTLIBRARIES = libhttpserver.la
 # builds. The WS-off branch in websocket_handler.cpp provides stub
 # definitions (every member throws feature_unavailable except is_valid()
 # which returns false).
-libhttpserver_la_SOURCES = string_utilities.cpp webserver.cpp http_utils.cpp file_info.cpp http_request.cpp http_request_auth.cpp http_response.cpp create_webserver.cpp create_test_request.cpp websocket_handler.cpp hook_handle.cpp detail/http_endpoint.cpp detail/body.cpp detail/ip_representation.cpp detail/http_request_impl.cpp detail/http_request_impl_tls.cpp detail/webserver_setup.cpp detail/webserver_register.cpp detail/webserver_routes.cpp detail/webserver_callbacks.cpp detail/webserver_callbacks_lifecycle.cpp detail/webserver_websocket.cpp detail/webserver_dispatch.cpp detail/webserver_request.cpp
+libhttpserver_la_SOURCES = string_utilities.cpp webserver.cpp http_utils.cpp file_info.cpp http_request.cpp http_request_auth.cpp http_response.cpp create_webserver.cpp create_test_request.cpp websocket_handler.cpp hook_handle.cpp detail/http_endpoint.cpp detail/body.cpp detail/ip_representation.cpp detail/http_request_impl.cpp detail/http_request_impl_tls.cpp detail/webserver_setup.cpp detail/webserver_register.cpp detail/webserver_routes.cpp detail/webserver_callbacks.cpp detail/webserver_callbacks_lifecycle.cpp detail/webserver_websocket.cpp detail/webserver_dispatch.cpp detail/webserver_request.cpp detail/webserver_body_pipeline.cpp
 # noinst_HEADERS: shipped in the tarball but NEVER installed under $prefix/include.
 # Detail headers (httpserver/detail/*.hpp) live here so they cannot leak to
 # downstream consumers — the public surface comes in through <httpserver.hpp>.

src/detail/webserver_body_pipeline.cpp

@@ -0,0 +1,207 @@
+/*
+     This file is part of libhttpserver
+     Copyright (C) 2011-2026 Sebastiano Merlino
+
+     This library is free software; you can redistribute it and/or
+     modify it under the terms of the GNU Lesser General Public
+     License as published by the Free Software Foundation; either
+     version 2.1 of the License, or (at your option) any later version.
+
+     This library is distributed in the hope that it will be useful,
+     but WITHOUT ANY WARRANTY; without even the implied warranty of
+     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+     Lesser General Public License for more details.
+
+     You should have received a copy of the GNU Lesser General Public
+     License along with this library; if not, write to the Free Software
+     Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301
+     USA
+*/
+
+// TASK-047 -- The two body-pipeline stages
+// (requests_answer_first_step / requests_answer_second_step) extracted
+// from webserver_request.cpp to keep that TU under the 500-LOC ceiling
+// after adding the request_received and body_chunk firing sites
+// (matches the TASK-046 split pattern that carved
+// webserver_callbacks_lifecycle.cpp out of webserver_callbacks.cpp).
+//
+// Also hosts the small anon-ns helper that wraps the body_chunk firing
+// site, both to keep the second-step orchestrator at CCN <= 10 and to
+// make the short-circuit teardown path single-sourced.
+
+#include "httpserver/webserver.hpp"
+#include "httpserver/detail/webserver_impl.hpp"
+
+#include <microhttpd.h>
+
+#include <strings.h>
+
+#include <chrono>
+#include <cstddef>
+#include <cstdint>
+#include <cstring>
+#include <iostream>
+#include <optional>
+#include <span>
+#include <string>
+#include <utility>
+
+#include "httpserver/create_webserver.hpp"
+#include "httpserver/hook_action.hpp"
+#include "httpserver/hook_context.hpp"
+#include "httpserver/hook_phase.hpp"
+#include "httpserver/http_request.hpp"
+#include "httpserver/http_response.hpp"
+#include "httpserver/http_utils.hpp"
+#include "httpserver/detail/modded_request.hpp"
+
+namespace httpserver {
+
+using httpserver::http::http_utils;
+
+namespace detail {
+
+namespace {
+
+// Wrap the body_chunk firing site so requests_answer_second_step stays a
+// flat sequence of small steps. Returns true iff a hook short-circuited
+// and the caller should signal MHD that the chunk was consumed (returns
+// MHD_YES with *upload_data_size = 0). Side effects on short-circuit:
+//   - mr->response_ is populated with the hook-supplied response,
+//   - mr->skip_handler is set so finalize_answer routes through the
+//     skip branch,
+//   - any in-flight MHD_PostProcessor is destroyed (32 KB buffer freed).
+bool fire_and_maybe_short_circuit_body_chunk(webserver_impl* impl,
+                                             modded_request* mr,
+                                             const char* upload_data,
+                                             size_t upload_data_size) {
+    ::httpserver::body_chunk_ctx ctx{
+        mr->dhr.get(),
+        std::span<const std::byte>(
+            reinterpret_cast<const std::byte*>(upload_data),
+            upload_data_size),
+        static_cast<std::uint64_t>(mr->dhr->get_content().size()),
+        /*is_final=*/false};
+    auto sc = impl->fire_body_chunk(ctx);
+    if (!sc) return false;
+    mr->response_.emplace(std::move(*sc));
+    mr->skip_handler = true;
+    if (mr->pp != nullptr) {
+        MHD_destroy_post_processor(mr->pp);
+        mr->pp = nullptr;
+    }
+    return true;
+}
+
+// Feed @p upload_data through MHD's post processor (when one is
+// attached) and close any open upload-target stream. Pulled out of the
+// second_step orchestrator so the orchestrator stays under the CCN bar.
+void run_post_processor_if_attached(modded_request* mr,
+                                    webserver* parent,
+                                    const char* upload_data,
+                                    size_t upload_data_size) {
+    if (mr->pp == nullptr) return;
+    mr->ws = parent;
+    MHD_post_process(mr->pp, upload_data, upload_data_size);
+    if (mr->upload_ostrm != nullptr && mr->upload_ostrm->is_open()) {
+        mr->upload_ostrm->close();
+    }
+}
+
+}  // namespace
+
+MHD_Result webserver_impl::requests_answer_first_step(MHD_Connection* connection, struct detail::modded_request* mr) {
+    mr->dhr.reset(new http_request(connection, parent->unescaper));
+    mr->dhr->set_file_cleanup_callback(parent->file_cleanup_callback);
+
+    // TASK-047 -- request_received hook. Fires after the http_request is
+    // populated but before any body bytes are read (and before any
+    // post-processor is created). Mutable ref so a hook may adjust
+    // per-request state. Short-circuit: stash the response, mark
+    // skip-to-finalize, and return MHD_YES. MHD will call back into
+    // requests_answer_second_step with *upload_data_size == 0, which
+    // routes through complete_request -> finalize_answer, where the
+    // skip_handler branch goes straight to materialize_and_queue_response.
+    // No post-processor exists at this point, so no teardown is needed.
+    if (any_hooks_[static_cast<std::size_t>(
+                ::httpserver::hook_phase::request_received)]
+            .load(std::memory_order_relaxed)) {
+        ::httpserver::request_received_ctx ctx{
+            mr->dhr.get(),
+            std::chrono::steady_clock::now()};
+        if (auto sc = fire_request_received(ctx)) {
+            mr->response_.emplace(std::move(*sc));
+            mr->skip_handler = true;
+            return MHD_YES;
+        }
+    }
+
+    if (!mr->has_body) {
+        return MHD_YES;
+    }
+
+    mr->dhr->set_content_size_limit(parent->content_size_limit);
+    const char *encoding = MHD_lookup_connection_value(connection, MHD_HEADER_KIND, http_utils::http_header_content_type);
+
+    if (parent->post_process_enabled &&
+        (nullptr != encoding &&
+            ((0 == strncasecmp(http_utils::http_post_encoding_form_urlencoded, encoding, strlen(http_utils::http_post_encoding_form_urlencoded))) ||
+             (0 == strncasecmp(http_utils::http_post_encoding_multipart_formdata, encoding, strlen(http_utils::http_post_encoding_multipart_formdata)))))) {
+        const size_t post_memory_limit(32 * 1024);  // Same as #MHD_POOL_SIZE_DEFAULT
+        mr->pp = MHD_create_post_processor(connection, post_memory_limit, &webserver_impl::post_iterator, mr);
+    } else {
+        mr->pp = nullptr;
+    }
+    return MHD_YES;
+}
+
+MHD_Result webserver_impl::requests_answer_second_step(MHD_Connection* connection, const char* method,
+        const char* version, const char* upload_data,
+        size_t* upload_data_size, struct detail::modded_request* mr) {
+    if (0 == *upload_data_size) return complete_request(connection, mr, version, method);
+
+    if (!mr->has_body) {
+        *upload_data_size = 0;
+        return MHD_YES;
+    }
+
+    // TASK-047 -- a prior pre-handler short-circuit (request_received in
+    // first_step, or body_chunk on an earlier chunk) already populated
+    // mr->response_. Consume the chunk so MHD advances; the next
+    // *upload_data_size == 0 callback will route to finalize_answer's
+    // skip_handler branch.
+    if (mr->skip_handler) {
+        *upload_data_size = 0;
+        return MHD_YES;
+    }
+
+    // TASK-047 -- body_chunk hook fires per chunk BEFORE the bytes are
+    // appended to mr->dhr / fed to MHD_post_process.
+    if (any_hooks_[static_cast<std::size_t>(
+                ::httpserver::hook_phase::body_chunk)]
+            .load(std::memory_order_relaxed)) {
+        if (fire_and_maybe_short_circuit_body_chunk(
+                this, mr, upload_data, *upload_data_size)) {
+            *upload_data_size = 0;
+            return MHD_YES;
+        }
+    }
+
+#ifdef DEBUG
+    std::cout << "Writing content: " << std::string(upload_data, *upload_data_size) << std::endl;
+#endif  // DEBUG
+    // The post iterator is only created from the libmicrohttpd for content of type
+    // multipart/form-data and application/x-www-form-urlencoded
+    // all other content (which is indicated by mr-pp == nullptr)
+    // has to be put to the content even if put_processed_data_to_content is set to false
+    if (mr->pp == nullptr || parent->put_processed_data_to_content) {
+        mr->dhr->grow_content(upload_data, *upload_data_size);
+    }
+    run_post_processor_if_attached(mr, parent, upload_data, *upload_data_size);
+
+    *upload_data_size = 0;
+    return MHD_YES;
+}
+
+}  // namespace detail
+}  // namespace httpserver