Fase 6: close the subs-loader in-flight dedup gap, raise batch limits, add /v1/invoke-batch

Fase 0's telemetry work had already flagged that subs-loader deduped
load! calls that landed before a flush, but not ones arriving while a
flush's own worker request was still in flight -- flush! cleared *batch
via take-batch! before the request it just sent even resolved, so a
load! for the same slot in that window fired a duplicate request. Fixed
by tracking in-flight requests in a separate *inflight map, untracked
only once the request settles (db/subs_loader.cljs), with a real e2e
test proving the dedup and that a later load! after settlement does
fire a fresh request (db/subs_loader_test.cljs).

Also raises the :children/:resources get-render-snapshots batch limits
from 25 to 200 -- 25 was calibrated for postMessage-to-a-worker being
free; over the network that trade inverts. Two existing subs-test.cljs
tests hardcoded the old 25-item batching behavior with 100 items across
4 calls; updated to the new single-call-of-100 behavior under the 200
limit.

Deliberately did NOT widen db/subs.cljs's load-batch-coalesce-window
from the same-tick 0ms setTimeout to a few ms, despite that being in
the original plan: doing so broke ~212 assertions across subs_test.cljs
that synchronize on same-tick flushing via (p/delay 0), and would be
redundant with the invoke-batch coalescer below, which already captures
the same "batch several concurrent RPCs into fewer HTTP round trips"
win at the transport layer instead.

New POST /v1/invoke-batch on the daemon (db_worker_node.cljs) runs an
array of {method, argsTransit} through the same repo-check/lock/invoke
path as /v1/invoke, but never rejects the whole batch on one item's
failure -- each item independently becomes {ok, resultTransit} or {ok:
false, error}. Verified against a live daemon: mixed success/repo-
mismatch/success in one request, 401 without auth, 405 on non-POST.

On the client, frontend.persist-db.remote/create-invoke-batcher
coalesces wrapped-worker calls (the RPC every state/<invoke-db-worker
call funnels through for a remote client) issued within a 10ms window
into one POST /v1/invoke-batch, generalizing subs-loader's own
same-method batching to every thread-api call a remote client makes.
Two invokes only land in the same batch when neither is awaited before
the other fires, so this can't reorder anything a caller depends on
sequentially.

New db_worker_node_test.cljs coverage: invoke-batch mixing success and
repo-mismatch failure in one request, and invoke-batch auth/method
enforcement.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-26 11:13:04 +02:00
parent 902bf7f568
commit 2164bbdf1e
7 changed files with 335 additions and 15 deletions

View File

@@ -448,6 +448,34 @@
[^js buf]
(str "\"" (-> (crypto/createHash "sha1") (.update buf) (.digest "hex")) "\""))
(defn- <process-batch-invoke!
"One entry of a /v1/invoke-batch request, run through the same
repo-check/lock/invoke path as /v1/invoke but never rejecting -- a failed
item becomes {:ok false :error ...} in its slot of the batch response
instead of failing every other invoke that happened to coalesce into the
same HTTP request (Fase 6)."
[proxy bound-repo {:keys [method argsTransit args]}]
(let [method-kw (normalize-method-kw method)
method-str (normalize-method-str method)
args' (or argsTransit args)]
(-> (p/let [args-for-validation (if (string? args')
(ldb/read-transit-str args')
args')]
(if-let [{:keys [error]} (repo-error method-kw args-for-validation bound-repo)]
{:ok false :error error}
(p/let [_ (when-not (contains? non-repo-methods method-kw)
(let [{:keys [path lock]} @*lock-info]
(db-lock/assert-lock-owner! path lock)))
_ (invoke-started! bound-repo)
result (-> (<invoke! proxy method-str method-kw args')
(p/finally (fn [] (invoke-finished! bound-repo))))]
{:ok true :resultTransit result})))
(p/catch (fn [error]
(let [data (ex-data error)]
{:ok false
:error {:code (invoke-error-code data)
:message (invoke-error-message error data)}}))))))
(defn- ws-upgrade-handler
[^js wss bound-repo port extra-allowed-hosts]
(fn [^js req ^js socket head]
@@ -611,6 +639,16 @@
(log-invoke-error! res error :thread-api/asset-write-bytes))))
(send-json! res 400 {:ok false :error {:code :invalid-request :message "invalid asset path"}}))
(= request-path "/v1/invoke-batch")
(if (= method "POST")
(-> (p/let [body (<read-body req)
{:keys [invokes]} (js->clj (js/JSON.parse body) :keywordize-keys true)
results (p/all (map #(<process-batch-invoke! proxy bound-repo %) invokes))]
(send-json! res 200 {:ok true :results results}))
(p/catch (fn [error]
(log-invoke-error! res error nil))))
(send-text! res 405 "method-not-allowed"))
(= request-path "/v1/invoke")
(if (= method "POST")
(->