Fase 7: cache the daemon's per-invoke lock check, harden mobile server config storage
assert-lock-owner! ran on every write invoke and did fs.existsSync + fs.readFileSync + JSON.parse + a process.kill(pid, 0) syscall each time -- all synchronous on the Node event loop, once per request. Now cached in db_worker_node_lock.cljs: the lock file's content is cached and invalidated by watching its containing directory (survives an unlink+recreate, unlike watching the file path directly), and the liveness check is skipped entirely when the recorded pid is this process's own -- the dominant case, since assert-lock-owner! is the owner checking its own lock, not a stranger checking someone else's. Writes this process makes itself (create-lock!/update-lock!/ remove-lock!) invalidate the cache synchronously in the same call, so the only staleness window is for a lock file changed by another process -- a few ms until the fs.watch callback fires, which doesn't weaken the two-daemons-can't-both-hold-a-live-lock guarantee. Verified against a real daemon (5 consecutive write invokes, no lock-watch-failed warnings) and with two new tests: one proving repeated assert-lock-owner! calls keep agreeing with an unchanged lock, one proving a lock file rewritten by "another process" (a raw fs.writeFileSync bypassing update-lock!'s own invalidation) is picked up within the polling window instead of being served stale forever. All 8 pre-existing lock tests and all 61 db-worker-node-test tests still pass. frontend.config's db-worker-server-url/token getters/setters stay synchronous localStorage reads everywhere (retrofitting their many callers to await an async native store is out of scope), but on native mobile, writes now also mirror into frontend.mobile.secure-storage (OS keychain/keystore) as a durability backstop -- WebView localStorage can be evicted under storage pressure or not survive a reinstall. mobile.init/init! now fires config/<restore-server-config-from-secure- storage! at startup to copy a surviving value back into localStorage if it came back empty. Guarded so this is a true no-op outside native mobile (including under Node tests, where it's additionally gated by the same util/node-test? check the existing localStorage calls already used) -- verified via the :mobile shadow-cljs target compiling clean and the full test suite (subs-test, config-test, db-worker-node-test, db-worker-node-lock-test, subs-loader-test) staying green. iOS ATS and Android cleartext traffic, also called out in this phase of the plan, turned out to already be configured permissively (NSAllowsArbitraryLoads in Info.plist, cleartextTrafficPermitted in network_security_config.xml) -- no changes needed there. Investigated whether db-core's .remoteInvoke serializes concurrent invocations, per the plan's own open question: it does not -- frontend.common.thread-api/remote-function dispatches straight into the handler with no per-repo queueing, so two concurrent write invokes to the same repo could in principle interleave at any await point inside the handler. Deliberately did NOT add a write queue for this: the plan gates that on Fase 0 telemetry showing real invoke overlap in practice, and doing it without solid evidence of need risks a mistake in read-vs-write method classification for uncertain benefit -- querying methods (:thread-api/q, :thread-api/get-render-snapshots, etc.) share the same repo-scoped lock-check path Fase 6 just optimized for read latency, and serializing those too would undo that work. Left as an explicitly open question rather than guessed at. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -84,10 +84,66 @@
|
||||
(js->clj (js/JSON.parse (.toString (fs/readFileSync path) "utf8"))
|
||||
:keywordize-keys true))))
|
||||
|
||||
;; assert-lock-owner! runs on every write invoke (db_worker_node.cljs), so a
|
||||
;; naive read-lock there means fs.existsSync + fs.readFileSync + JSON.parse
|
||||
;; as a synchronous syscall on the Node event loop per request. Cached here,
|
||||
;; invalidated by watching the lock file's *directory* (not the file itself
|
||||
;; -- fs.watch on a path can stop firing across an unlink+recreate on some
|
||||
;; platforms, watching the containing directory and filtering by filename
|
||||
;; survives it) so an external change to the lock is picked up on the next
|
||||
;; read instead of the cache going permanently stale (Fase 7).
|
||||
;;
|
||||
;; Writes this process makes itself (create-lock!/update-lock!/remove-lock!)
|
||||
;; invalidate synchronously in the same call, before the fs.watch callback
|
||||
;; would even fire -- the only staleness window is for a lock file changed
|
||||
;; by some *other* process, on the order of a few ms until the watch
|
||||
;; callback runs. That's the same window any lock-based coordination has
|
||||
;; between the change happening and a reader noticing it; it does not weaken
|
||||
;; the guarantee two daemons can't both hold a live lock on the same repo.
|
||||
(defonce ^:private *lock-cache (atom {}))
|
||||
(def ^:private absent ::absent)
|
||||
|
||||
(defn- invalidate-cached-lock!
|
||||
[path]
|
||||
(swap! *lock-cache update path dissoc :lock))
|
||||
|
||||
(defn- close-watcher!
|
||||
[path]
|
||||
(when-let [watcher (get-in @*lock-cache [path :watcher])]
|
||||
(try (.close watcher) (catch :default _ nil)))
|
||||
(swap! *lock-cache dissoc path))
|
||||
|
||||
(defn- ensure-watcher!
|
||||
[path]
|
||||
(when-not (get-in @*lock-cache [path :watcher])
|
||||
(try
|
||||
(let [dir (node-path/dirname path)
|
||||
file-name (node-path/basename path)
|
||||
watcher (fs/watch dir
|
||||
(fn [_event changed-name]
|
||||
(when (or (nil? changed-name) (= changed-name file-name))
|
||||
(invalidate-cached-lock! path))))]
|
||||
(swap! *lock-cache assoc-in [path :watcher] watcher))
|
||||
(catch :default e
|
||||
;; Directory may not exist yet (lock not created yet) -- next call
|
||||
;; retries; falls back to always-fresh reads until it succeeds.
|
||||
(log/warn :db-worker-node-lock-watch-failed {:path path :error (ex-message e)})))))
|
||||
|
||||
(defn- cached-lock
|
||||
[path]
|
||||
(ensure-watcher! path)
|
||||
(let [cached (get-in @*lock-cache [path :lock] absent)]
|
||||
(if (= cached absent)
|
||||
(let [lock (read-lock path)]
|
||||
(swap! *lock-cache assoc-in [path :lock] lock)
|
||||
lock)
|
||||
cached)))
|
||||
|
||||
(defn remove-lock!
|
||||
[path]
|
||||
(when (and (seq path) (fs/existsSync path))
|
||||
(fs/unlinkSync path)))
|
||||
(fs/unlinkSync path))
|
||||
(close-watcher! path))
|
||||
|
||||
(defn create-lock!
|
||||
[{:keys [root-dir repo owner-source]}]
|
||||
@@ -111,6 +167,7 @@
|
||||
(fs/writeFileSync fd (js/JSON.stringify (clj->js lock)))
|
||||
(finally
|
||||
(fs/closeSync fd)))
|
||||
(invalidate-cached-lock! path)
|
||||
(resolve lock)))
|
||||
(catch :default e
|
||||
(log/error :db-worker-node-lock-create-failed e)
|
||||
@@ -133,14 +190,26 @@
|
||||
:owner-source (normalize-owner-source (:owner-source lock))})]
|
||||
(fs/mkdirSync (node-path/dirname path) #js {:recursive true})
|
||||
(fs/writeFileSync path (js/JSON.stringify (clj->js lock')))
|
||||
(invalidate-cached-lock! path)
|
||||
(resolve lock'))
|
||||
(catch :default e
|
||||
(log/error :db-worker-node-lock-update-failed e)
|
||||
(reject e))))))
|
||||
|
||||
(defn- lock-owner-alive?
|
||||
"The pid recorded in the lock file is, on the hot path, this very
|
||||
process's own pid -- assert-lock-owner! is the *owner* checking its own
|
||||
lock hasn't been usurped, not a stranger checking someone else's. A
|
||||
process can always signal itself, so that case needs no
|
||||
process.kill(pid, 0) syscall at all; only a genuinely foreign pid (a
|
||||
stolen/stale lock, already on the throw path) pays for the real check."
|
||||
[pid]
|
||||
(or (= pid (.-pid js/process))
|
||||
(= :alive (pid-status pid))))
|
||||
|
||||
(defn assert-lock-owner!
|
||||
[path {:keys [repo pid lock-id] :as owner-lock}]
|
||||
(let [lock (read-lock path)]
|
||||
(let [lock (cached-lock path)]
|
||||
(cond
|
||||
(nil? owner-lock)
|
||||
(throw (ex-info "lock owner missing"
|
||||
@@ -152,7 +221,7 @@
|
||||
{:code :repo-locked
|
||||
:path path}))
|
||||
|
||||
(not= :alive (pid-status (:pid lock)))
|
||||
(not (lock-owner-alive? (:pid lock)))
|
||||
(throw (ex-info "graph lock is stale"
|
||||
{:code :repo-locked
|
||||
:path path
|
||||
|
||||
Reference in New Issue
Block a user