Fase 1: require session auth on the db-worker-node HTTP/SSE server

Prepares the daemon to be reachable off loopback as part of the
server-client migration (see the approved implementation plan). Today it
trusts any caller that can reach the port; this makes every route except
health checks and login require a bearer session token.

- New frontend.worker.db-worker-node-auth: scrypt-hashed username/password
  credential persisted at <root-dir>/db-worker-auth.json (mode 0600),
  bootstrapped from LOGSEQ_DAEMON_USERNAME/PASSWORD on first launch since
  there's no interactive setup yet. In-memory bearer sessions with
  constant-time comparison, a failed-login lockout, and list/revoke by
  device. The daemon now refuses to start with no credential configured at
  all -- not just when binding beyond loopback -- since without one no
  token could ever be issued.
- db_worker_node.cljs: POST /v1/auth/login and GET/DELETE
  /v1/auth/sessions; every other route requires Authorization: Bearer
  except /healthz, which stays open (readiness polling happens before any
  session exists) but now omits :pid/:root-dir from its payload. Adds a
  Host-header allowlist ahead of auth to guard against DNS rebinding, and
  configurable --host/--port/--allowed-origin/--allowed-host in place of
  the hardcoded 127.0.0.1 bind and single CORS origin.
- EventSource can't send custom headers, so /v1/events accepts the bearer
  token as a query param instead; persist_db/remote.cljs's events-url
  carries it there now (interim until the Fase 4 WebSocket migration).
- Electron always sent :auth-token nil (verified by reading db_worker.cljs
  -- one of the plan's open questions). daemon/spawn-server! now seeds a
  per-root-dir local device credential
  (<root-dir>/db-worker-local-credential.json, reused across restarts) via
  LOGSEQ_DAEMON_USERNAME/PASSWORD, and cli/server.cljs logs in once the
  spawned or reused daemon is ready, threading the resulting token through
  ensure-server! to Electron's runtime config.

Not done: the OCaml CLI (cli/lib/transport.ml) also calls /v1/invoke
directly and will start receiving 401s once this ships -- confirmed with
the user to leave that gap for now rather than touch the OCaml toolchain
in this pass.

Verified with clojure -M:cljs compile app db-worker-node electron (0
warnings) and the full logseq.db-worker.daemon-test,
logseq.cli.server-test, frontend.worker.db-worker-node-test,
electron.db-worker-manager-test suites (113 tests, 480 assertions, 0
failures) plus frontend.state-test and frontend.persist-db.remote-test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-25 18:59:08 +02:00
parent 52971f8ec9
commit 49d9f7da84
9 changed files with 507 additions and 74 deletions

10
.gitignore vendored
View File

@@ -12,17 +12,15 @@ out/
.lsp/.cache/
# Build output
static/js/
static/out/
static/mobile/js/
static/db-worker-node.js
static/db-worker-node-assets.json
static/electron.js
static/
dist/
cli/_build/
cli/**/*.install
resources/.cache/
# Test run temp dirs (frontend.test.node-helper/create-tmp-dir)
tmp/
# OCaml/Dune
cli/_opam/

View File

@@ -246,7 +246,7 @@
repo)]
{:repo repo
:base-url (:base-url config)
:auth-token nil
:auth-token (:auth-token config)
:owned? (:owned? config)})))
(defn- stop-managed-daemon!

View File

@@ -23,8 +23,14 @@
(js/encodeURIComponent repo)))
(defn- events-url
[base-url]
(str (normalize-base-url base-url) "/v1/events"))
"EventSource has no header API, so the bearer token travels as a query
param here instead of the `Authorization` header every other request
uses -- the server accepts it for this route only (see
request-authorized? in db_worker_node.cljs). Interim until the Fase 4
WebSocket migration, which carries it via Sec-WebSocket-Protocol instead."
[base-url auth-token]
(cond-> (str (normalize-base-url base-url) "/v1/events")
(seq auth-token) (str "?token=" (js/encodeURIComponent auth-token))))
(defn- base-headers
[auth-token]
@@ -175,7 +181,7 @@
(when @connected?
(reset! subscription
(open-sse-fn
{:url (events-url base-url)
{:url (events-url base-url auth-token)
:headers (base-headers auth-token)
:on-message (fn [chunk]
(when @connected?

View File

@@ -3,6 +3,7 @@
(:require ["http" :as http]
[clojure.string :as string]
[frontend.worker.db-core :as db-core]
[frontend.worker.db-worker-node-auth :as auth]
[frontend.worker.db-worker-node-lock :as db-lock]
[frontend.worker.platform.node :as platform-node]
[frontend.worker.state :as worker-state]
@@ -44,28 +45,44 @@
[root-dir]
(server-list/path root-dir))
(def ^:private cors-headers
#js {"Access-Control-Allow-Origin" "lsp://logseq.com"
"Access-Control-Allow-Methods" "GET,POST,OPTIONS"
"Access-Control-Allow-Headers" "Content-Type,Authorization"})
(def ^:private default-allowed-origins #{"lsp://logseq.com"})
(defn- cors-headers-for-request
"CORS is a browser-only safety layer, not access control (that's the
bearer-token + Host checks below) -- it only decides whether a browser is
allowed to read the response. Reflects the request Origin when allowed,
per the standard multi-origin CORS pattern (a static header can't list
more than one origin)."
[^js req allowed-origins]
(let [origin (some-> req .-headers .-origin)
base #js {"Access-Control-Allow-Methods" "GET,POST,DELETE,OPTIONS"
"Access-Control-Allow-Headers" "Content-Type,Authorization"
"Vary" "Origin"}]
(if (and origin (contains? allowed-origins origin))
(doto base (aset "Access-Control-Allow-Origin" origin))
base)))
(defn- set-cors-headers!
[^js res headers]
(set! (.-corsHeaders res) headers))
(defn- response-headers
[headers]
(js/Object.assign #js {} cors-headers headers))
[^js res headers]
(js/Object.assign #js {} (or (.-corsHeaders res) #js {}) headers))
(defn- send-no-content!
[^js res]
(.writeHead res 204 cors-headers)
(.writeHead res 204 (or (.-corsHeaders res) #js {}))
(.end res))
(defn- send-json!
[^js res status payload]
(.writeHead res status (response-headers #js {"Content-Type" "application/json"}))
(.writeHead res status (response-headers res #js {"Content-Type" "application/json"}))
(.end res (js/JSON.stringify (clj->js payload))))
(defn- send-text!
[^js res status text]
(.writeHead res status (response-headers #js {"Content-Type" "text/plain"}))
(.writeHead res status (response-headers res #js {"Content-Type" "text/plain"}))
(.end res text))
(defn- <read-body-buffer
@@ -98,6 +115,10 @@
"--log-level" (recur (subvec args 2) (assoc opts :log-level (second args)))
"--embedding-endpoint" (recur (subvec args 2) (assoc opts :embedding-endpoint (second args)))
"--embedding-model-id" (recur (subvec args 2) (assoc opts :embedding-model-id (second args)))
"--host" (recur (subvec args 2) (assoc opts :host (second args)))
"--port" (recur (subvec args 2) (assoc opts :port (js/parseInt (second args) 10)))
"--allowed-origin" (recur (subvec args 2) (update opts :allowed-origins (fnil conj #{}) (second args)))
"--allowed-host" (recur (subvec args 2) (update opts :allowed-hosts (fnil conj #{}) (second args)))
"--create-empty-db" (recur (subvec args 1) (assoc opts :create-empty-db? true))
"--version" (recur (subvec args 1) (assoc opts :version? true))
"--help" (recur (subvec args 1) (assoc opts :help? true))
@@ -150,9 +171,9 @@
(defn- sse-handler
[^js req ^js res]
(.writeHead res 200 (response-headers #js {"Content-Type" "text/event-stream"
"Cache-Control" "no-cache"
"Connection" "keep-alive"}))
(.writeHead res 200 (response-headers res #js {"Content-Type" "text/event-stream"
"Cache-Control" "no-cache"
"Connection" "keep-alive"}))
(.write res "\n")
(swap! *sse-clients conj res)
(.on req "close" (fn []
@@ -299,14 +320,16 @@
(str error)))
(defn- health-payload
[{:keys [bound-repo host port owner-source root-dir]}]
"Unauthenticated by design (see request-authorized?), so this intentionally
omits :pid and :root-dir -- filesystem/process details that shouldn't be
handed to anything reachable on the allowed network segment without a
bearer token."
[{:keys [bound-repo host port owner-source]}]
{:repo bound-repo
:status (if @*ready? "ready" "starting")
:host host
:port (if (satisfies? IDeref port) @port port)
:pid (.-pid js/process)
:owner-source (name (normalize-owner-source owner-source))
:root-dir root-dir
:revision (build-version/revision)})
(defn- log-invoke-error!
@@ -326,25 +349,101 @@
:method method-kw})
(send-json! res status payload)))
(defn- effective-port
[port]
(if (satisfies? IDeref port) @port port))
(defn- allowed-hosts-for
"Host header allowlist, guarding against DNS rebinding: without this, any
web page the operator visits could script a request to the daemon on
loopback. Always includes loopback for the bound port; a non-default
--host (e.g. 0.0.0.0 behind Tailscale) needs its reachable name/IP added
explicitly via --allowed-host."
[{:keys [port extra-allowed-hosts]}]
(let [p (effective-port port)]
(into #{(str "localhost:" p) (str "127.0.0.1:" p) (str "[::1]:" p)}
extra-allowed-hosts)))
(defn- host-allowed?
[^js req allowed-hosts]
(contains? allowed-hosts (some-> req .-headers .-host)))
(defn- request-authorized?
"/v1/events (SSE) can't carry an Authorization header -- EventSource has no
header API -- so it accepts the bearer token as a query param instead.
This is interim: the WebSocket migration (see docs/agent-guide) replaces
it with Sec-WebSocket-Protocol, which doesn't leak into proxy/access logs
the way a query string does."
[^js req parsed-url request-path]
(if (= request-path "/v1/events")
(auth/authorized? (some->> (.get (.-searchParams parsed-url) "token")
(str "Bearer ")))
(auth/authorized? (.. req -headers -authorization))))
(defn- make-server
[proxy {:keys [bound-repo stop-fn host port owner-source root-dir]}]
[proxy {:keys [bound-repo stop-fn host port owner-source root-dir
allowed-origins extra-allowed-hosts]}]
(http/createServer
(fn [^js req ^js res]
(let [url (.-url req)
parsed-url (js/URL. url "http://127.0.0.1")
request-path (.-pathname parsed-url)
method (.-method req)]
method (.-method req)
;; Computed per-request (not once at server start) because `port`
;; is an atom that only resolves to the real OS-assigned port
;; after `.listen` calls back -- see `start-http-server!`.
allowed-hosts (allowed-hosts-for {:port port :extra-allowed-hosts extra-allowed-hosts})]
(set-cors-headers! res (cors-headers-for-request req allowed-origins))
(cond
(= method "OPTIONS")
(send-no-content! res)
(not (host-allowed? req allowed-hosts))
(send-json! res 403 {:ok false :error {:code :host-not-allowed
:message "Host header not allowed"}})
;; Unauthenticated on purpose: readiness polling (daemon lifecycle,
;; CLI/Electron discovery) happens before any session exists, so
;; this can't require a token without a chicken-and-egg startup
;; deadlock. Still behind the Host allowlist above. health-payload
;; omits root-dir/pid so it doesn't leak filesystem/process details
;; to anything reachable on the allowed network segment.
(= request-path "/healthz")
(send-json! res (if @*ready? 200 503)
(health-payload {:bound-repo bound-repo
:host host
:port port
:owner-source owner-source
:root-dir root-dir}))
:owner-source owner-source}))
(and (= request-path "/v1/auth/login") (= method "POST"))
(-> (p/let [body (<read-body req)
{:keys [username password deviceLabel]}
(js->clj (js/JSON.parse body) :keywordize-keys true)
{:keys [session error]} (auth/login! root-dir
{:username username
:password password
:device-label deviceLabel})]
(if session
(send-json! res 200 {:ok true
:token (:token session)
:sessionId (:id session)
:deviceLabel (:device-label session)})
(send-json! res 401 {:ok false :error {:code error :message "invalid credentials"}})))
(p/catch (fn [_error]
(send-json! res 400 {:ok false :error {:code :invalid-request
:message "invalid login request body"}}))))
(not (request-authorized? req parsed-url request-path))
(send-json! res 401 {:ok false :error {:code :unauthorized
:message "missing or invalid credentials"}})
(and (= request-path "/v1/auth/sessions") (= method "GET"))
(send-json! res 200 {:ok true :sessions (auth/list-sessions)})
(and (string/starts-with? request-path "/v1/auth/sessions/") (= method "DELETE"))
(send-json! res 200 {:ok true
:revoked (auth/revoke-session!
(subs request-path (count "/v1/auth/sessions/")))})
(= request-path "/v1/events")
(sse-handler req res)
@@ -427,7 +526,13 @@
(println (str " " (style/bold "--embedding-endpoint") " <url>"))
(println (str " " (style/bold "--embedding-model-id") " <id>"))
(println (str " " (style/bold "--log-level") " <level> (default info)"))
(println (str " " (style/bold "--host") " <host> (default 127.0.0.1; use an explicit --allowed-host too when binding beyond loopback)"))
(println (str " " (style/bold "--port") " <port> (default OS-assigned)"))
(println (str " " (style/bold "--allowed-origin") " <origin> (repeatable; browser CORS allowlist, not access control)"))
(println (str " " (style/bold "--allowed-host") " <host:port> (repeatable; Host header allowlist beyond loopback)"))
(println (str " " (style/bold "--version") " (print build metadata and exit)"))
(println " Requires a credential: set LOGSEQ_DAEMON_USERNAME and LOGSEQ_DAEMON_PASSWORD")
(println " on first run to bootstrap <root-dir>/db-worker-auth.json, or restore that file.")
(println " logs: <root-dir>/graphs/<graph-dir>/db-worker-node-YYYYMMDD.log (retains 7)"))
(defn- startup-db-opts
@@ -521,7 +626,8 @@
:stop! stop!})))
(defn- start-http-server!
[{:keys [proxy repo host port owner-source root-dir on-stopped!]}]
[{:keys [proxy repo host port owner-source root-dir on-stopped!
allowed-origins extra-allowed-hosts]}]
(let [stop!* (atom nil)
stopped? (atom false)
port* (atom nil)
@@ -530,6 +636,8 @@
:port port*
:owner-source owner-source
:root-dir root-dir
:allowed-origins allowed-origins
:extra-allowed-hosts extra-allowed-hosts
:stop-fn (fn []
(when-let [stop! @stop!*]
(stop!)))})]
@@ -554,9 +662,11 @@
(reject error)))))))
(defn start-daemon!
[{:keys [root-dir repo log-level owner-source on-stopped!] :as opts}]
(let [host "127.0.0.1"
port 0
[{:keys [root-dir repo log-level owner-source on-stopped!
host port allowed-origins allowed-hosts] :as opts}]
(let [host (or host "127.0.0.1")
port (or port 0)
allowed-origins (into default-allowed-origins allowed-origins)
owner-source (normalize-owner-source owner-source)]
(cond
(not (seq root-dir))
@@ -568,6 +678,7 @@
:else
(try
(let [root-dir (root-dir/ensure-root-dir! root-dir)
_ (auth/ensure-credential! root-dir)
server-list-file (server-list-file-path root-dir)]
(db-worker-log/install! {:root-dir root-dir
:repo repo
@@ -600,6 +711,8 @@
:port port
:owner-source owner-source
:root-dir root-dir
:allowed-origins allowed-origins
:extra-allowed-hosts allowed-hosts
:on-stopped! on-stopped!}))
(p/catch (fn [e]
(when-let [lock-path (:path @*lock-info)]
@@ -631,6 +744,10 @@
:owner-source owner-source
:embedding-endpoint (:embedding-endpoint opts)
:embedding-model-id (:embedding-model-id opts)
:host (:host opts)
:port (:port opts)
:allowed-origins (:allowed-origins opts)
:allowed-hosts (:allowed-hosts opts)
:on-stopped! (fn []
(log/info :db-worker-node-stopped nil)
(.exit js/process 0))

View File

@@ -0,0 +1,171 @@
(ns frontend.worker.db-worker-node-auth
"Session-based authentication for the db-worker-node HTTP/SSE server.
A single username/password credential is stored, scrypt-hashed, in
<root-dir>/db-worker-auth.json. There is no interactive setup yet: the
credential is bootstrapped from the LOGSEQ_DAEMON_USERNAME /
LOGSEQ_DAEMON_PASSWORD env vars on first run (see `ensure-credential!`).
Sessions are bearer tokens held in memory only; a daemon restart revokes
every session, which is acceptable for a personal single-user server."
(:require ["crypto" :as crypto]
["fs" :as fs]
["path" :as node-path]
[clojure.string :as string]
[lambdaisland.glogi :as log]))
(defonce ^:private *sessions (atom {}))
(defonce ^:private *failed-attempts (atom {}))
(def ^:private max-failed-attempts 5)
(def ^:private lockout-ms (* 5 60 1000))
(def ^:private scrypt-key-length 64)
(defn- credential-file-path
[root-dir]
(node-path/join root-dir "db-worker-auth.json"))
(defn- try-chmod-owner-only!
[path]
(try
(fs/chmodSync path 384) ;; 384 = 0o600, owner read/write only
(catch :default _ nil)))
(defn- scrypt-hash
[password salt]
(.toString (crypto/scryptSync password salt scrypt-key-length) "hex"))
(defn- gen-salt-hex [] (.toString (crypto/randomBytes 16) "hex"))
(defn- gen-token [] (.toString (crypto/randomBytes 32) "base64url"))
(defn- gen-session-id [] (.toString (crypto/randomBytes 8) "hex"))
(defn- now-iso [] (.toISOString (js/Date.)))
(defn- timing-safe-equal?
"Constant-time string comparison. Guards the length check outside
crypto.timingSafeEqual, which throws on mismatched buffer lengths instead
of returning false."
[a b]
(let [ba (js/Buffer.from (or a "") "utf8")
bb (js/Buffer.from (or b "") "utf8")]
(and (= (.-length ba) (.-length bb))
(crypto/timingSafeEqual ba bb))))
(defn read-credential
"Returns {:username :salt :hash :created-at} or nil if none is configured."
[root-dir]
(let [path (credential-file-path root-dir)]
(when (fs/existsSync path)
(-> (fs/readFileSync path "utf8")
js/JSON.parse
(js->clj :keywordize-keys true)))))
(defn set-credential!
[root-dir username password]
(let [salt (gen-salt-hex)
path (credential-file-path root-dir)
credential {:username username
:salt salt
:hash (scrypt-hash password salt)
:created-at (now-iso)}]
(fs/writeFileSync path (js/JSON.stringify (clj->js credential)) "utf8")
(try-chmod-owner-only! path)
credential))
(defn ensure-credential!
"Returns the configured credential, bootstrapping one from
LOGSEQ_DAEMON_USERNAME / LOGSEQ_DAEMON_PASSWORD on first run. Throws if
none is configured and no bootstrap env vars are set: the daemon must
refuse to start rather than serve with no way to ever authenticate."
[root-dir]
(or (read-credential root-dir)
(let [username (not-empty (.. js/process -env -LOGSEQ_DAEMON_USERNAME))
password (not-empty (.. js/process -env -LOGSEQ_DAEMON_PASSWORD))]
(if (and username password)
(let [credential (set-credential! root-dir username password)]
(log/info :db-worker-node-auth-bootstrapped
{:username username :path (credential-file-path root-dir)})
credential)
(throw (ex-info
(str "No db-worker-node credential configured. Set "
"LOGSEQ_DAEMON_USERNAME and LOGSEQ_DAEMON_PASSWORD to "
"bootstrap one on first run, or restore "
(credential-file-path root-dir) ".")
{:code :missing-daemon-credential}))))))
(defn- verify-password?
[{:keys [username salt hash]} candidate-username candidate-password]
(and hash salt username
(= username candidate-username)
(timing-safe-equal? hash (scrypt-hash (or candidate-password "") salt))))
(defn- locked-out?
[username]
(when-let [{:keys [locked-until]} (get @*failed-attempts username)]
(and locked-until (< (js/Date.now) locked-until))))
(defn- record-failed-attempt!
[username]
(swap! *failed-attempts update username
(fn [{:keys [count] :or {count 0}}]
(let [count' (inc count)]
(cond-> {:count count'}
(>= count' max-failed-attempts)
(assoc :locked-until (+ (js/Date.now) lockout-ms)))))))
(defn- clear-failed-attempts!
[username]
(swap! *failed-attempts dissoc username))
(defn create-session!
[device-label]
(let [id (gen-session-id)
now (now-iso)
session {:id id
:token (gen-token)
:device-label (or (not-empty device-label) "unknown")
:created-at now
:last-seen-at now}]
(swap! *sessions assoc id session)
session))
(defn login!
"Verifies username/password against the stored credential and, on
success, creates a session. Returns {:session {...}} or {:error kw}."
[root-dir {:keys [username password device-label]}]
(cond
(string/blank? username)
{:error :invalid-credentials}
(locked-out? username)
{:error :locked-out}
:else
(let [credential (read-credential root-dir)]
(if (and credential (verify-password? credential username password))
(do (clear-failed-attempts! username)
{:session (create-session! device-label)})
(do (record-failed-attempt! username)
{:error :invalid-credentials})))))
(defn authorized?
"Given an `Authorization: Bearer <token>` header value (or nil), returns
true and refreshes :last-seen-at when it matches an active session."
[authorization-header]
(when-let [token (some-> authorization-header
(as-> h (when (string/starts-with? h "Bearer ") (subs h 7))))]
(when-let [session (some #(when (timing-safe-equal? token (:token %)) %)
(vals @*sessions))]
(swap! *sessions assoc-in [(:id session) :last-seen-at] (now-iso))
true)))
(defn list-sessions
[]
(->> (vals @*sessions)
(map #(dissoc % :token))
(sort-by :created-at)
vec))
(defn revoke-session!
[id]
(let [existed? (contains? @*sessions id)]
(swap! *sessions dissoc id)
existed?))

View File

@@ -329,11 +329,19 @@
(p/let [_ (profile/time! profile-session
"server.wait-ready"
(fn []
(wait-for-ready repo-server')))]
(wait-for-ready repo-server')))
auth-token (profile/time! profile-session
"server.login"
(fn []
(daemon/login! {:host (:host repo-server')
:port (:port repo-server')
:root-dir root-dir
:device-label (name requester-owner)})))]
(let [lock-owner (lock-owner-source lock)]
(assoc repo-server'
:owner-source lock-owner
:owned? (owner-manageable? requester-owner lock-owner)))))))))
:owned? (owner-manageable? requester-owner lock-owner)
:auth-token auth-token))))))))
(declare stop-version-mismatched-server!)
@@ -382,7 +390,8 @@
(assoc config
:base-url (base-url lock)
:owner-source (:owner-source lock)
:owned? (:owned? lock))))
:owned? (:owned? lock)
:auth-token (:auth-token lock))))
(defn- shutdown!
[{:keys [host port]}]

View File

@@ -1,8 +1,10 @@
(ns logseq.db-worker.daemon
"Shared db-worker-node lifecycle helpers for CLI and Electron."
(:require ["child_process" :as child-process]
["crypto" :as crypto]
["fs" :as fs]
["http" :as http]
["path" :as node-path]
[clojure.string :as string]
[lambdaisland.glogi :as log]
[logseq.db-worker.log :as db-worker-log]
@@ -103,6 +105,55 @@
(contains? #{200 503} status))
(p/catch (fn [_] false))))
(defn- local-credential-path
[root-dir]
(node-path/join root-dir "db-worker-local-credential.json"))
(defn ensure-local-credential!
"A local automated device credential for the CLI/Electron process that
spawns and talks to its own db-worker-node daemon (see spawn-server!,
which bootstraps the daemon's stored credential from this same
username/password). Generated once and reused across restarts so the
spawning process can log in without interactive input.
Lives in root-dir alongside the daemon's other discovery/lock files:
anyone with filesystem access to root-dir already has full read/write
access to the graph's SQLite files, so a plaintext bootstrap password
here doesn't widen the local trust boundary. It only bootstraps HTTP
session auth, which exists to stop *network* clients, not local ones."
[root-dir]
(let [path (local-credential-path root-dir)]
(if (fs/existsSync path)
(js->clj (js/JSON.parse (.toString (fs/readFileSync path) "utf8")) :keywordize-keys true)
(let [credential {:username "local"
:password (.toString (crypto/randomBytes 24) "base64url")}]
(when-not (fs/existsSync root-dir)
(fs/mkdirSync root-dir #js {:recursive true}))
(fs/writeFileSync path (js/JSON.stringify (clj->js credential)))
(try (fs/chmodSync path 384) (catch :default _ nil)) ;; 384 = 0o600, owner read/write only
credential))))
(defn login!
"Logs into a db-worker-node daemon's HTTP auth using the local device
credential for root-dir, returning a bearer token. Always succeeds
against a daemon this process itself spawned, since spawn-server! seeds
that daemon's stored credential from the same local file on first run."
[{:keys [host port root-dir device-label]}]
(p/let [{:keys [username password]} (ensure-local-credential! root-dir)
request-body (js/JSON.stringify (clj->js {:username username
:password password
:deviceLabel (or device-label "local")}))
{:keys [status body]} (http-request {:method "POST"
:host host
:port port
:path "/v1/auth/login"
:headers {"Content-Type" "application/json"}
:body request-body
:timeout-ms 5000})]
(if (= 200 status)
(:token (js->clj (js/JSON.parse body) :keywordize-keys true))
(throw (ex-info "db-worker-node login failed" {:code :login-failed :status status})))))
(defn valid-lock?
[lock]
(and (seq (:repo lock))
@@ -227,11 +278,15 @@
[{:keys [script repo root-dir owner-source create-empty-db? embedding-endpoint embedding-model-id]}]
(let [owner-source (normalize-owner-source owner-source)
detached? (not= owner-source :electron)
{:keys [username password]} (ensure-local-credential! root-dir)
args (clj->js (cond-> [script "--repo" repo "--root-dir" root-dir "--owner-source" (name owner-source)]
create-empty-db? (conj "--create-empty-db")
(seq embedding-endpoint) (conj "--embedding-endpoint" embedding-endpoint)
(seq embedding-model-id) (conj "--embedding-model-id" embedding-model-id)))
env (js/Object.assign #js {} (.-env js/process) #js {:ELECTRON_RUN_AS_NODE "1"})]
env (js/Object.assign #js {} (.-env js/process)
#js {:ELECTRON_RUN_AS_NODE "1"
:LOGSEQ_DAEMON_USERNAME username
:LOGSEQ_DAEMON_PASSWORD password})]
(when (and (= :electron owner-source)
(not (seq embedding-endpoint)))
(js-delete env embedding-url-env))

View File

@@ -7,6 +7,7 @@
[frontend.test.node-helper :as node-helper]
[frontend.worker.db-core :as db-core]
[frontend.worker.db-worker-node :as db-worker-node]
[frontend.worker.db-worker-node-auth :as db-worker-node-auth]
[frontend.worker.db-worker-node-lock :as db-lock]
[frontend.worker.platform.node :as platform-node]
[goog.object :as gobj]
@@ -20,6 +21,17 @@
[logseq.db-worker.log :as db-worker-log]
[promesa.core :as p]))
;; Fase 1 (server-client migration) made a credential mandatory for every
;; db-worker-node daemon, bootstrapped from these env vars on first launch
;; against each test's fresh root-dir (see db-worker-node-auth/ensure-credential!).
;; Tests that hit the HTTP surface directly must log in with the same
;; username/password to get a bearer token -- see `auth-headers!` below.
(def ^:private test-daemon-username "db-worker-node-test-user")
(def ^:private test-daemon-password "db-worker-node-test-password")
(aset (.-env js/process) "LOGSEQ_DAEMON_USERNAME" test-daemon-username)
(aset (.-env js/process) "LOGSEQ_DAEMON_PASSWORD" test-daemon-password)
(defn- http-request
[opts body]
(p/create
@@ -56,17 +68,35 @@
:method "GET"}
nil))
(defn- auth-headers!
[host port]
(p/let [payload (js/JSON.stringify
(clj->js {:username test-daemon-username
:password test-daemon-password
:deviceLabel "test"}))
{:keys [status body]} (http-request {:hostname host
:port port
:path "/v1/auth/login"
:method "POST"
:headers {"Content-Type" "application/json"}}
payload)]
(when (not= 200 status)
(println "[db-worker-node-test] login failed" {:status status :body body}))
{"Content-Type" "application/json"
"Authorization" (str "Bearer " (:token (js->clj (js/JSON.parse body) :keywordize-keys true)))}))
(defn- invoke
[host port method args]
(let [payload (js/JSON.stringify
(clj->js {:method method
:argsTransit (ldb/write-transit-str args)}))]
(p/let [{:keys [status body]}
(p/let [headers (auth-headers! host port)
{:keys [status body]}
(http-request {:hostname host
:port port
:path "/v1/invoke"
:method "POST"
:headers {"Content-Type" "application/json"}}
:headers headers}
payload)
parsed (js->clj (js/JSON.parse body) :keywordize-keys true)]
(when (not= 200 status)
@@ -83,21 +113,23 @@
(let [payload (js/JSON.stringify
(clj->js {:method method
:argsTransit (ldb/write-transit-str args)}))]
(http-request {:hostname host
:port port
:path "/v1/invoke"
:method "POST"
:headers {"Content-Type" "application/json"}}
payload)))
(p/let [headers (auth-headers! host port)]
(http-request {:hostname host
:port port
:path "/v1/invoke"
:method "POST"
:headers headers}
payload))))
(defn- invoke-import-db-binary-raw
[host port repo payload]
(http-request {:hostname host
:port port
:path (str "/v1/import-db-binary?repo=" (js/encodeURIComponent repo))
:method "POST"
:headers {"Content-Type" "application/octet-stream"}}
payload))
(p/let [headers (auth-headers! host port)]
(http-request {:hostname host
:port port
:path (str "/v1/import-db-binary?repo=" (js/encodeURIComponent repo))
:method "POST"
:headers (assoc headers "Content-Type" "application/octet-stream")}
payload)))
(defn- lock-path
[root-dir repo]
@@ -385,18 +417,29 @@
"db-worker-node-20240109.log"]
remaining)))))
(deftest db-worker-node-parse-args-ignores-host-and-port
(deftest db-worker-node-parse-args-parses-host-and-port
(let [parse-args #'db-worker-node/parse-args
result (parse-args #js ["node" "dist/db-worker-node.js"
"--host" "0.0.0.0"
"--port" "1234"
"--repo" "logseq_db_parse_args"
"--root-dir" "/tmp/logseq-root"])]
(is (nil? (:host result)))
(is (nil? (:port result)))
(is (= "0.0.0.0" (:host result)))
(is (= 1234 (:port result)))
(is (= "logseq_db_parse_args" (:repo result)))
(is (= "/tmp/logseq-root" (:root-dir result)))))
(deftest db-worker-node-parse-args-collects-allowed-origins-and-hosts
(let [parse-args #'db-worker-node/parse-args
result (parse-args #js ["node" "dist/db-worker-node.js"
"--allowed-origin" "https://a.example"
"--allowed-origin" "https://b.example"
"--allowed-host" "example.local:1234"
"--repo" "logseq_db_parse_args"
"--root-dir" "/tmp/logseq-root"])]
(is (= #{"https://a.example" "https://b.example"} (:allowed-origins result)))
(is (= #{"example.local:1234"} (:allowed-hosts result)))))
(deftest db-worker-node-parse-args-ignores-auth-token
(let [parse-args #'db-worker-node/parse-args
result (parse-args #js ["node" "dist/db-worker-node.js"
@@ -907,19 +950,24 @@
(start-daemon! {:root-dir data-dir
:repo repo})
health (http-get host port "/healthz")
;; /healthz is unauthenticated by design (readiness polling
;; happens before any session exists), but every other route
;; -- including one that doesn't exist -- requires a bearer
;; token first: an unauthenticated caller shouldn't learn
;; whether a route exists.
missing-readyz (http-get host port "/readyz")
health-body (js->clj (js/JSON.parse (:body health)) :keywordize-keys true)
server-list-contents (.toString (fs/readFileSync server-list-file) "utf8")
_ (do
(reset! daemon {:host host :port port :stop! stop!})
(is (= 200 (:status health)))
(is (= 404 (:status missing-readyz)))
(is (= 401 (:status missing-readyz)))
(is (= repo (:repo health-body)))
(is (= "ready" (:status health-body)))
(is (= host (:host health-body)))
(is (= port (:port health-body)))
(is (= (.-pid js/process) (:pid health-body)))
(is (= (node-path/resolve data-dir) (:root-dir health-body)))
(is (not (contains? health-body :pid)))
(is (not (contains? health-body :root-dir)))
(is (contains? health-body :owner-source))
(is (contains? health-body :revision))
(is (string/includes? server-list-contents (str (.-pid js/process) " " port))))
@@ -1557,17 +1605,31 @@
(deftest db-worker-node-desktop-and-cli-share-same-graph-daemon
(async done
(let [daemon (atom nil)
(let [daemon-state (atom nil)
data-dir (node-helper/create-tmp-dir "db-worker-desktop-cli")
config-path (node-path/join data-dir "cli.edn")
server-list-file (cli-config/server-list-path data-dir)
repo (str "logseq_db_desktop_cli_" (subs (str (random-uuid)) 0 8))
now (js/Date.now)
page-uuid (random-uuid)]
page-uuid (random-uuid)
;; This test's own `invoke` calls authenticate via the shared
;; test-daemon-username/password constants (see auth-headers!
;; above); ensure-server! (used below to simulate the CLI
;; discovering the desktop-started daemon) instead logs in via
;; the root-dir-local device credential that spawn-server!
;; seeds in production (daemon/ensure-local-credential!). Seed
;; both directly with the same test constants -- rather than
;; via daemon/ensure-local-credential!'s random password, or
;; via the global LOGSEQ_DAEMON_USERNAME/PASSWORD env vars --
;; so every path in this test agrees on one credential.
_ (db-worker-node-auth/set-credential! data-dir test-daemon-username test-daemon-password)
_ (fs/writeFileSync (node-path/join data-dir "db-worker-local-credential.json")
(js/JSON.stringify (clj->js {:username test-daemon-username
:password test-daemon-password})))]
(-> (p/let [{:keys [host port stop!]}
(start-daemon! {:root-dir data-dir
:repo repo})
_ (reset! daemon {:stop! stop!})
_ (reset! daemon-state {:stop! stop!})
_ (invoke host port "thread-api/create-or-open-db" [repo {}])
_ (invoke host port "thread-api/transact"
[repo
@@ -1601,7 +1663,7 @@
(p/catch (fn [e]
(is false (str "unexpected error: " e))))
(p/finally (fn []
(if-let [stop! (:stop! @daemon)]
(if-let [stop! (:stop! @daemon-state)]
(-> (stop!)
(p/finally (fn [] (done))))
(done))))))))

View File

@@ -117,7 +117,8 @@
daemon/http-request (fn [_]
(swap! shutdown-calls inc)
(p/resolved {:status 200 :body ""}))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:owner-source :cli
:expected-revision "expected-revision"}
@@ -152,7 +153,8 @@
(if matched?
true
(throw (ex-info "timed out" {:code :timeout})))))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:owner-source :cli
:expected-revision "expected-revision"}
@@ -204,7 +206,8 @@
2 []
[new-server]))))
daemon/wait-for-lock (fn [_] (p/resolved true))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(log/add-handler log-handler)
(cli-server/ensure-server! {:root-dir root-dir
:owner-source :cli
@@ -265,7 +268,8 @@
2 []
[new-server]))))
daemon/wait-for-lock (fn [_] (p/resolved true))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:owner-source :cli
:expected-revision "expected-revision"}
@@ -313,7 +317,8 @@
2 []
[new-server]))))
daemon/wait-for-lock (fn [_] (p/resolved true))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:owner-source :cli
:expected-revision "expected-revision"}
@@ -361,7 +366,8 @@
2 []
[wrong-server]))))
daemon/wait-for-lock (fn [_] (p/resolved true))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:owner-source :cli
:expected-revision "expected-revision"}
@@ -408,7 +414,8 @@
2 []
[wrong-server]))))
daemon/wait-for-lock (fn [_] (p/resolved true))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/start-server! {:root-dir root-dir
:owner-source :cli
:expected-revision "expected-revision"}
@@ -488,6 +495,8 @@
:pid (.-pid js/process)
:owner-source "cli"
:revision "server-revision"})))
"/v1/auth/login" (do (.writeHead res 200 #js {"Content-Type" "application/json"})
(.end res (js/JSON.stringify #js {:ok true :token "test-token"})))
(do (.writeHead res 404 #js {"Content-Type" "text/plain"})
(.end res "not-found")))))]
(.listen server 0 host
@@ -668,7 +677,8 @@
:owner-source :cli
:revision (version/revision)
:status :ready}])))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:config-path config-path
:create-empty-db? true}
@@ -712,7 +722,8 @@
:owner-source :cli
:revision (version/revision)
:status :ready}])))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:profile-session session}
repo))
@@ -757,7 +768,8 @@
(p/resolved (if (< call 3)
[]
[server]))))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir} repo))
(p/then (fn [config]
(is (= "http://127.0.0.1:9311" (:base-url config)))
@@ -808,7 +820,8 @@
:root-dir root-dir
:revision "server-revision"}))})))
(p/resolved {:status 200 :body ""})))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir
:config-path config-path
:expected-revision "server-revision"}
@@ -892,7 +905,8 @@
(p/resolved (if @spawned?
[remote-server local-server]
[remote-server])))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir root-dir-b
:config-path config-path}
repo))
@@ -932,7 +946,8 @@
:revision (version/revision)
:status :ready
:root-dir symlink-root-dir}]))
daemon/wait-for-ready (fn [_] (p/resolved true))]
daemon/wait-for-ready (fn [_] (p/resolved true))
daemon/login! (fn [_] (p/resolved "test-token"))]
(cli-server/ensure-server! {:root-dir real-root-dir}
repo))
(p/then (fn [config]