mirror of
https://github.com/volcengine/OpenViking.git
synced 2026-09-28 19:53:23 +08:00
main
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 t0saki
|
f905562534 |
feat(plugins): stdio MCP proxy, remote marketplace install, and type-quota recall for memory plugins (#3039)
* feat: add memory plugin mcp harness
* refactor: vendor shared memory plugin modules
* feat: add type quota recall api
* feat: commit codex memory by token threshold
* feat: capture codex tool calls as parts
* feat: add claude skill experience recall
* chore: fix lint in type quota recall server files
* feat: remote marketplace install with unified openviking naming
- Fix root .claude-plugin/marketplace.json git-subdir discriminator key
("type" -> "source"); claude plugin validate now passes.
- Unified installer gains --source remote|archive|dev: remote registers a
synthesized git-subdir marketplace for Claude Code and a git marketplace
for Codex (no repo clone); archive consumes the slim TOS marketplace zip;
dev registers the checkout's examples/ directory for both harnesses.
- One marketplace name (openviking) across all modes and harnesses, so the
plugin id is always openviking-memory@openviking; installer migrates old
openviking-plugins-local registrations and config.toml sections.
- Restore legacy Claude Code (<2.0) support: claude mcp add (stdio proxy)
plus node-based hooks merge into ~/.claude/settings.json.
- Restore optional statusline registration (fetches sources on opt-in).
- Checkbox TUI harness selection via /dev/tty with non-tty fallback.
- Add examples/.agents/plugins/marketplace.json so Codex directory installs
drop the synthetic symlink marketplace.
- Add shared setup wizard (scripts/setup.mjs) for pure-marketplace installs.
- release-tos.yml: upload memory-plugin-shared/install.sh and build/upload
the memory-plugin-marketplace zip; tos-install.sh prefers it and pins all
fetches to TOS via OPENVIKING_SHARED_INSTALL_URL.
- CI: bash -n on installer scripts; marketplace contract tests updated.
* fix(installer): register Claude remote marketplace as a directory
File-type marketplaces (bare marketplace.json path) make Claude Code derive
a wrong installLocation and 'marketplace update' fails with EISDIR. Write
the synthesized manifest to <dir>/.claude-plugin/marketplace.json and add
the directory instead; compare registered sources by exact match so the
old file registration migrates cleanly.
* feat(statusline): show model name and native-style context percentage
A custom statusLine replaces Claude Code's native line including its context
indicator, so reproduce it from the statusline stdin payload: 'Fable 5 ·
ctx 42%' right after the health segment, with native color thresholds
(<70% dim, 70-89% yellow, >=90% red). Falls back from used_percentage to
remaining_percentage to token counts, and stays visible in bypass mode
since it describes the CC conversation, not OV. Opt out with
OPENVIKING_STATUSLINE_CTX=off. Line cap raised 80 -> 100 visible chars.
* fix(installer): keep checkout progress off stdout in plugin_dir_on_disk
Callers capture the function's stdout, so ensure_checkout's info lines were
concatenated into the statusline command registered in settings.json.
* fix(installer): re-register codex git marketplace instead of upgrading
Codex doesn't expose which --ref a git marketplace was added with, and
'marketplace upgrade' refreshes the old ref — so a URL match must not skip
re-registration or a ref override installs the wrong snapshot. Also remove
the stale pre-unification plugin cache directory during migration.
* fix(installer): include .agents in codex sparse checkout
A plugin-dir-only sparse checkout omits the repo-root marketplace manifest
and fails with 'marketplace root does not contain a supported manifest'.
Adding --sparse .agents keeps the snapshot slim (~7.5M vs full repo).
* feat(installer): bilingual prompts, dist channel selection, and TOS git marketplace for codex
- Interactive language selection (English/中文, --lang, auto-detected from
locale); every user-facing prompt is bilingual.
- Download-source selection (--dist github|tos, prompted interactively):
github keeps the remote marketplaces; tos serves GitHub-blocked regions.
- Credentials step now always shows the current ovcli.conf values (masked
key) and offers keep-or-reconfigure instead of silently reusing them.
- Codex on TOS installs from a TOS-hosted git repo over dumb HTTP and keeps
remote updates (codex plugin marketplace upgrade); falls back to the
archive directory if the repo is unavailable. release-tos.yml builds and
uploads the single-commit bare repo (repack + update-server-info).
- Claude Code on TOS warns that directory marketplaces cannot auto-update.
- tos-install.sh bootstraps shrink to TOS_BASE + --dist tos.
- Docs (READMEs, agent-integrations pages, image cards, en+zh) now all use
the single shared installer and drop the deleted wrapper instructions.
* feat(installer): unify all choice prompts on an arrow-key TUI menu
Language, download source, connection mode, keep-or-reconfigure
credentials, statusline enable/replace, and legacy-mode confirmation all
render as the same single-select menu (arrow keys / digit shortcuts /
enter, radio-style highlight) instead of mixed numbered and y/N prompts.
Falls back to numbered input when /dev/tty can't be drawn on and to the
default choice when non-interactive. Free-text fields (URL, API key) stay
line inputs; the harness picker keeps its checkbox multi-select.
* fix(installer): stop piping plugin lists into grep -q under pipefail
grep -q exits on first match and SIGPIPEs the producer, so with pipefail
the 'codex plugin list | grep -q' check read as a miss every time (codex's
list is long; claude's short list masked the bug). Capture the output and
substring-match in bash instead — validation no longer false-warns.
Also: drop the stdio-proxy line from the Done summary; always offer the
install-source menu unless --dist/--source was given (with a checkout the
menu gains a dev option and defaults to it); surface the Claude-on-TOS
no-auto-update warning at source resolution instead of after install.
* fix: unignore examples/memory-plugin-shared/lib and commit the shared modules
The Python build-artifact 'lib/' gitignore rule silently swallowed the
shared plugin module source, so CI checkouts had only the vendored copies
and sync.test.mjs failed with ENOENT on the source directory.
* fix(recall): budget summary/uri fallbacks and sanitize non-finite scores
max_chars is the recall API's contract, but only full fragments counted
toward it — VikingBot's client-side heuristic, faithfully ported, lets
summary and uri fallbacks render far past the budget (repro: max_chars=100
rendered 548 chars). Every fragment now counts; oversized summaries degrade
to uri fragments and entries that can't even fit a uri line are dropped
(reported via stats.dropped). VikingBot itself is intentionally unchanged.
Also run _sanitize_floats over the /recall response like the neighboring
/find and /search routes, so inf/nan scores return 0.0 instead of a 500.
|
||
|
|
e92180a7e1 |
feat(plugin/codex): add lifecycle hooks (recall, capture, pre-compact) to codex-memory-plugin (#1957)
* feat(plugin/codex): add lifecycle hooks (recall, capture, pre-compact)
Brings the codex-memory-plugin to feature parity with the claude-code-memory-plugin
by wiring the four Codex lifecycle hooks via `hooks.json`:
- SessionStart -> bootstrap-runtime.mjs (npm ci into ${CODEX_PLUGIN_DATA}/runtime)
- UserPromptSubmit -> auto-recall.mjs (search OV, inject via hookSpecificOutput.additionalContext)
- Stop -> auto-capture.mjs (incremental transcript capture + last_assistant_message commit)
- PreCompact -> pre-compact-capture.mjs (full transcript -> single OV session -> commit)
Differences from the Claude Code plugin baked into the scripts:
- Codex output schema does not allow `decision: "approve"`; no-op is `{}`
- Stop/PreCompact only support `systemMessage`, not `additionalContext`
- Plugin envs are CODEX_PLUGIN_ROOT / CODEX_PLUGIN_DATA
- Config section is `codex` (was `claude_code`); config file defaults to
`~/.openviking/ovcli.conf`, falling back to legacy `~/.openviking/ov.conf`
Other changes:
- src/memory-server.ts now reads ovcli.conf-style configs (top-level `url`,
`api_key`, `account`, `user`, `agent_id`) so the plugin works against
hosted OpenViking deployments out of the box. Env-var-only operation
(OPENVIKING_URL set, no config file) is also supported.
- .mcp.json points at scripts/start-memory-server.mjs, which boots the same
runtime the hooks use, so the MCP path benefits from npm-ci bootstrap.
- README rewritten with architecture diagram, validation SOP, configuration
reference, and a Codex-vs-Claude-Code differences table.
Validated end-to-end against an OpenViking deployment:
- Auto-recall returns ranked memories with full content and emits
hookSpecificOutput.additionalContext.
- Auto-capture (last_assistant_message path) creates a session, commits, and
the OV pipeline extracts events + preferences within ~60s.
- Pre-compact-capture posts a full 4-turn transcript to one OV session,
commits with archived=true, and produces structured leaf memories
(preferences, events, entities) under viking://user/<user>/memories/.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(plugin/codex): drop SessionStart, split Stop=add_message vs PreCompact=commit
Codex's `Stop` hook fires per turn, not at session end, so committing per-Stop
over-fragments memory extraction. And codex re-fires `SessionStart` on short
reconnects, so registering an `npm ci` bootstrap there reinstalls the runtime
unnecessarily.
This change keeps one long-lived OpenViking session per codex `session_id`
across all `Stop` invocations, and only triggers the OV memory extractor on
`PreCompact` (or via an idle-sweep best-effort commit when codex exits without
compacting).
- hooks.json: drop SessionStart entry; keep UserPromptSubmit/Stop/PreCompact
- scripts/session-state.mjs (new): per-codex-session state under
~/.openviking/codex-plugin-state/, tracks ovSessionId + capturedTurnCount
- scripts/auto-capture.mjs (Stop): incremental add_message only, idle-sweep at
the tail to commit stale codex sessions (default IDLE_TTL=30 min, override
with OPENVIKING_CODEX_IDLE_TTL_MS)
- scripts/pre-compact-capture.mjs (PreCompact): catch-up append + commit the
long-lived OV session, then null out ovSessionId so the next Stop opens a
fresh OV session for the post-compact half
- MCP runtime install stays lazy in start-memory-server.mjs (already there);
no SessionStart hook means short reconnects don't re-trigger npm ci
- VERIFICATION.md: end-to-end SOP against a live OV server (~3 min)
- bump plugin to 0.3.0
Verified end-to-end against ov.zaynjarvis.com:
Stop adds turns idempotently and incrementally; PreCompact commits to
history/archive_001/ with extractor producing memories under
viking://user/<user>/memories/profile.md after ~30 s; post-compact Stop
opens a fresh OV session; idle-sweep commits stale state files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(plugin/codex): replace idle-sweep with SessionStart(source=clear) commit
Per Zayn's followup ("非必要不要加 idle commit"): drop the idle-sweep added
in the previous commit and use codex's actual context-disappearing signal —
SessionStart with source=clear — to commit orphaned sessions.
Codex hook signal map:
- /compact → PreCompact ✅ commit (already)
- /clear → SessionStart(source=clear) for the NEW session_id;
the prior transcript is orphaned. Now committed.
- /new → SessionStart(source=startup); ambiguous with fresh
codex startup, so we don't act on it.
- /resume / short reconnect → SessionStart(source=resume|startup); no-op
to avoid corrupting still-active sessions.
- SIGTERM/Ctrl+C/exit → no hook fires. Documented as a known gap; users
should /compact before /exit if they want commit.
Changes:
- new scripts/session-start-commit.mjs: gates internally on source=clear,
iterates listStates(), and commits any state file whose codexSessionId
!= the new SessionStart session_id, then clears that state file
- hooks/hooks.json: re-register SessionStart pointing at the new script
(timeout 30s)
- scripts/auto-capture.mjs: remove sweepIdleSessions() and
IDLE_TTL_MS env handling; Stop is now strictly add_message
- README/VERIFICATION.md: update arch diagram, replace idle-sweep step
with SessionStart(source=clear) verify (positive + negative paths),
add "Known gap: SIGTERM/exit are silent" section
- bump to 0.3.1
Verified end-to-end against ov.zaynjarvis.com:
Stop add+idempotent ✓
SessionStart source=startup → {} ✓
SessionStart source=resume → {} ✓
SessionStart source=clear → committed prior OV session, history/archive_001/
appeared, profile.md gained "Favorite snack: dark chocolate" within 30 s.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(plugin/codex): SessionStart matcher = "clear" (native dispatcher gate)
Codex's hooks dispatcher matches the SessionStart hook's `matcher` field
against the SessionStart `source` value. Setting matcher to "clear" means
codex won't even spawn our script on `source=startup` or `source=resume`
(short reconnects); we previously gated this in-script. The internal
source check in session-start-commit.mjs is kept as defense-in-depth.
Source: codex-rs/hooks/src/events/session_start.rs `select_handlers(...,
matcher_input: Some(request.source.as_str()))` and
codex-rs/hooks/src/events/common.rs `is_exact_matcher` — "clear" is
all-alphanumeric so it's matched as exact equality, not regex.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(plugin/codex): active-window heuristic + idle-TTL sweep at SessionStart (v0.4.0)
Source of truth: examples/codex-memory-plugin/DESIGN.md (added in this commit).
Behavioral changes:
- SessionStart matcher widens from `clear` to `clear|startup`. Both sources
run the same active-window heuristic; `resume` is a hard no-op (still fires
on short reconnects).
- Heuristic (DESIGN.md §3): count state files (excluding new session_id) within
ACTIVE_WINDOW_MS (default 2 min). 0 → noop, 1 → commit it (just-ended
session), ≥2 → skip and rely on idle TTL. Tunable via
OPENVIKING_CODEX_ACTIVE_WINDOW_MS.
- Idle-TTL sweep returns at the tail of session-start-commit.mjs only (not
every Stop). Default IDLE_TTL_MS = 30 min via OPENVIKING_CODEX_IDLE_TTL_MS.
Catches SIGTERM/Ctrl+C/`/exit` orphans and the ≥2-active skip path.
- Stop hook deliberately does NOT sweep — state-write-on-every-turn already
gives us the freshness signal. Marker comment added.
- Stop hook adds post-compact transcript-shrink defense: if
allTurns.length < state.capturedTurnCount, reset capturedTurnCount = 0.
- Commit-on-failure preserves state everywhere (PreCompact, heuristic,
idle sweep). A non-2xx /commit no longer clears ovSessionId; the next
sweep retries.
- session-state.mjs saveState now uses atomic write (tmpfile + rename) for
crash safety. listStates ignores the brief `<id>.json.tmp` window.
Bump: package.json + .codex-plugin/plugin.json → 0.4.0.
Docs: README "How It Works" gained a DESIGN.md pointer and rewrites the
SessionStart section to reflect heuristic + idle TTL. VERIFICATION.md step 6
now exercises all four heuristic branches (0/1/≥2 active, idle TTL, resume).
Phase-2 resume context inject documented in DESIGN.md but explicitly out of
scope here.
Verified locally with synthetic stdin tests against a fake OV server:
1-active commit, ≥2-active skip, idle TTL sweep, resume noop,
unreachable-server keeps state.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(plugin/codex): align config loading with claude-code plugin
Addresses three review points on PR #1957:
1. Honor OPENVIKING_CLI_CONFIG_FILE for the ovcli.conf override path
(matches the convention used by `ov` CLI and claude-code-memory-plugin).
OPENVIKING_CONFIG_FILE stays as the ov.conf override; for backward
compat it still works when pointed at an ovcli-shaped file.
2. Strict env-first priority for every connection / identity field
(baseUrl, apiKey, account, user, agentId). Env vars now win over
ovcli.conf, which wins over ov.conf's codex.* block / server.*,
which wins over built-in defaults.
3. Unify hook and MCP-server config loading: src/memory-server.ts now
imports loadConfig from scripts/config.mjs (relative path stays
valid post-compile because servers/ and scripts/ are siblings),
eliminating the divergent account/user/agentId fallback chains
the PR-Agent reviewer flagged.
Auth header: emit Authorization: Bearer (primary, required by OpenViking
Cloud) plus the legacy X-API-Key during the transition window. All six
fetch sites updated (4 hook scripts + memory-server.ts + compiled
servers/memory-server.js).
README: document the new resolution chain, OPENVIKING_CLI_CONFIG_FILE,
OPENVIKING_BEARER_TOKEN alias, and the Authorization: Bearer migration.
* docs(plugin/codex): put installation first
* fix(plugin/codex): harden runtime and capture paths
* docs(plugin/codex): align local marketplace name
* docs(plugin/codex): add one-line installer
* fix(plugin/codex): support branch installer testing
* fix(plugin/codex): keep installer env surface stable
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zhengxiao.wu <zhengxiao.wu@bytedance.com>
|