* fix(ragfs): preserve cache visibility on partial S3 deletes
Surface exact and per-object S3 deletion failures, while always invalidating the affected directory and stat cache scope after a recursive delete attempt.
Source-PR: #3407
Original-Commit: 8d6addf28e
* fix(session): preserve legacy policy and peer identity compatibility
Parse string false and other legacy boolean-like memory policy values without silently enabling extraction or breaking persisted configs. Encode mixed-script peers losslessly, while retaining their former lossy IDs as read-only retrieval and extraction aliases.
Source-PR: #3422
Original-Commit: 0dfd5a9ed9
* fix(memory): drain timer flush tasks during shutdown
Retain the shielded timer flush task and await it when close cancels the timer loop, so batch failures are observed and submitters are resolved without unhandled task exceptions.
Source-PR: #3438
Original-Commit: ca1d74e164
* fix(storage): preserve peer isolation and cache correctness
* fix(ingest): reserve encoded peer namespace
* ci: skip embedding-dependent resource test without secrets
* fix(ragfs): invalidate caches after partial remove
---------
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* feat: implement server-resolved OpenViking Assets manifests
Add the openviking-assets/1 declaration flow with server-owned configuration parsing and native Rust CLI execution.
- Resolve one flat Manifest against one Catalog through an authenticated server endpoint with strict schema and Git semantic validation.
- Reject recursive includes and unsafe clone URLs; return a resolved plan without submitting resources or running server-side batches.
- Keep local credential aliases, manifest state, dry-run, failure isolation, and per-asset create/sync orchestration in the CLI.
- Generate normalized stable asset identities on the server and remove the CLI direct SHA-1 dependency.
- Update flat examples and add server resolver/API plus Rust CLI coverage.
* feat: implement server-resolved OpenViking Assets manifests
* feat: implement server-resolved OpenViking Assets manifests
* fix(pathlock): tolerate missing lock token after recursive delete
* feat: implement server-resolved OpenViking Assets manifests
* feat: implement server-resolved OpenViking Assets manifests
* feat(agent-evolution): reload global switch at commit time
* feat(agent-evolution): expose configured account in status
* test(agent-evolution): cover account in status response
* fix(agent-evolution): align live config reload semantics
* fix(agent-evolution): tolerate non-object live config
* fix(usage-reporter): use snake case count fields
* feat(usage-reporter): add file log sink
* fix
* fix: address live reload and usage sink review findings
* fix(usage-reporter): complete file sink compatibility
* fix: make experience snapshot source unambiguous
* docs(usage-reporter): align count record implementation plan
* fix: address agent evolution review blockers
* fix(usage-reporter): preserve Windows rollover deadline
* fix(usage-reporter): encode file records as JSON envelopes
* fix(usage-reporter): use snake case unique id
* fix(bot): only require VKE credentials when the TOS storage path needs them
Sweep findings: C-14. Gate AK/SK validation on an actual TOS deployment and drop the unused cluster ID.
(cherry picked from commit 8790ba509f)
* fix(server): apply configured temp_upload.default_mode to uploads
Sweep findings: B-11, D-01. Apply the documented configured upload mode when requests omit it.
(cherry picked from commit a0dc2498e8)
* fix(bot): make one-click Docker deployment generate a working config and port mapping
Sweep findings: C-12. Generate the active ov.conf and keep gateway and Docker ports aligned.
(cherry picked from commit c22af83ab6)
* fix(server): make --bot work and propagate bot flags to workers
Sweep findings: B-09, B-15. Honor the public Bot alias and replay resolved Bot settings in worker processes.
(cherry picked from commit 32a898ca14)
* fix(docker): derive entrypoint/health port from configured server port
Sweep findings: F-06. Keep server startup and every container health check on the same effective port.
(cherry picked from commit 000795c7e3)
---------
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* fix(server): attribute MCP traffic in observability (route + identity)
MCP requests were audited as route=/__unmatched__, account_id=__unknown__
because (1) the /mcp app is registered as a plain Starlette Route so
scope["route"] is never set, and (2) _IdentityASGIMiddleware resolved
identity without calling update_root_span_identity.
Register /mcp via a _ScopedRoute subclass that sets child_scope["route"]
on match (mirroring APIRoute.matches) and stamp root-span identity after
resolution. 404 fallbacks and middleware code are untouched.
* test(server): cover MCP scope route resolution and root-span identity stamping
Assert the /mcp route sets scope["route"] on match (and that unmatched
paths still fall back without it), and that _IdentityASGIMiddleware stamps
the resolved account/user onto the root span attributes.
* fix(server): attribute MCP traffic in observability (route + identity)
MCP requests were audited as route=/__unmatched__, account_id=__unknown__
because (1) the /mcp app is registered as a plain Starlette Route so
scope["route"] is never set, and (2) _IdentityASGIMiddleware resolved
identity without calling update_root_span_identity.
Register /mcp via a _ScopedRoute subclass that sets child_scope["route"]
on match (mirroring APIRoute.matches) and stamp root-span identity after
resolution. 404 fallbacks and middleware code are untouched.
FastMCP derives tool input schemas from Python type hints, so Optional/
Union parameters become anyOf nodes with no top-level type, and nested
models become $ref/$defs. Valid JSON Schema, but clients that forward
these schemas verbatim to strict function-calling APIs break: Gemini's
OpenAPI 3.0 subset rejects the whole request (400: schema didn't specify
the schema type field), and n8n's JSON-schema-to-Zod conversion can
silently fall back and drop tool arguments entirely.
Rewrite the advertised schemas after registration: drop null branches,
collapse unions to their most general branch, inline $refs, and ensure
every node carries an explicit type. Runtime argument validation still
uses the original function signatures, so union parameters keep
accepting every branch (e.g. read still takes a bare URI string even
though the schema advertises an array).
Also type recall's other_peer_penalty honestly as
Optional[Union[float, Dict[str, float]]] instead of Optional[Any],
which produced an empty {} schema node.
* feat(connector): delegate add_resource imports to external Connector
Opt-in integration that routes add_resource data fetching and parsing
to external Connector service; the Connector stages source data and
calls back into OV through the standard add_resource pipeline.
- add ConnectorClient wrapping the control plane's inner doc/add and
task/info endpoints
- add [connector] config section: enable, connector/tracker endpoint
URLs, timeout_seconds, poll_interval_ms, allowed_add_types
- route add_resource via Connector when enabled and args.add_type is
in allowed_add_types; otherwise fall back to the standard pipeline
with an info log
- track imports as connector_import TaskRecords and poll Connector
task status in the background until terminal state or timeout
* fix(connector): delegate add_resource imports to external Connector
#2921 persisted the DCR scope when the registrar sends one and started
advertising scopes_supported=["mcp"] in the PRM document. ChatGPT's DCR
omits the scope field, so its client registers scope-less, then requests
the advertised scope=mcp at /authorize and gets bounced back with
error=invalid_scope before any consent page renders.
- app.py: pass default_scopes=["mcp"] to ClientRegistrationOptions so
scope-less registrations get the default grant; valid_scopes stays
unset so clients that register their own scope strings are not
rejected at DCR time
- provider.py: single-source the scope as MCP_SCOPE; get_client() falls
back to it for NULL-scope rows, repairing already-registered clients
without migration or re-registration
- router.py: reuse MCP_SCOPE in the PRM scopes_supported
- tests: provider fallback unit tests + end-to-end scope-less DCR and
legacy NULL-scope client authorize regressions
中文：抢写锁失败时改抛 ResourceBusyError，复用现有 CONFLICT 到 HTTP 409 的错误映射。
English: align ov write lock contention with existing conflict handling used by other resource operations.
Co-authored-by: chenpengfei <chenpengfei@bytedance.com>
* feat: add memory plugin mcp harness
* refactor: vendor shared memory plugin modules
* feat: add type quota recall api
* feat: commit codex memory by token threshold
* feat: capture codex tool calls as parts
* feat: add claude skill experience recall
* chore: fix lint in type quota recall server files
* feat: remote marketplace install with unified openviking naming
- Fix root .claude-plugin/marketplace.json git-subdir discriminator key
("type" -> "source"); claude plugin validate now passes.
- Unified installer gains --source remote|archive|dev: remote registers a
synthesized git-subdir marketplace for Claude Code and a git marketplace
for Codex (no repo clone); archive consumes the slim TOS marketplace zip;
dev registers the checkout's examples/ directory for both harnesses.
- One marketplace name (openviking) across all modes and harnesses, so the
plugin id is always openviking-memory@openviking; installer migrates old
openviking-plugins-local registrations and config.toml sections.
- Restore legacy Claude Code (<2.0) support: claude mcp add (stdio proxy)
plus node-based hooks merge into ~/.claude/settings.json.
- Restore optional statusline registration (fetches sources on opt-in).
- Checkbox TUI harness selection via /dev/tty with non-tty fallback.
- Add examples/.agents/plugins/marketplace.json so Codex directory installs
drop the synthetic symlink marketplace.
- Add shared setup wizard (scripts/setup.mjs) for pure-marketplace installs.
- release-tos.yml: upload memory-plugin-shared/install.sh and build/upload
the memory-plugin-marketplace zip; tos-install.sh prefers it and pins all
fetches to TOS via OPENVIKING_SHARED_INSTALL_URL.
- CI: bash -n on installer scripts; marketplace contract tests updated.
* fix(installer): register Claude remote marketplace as a directory
File-type marketplaces (bare marketplace.json path) make Claude Code derive
a wrong installLocation and 'marketplace update' fails with EISDIR. Write
the synthesized manifest to <dir>/.claude-plugin/marketplace.json and add
the directory instead; compare registered sources by exact match so the
old file registration migrates cleanly.
* feat(statusline): show model name and native-style context percentage
A custom statusLine replaces Claude Code's native line including its context
indicator, so reproduce it from the statusline stdin payload: 'Fable 5 ·
ctx 42%' right after the health segment, with native color thresholds
(<70% dim, 70-89% yellow, >=90% red). Falls back from used_percentage to
remaining_percentage to token counts, and stays visible in bypass mode
since it describes the CC conversation, not OV. Opt out with
OPENVIKING_STATUSLINE_CTX=off. Line cap raised 80 -> 100 visible chars.
* fix(installer): keep checkout progress off stdout in plugin_dir_on_disk
Callers capture the function's stdout, so ensure_checkout's info lines were
concatenated into the statusline command registered in settings.json.
* fix(installer): re-register codex git marketplace instead of upgrading
Codex doesn't expose which --ref a git marketplace was added with, and
'marketplace upgrade' refreshes the old ref — so a URL match must not skip
re-registration or a ref override installs the wrong snapshot. Also remove
the stale pre-unification plugin cache directory during migration.
* fix(installer): include .agents in codex sparse checkout
A plugin-dir-only sparse checkout omits the repo-root marketplace manifest
and fails with 'marketplace root does not contain a supported manifest'.
Adding --sparse .agents keeps the snapshot slim (~7.5M vs full repo).
* feat(installer): bilingual prompts, dist channel selection, and TOS git marketplace for codex
- Interactive language selection (English/中文, --lang, auto-detected from
locale); every user-facing prompt is bilingual.
- Download-source selection (--dist github|tos, prompted interactively):
github keeps the remote marketplaces; tos serves GitHub-blocked regions.
- Credentials step now always shows the current ovcli.conf values (masked
key) and offers keep-or-reconfigure instead of silently reusing them.
- Codex on TOS installs from a TOS-hosted git repo over dumb HTTP and keeps
remote updates (codex plugin marketplace upgrade); falls back to the
archive directory if the repo is unavailable. release-tos.yml builds and
uploads the single-commit bare repo (repack + update-server-info).
- Claude Code on TOS warns that directory marketplaces cannot auto-update.
- tos-install.sh bootstraps shrink to TOS_BASE + --dist tos.
- Docs (READMEs, agent-integrations pages, image cards, en+zh) now all use
the single shared installer and drop the deleted wrapper instructions.
* feat(installer): unify all choice prompts on an arrow-key TUI menu
Language, download source, connection mode, keep-or-reconfigure
credentials, statusline enable/replace, and legacy-mode confirmation all
render as the same single-select menu (arrow keys / digit shortcuts /
enter, radio-style highlight) instead of mixed numbered and y/N prompts.
Falls back to numbered input when /dev/tty can't be drawn on and to the
default choice when non-interactive. Free-text fields (URL, API key) stay
line inputs; the harness picker keeps its checkbox multi-select.
* fix(installer): stop piping plugin lists into grep -q under pipefail
grep -q exits on first match and SIGPIPEs the producer, so with pipefail
the 'codex plugin list | grep -q' check read as a miss every time (codex's
list is long; claude's short list masked the bug). Capture the output and
substring-match in bash instead — validation no longer false-warns.
Also: drop the stdio-proxy line from the Done summary; always offer the
install-source menu unless --dist/--source was given (with a checkout the
menu gains a dev option and defaults to it); surface the Claude-on-TOS
no-auto-update warning at source resolution instead of after install.
* fix: unignore examples/memory-plugin-shared/lib and commit the shared modules
The Python build-artifact 'lib/' gitignore rule silently swallowed the
shared plugin module source, so CI checkouts had only the vendored copies
and sync.test.mjs failed with ENOENT on the source directory.
* fix(recall): budget summary/uri fallbacks and sanitize non-finite scores
max_chars is the recall API's contract, but only full fragments counted
toward it — VikingBot's client-side heuristic, faithfully ported, lets
summary and uri fallbacks render far past the budget (repro: max_chars=100
rendered 548 chars). Every fragment now counts; oversized summaries degrade
to uri fragments and entries that can't even fit a uri line are dropped
(reported via stats.dropped). VikingBot itself is intentionally unchanged.
Also run _sanitize_floats over the /recall response like the neighboring
/find and /search routes, so inf/nan scores return 0.0 instead of a 500.
* feat(storage): optimize glob func
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* fix(localfs): offload blocking fs operations to spawn_blocking
* feat(glob): cap glob api default node_limit at 256
* feat(sdk): add node_limit options for glob in python and go SDKs
* Refactor recursive web import into HTTP accessor
Move ordinary web page import routing into HTTPAccessor and materialize crawled pages as a temporary directory via WebImporter.
Relocate Scrapy/Playwright crawling under parse.accessors.web_crawler, keep trafilatura extraction inside HTMLParser, and avoid repeated ResourceService.add_resource calls.
Add recursive crawl controls, safe request validation, page/download classification, and focused unit coverage.
* Document recursive web crawler options
* fix(web-crawler): stop SSRF sub-resource block from failing whole render
The playwright fallback validated every sub-resource request against the
SSRF guard and raised on the first disallowed host, failing the entire
page render. volcengine docs load a probe resource on an internal host,
so rendering always failed and the crawler stored the static anti-bot
"Please wait..." challenge page as content.
Now a blocked sub-resource is only aborted; the main document and final
URL still gate the result. Also wait past JS interstitials, retry reads
through in-flight navigation, and reject shell/challenge pages instead of
storing them.
* fix(web-crawler): surface renderer error hint on entry-page failure
When Playwright is unavailable, the renderer returns an actionable install
hint via RenderResult.error, but the spider silently kept the static shell
and WebImporter raised only the generic "Failed to fetch entry page". The
hint never reached the user.
Now the spider records rendered.error on the failed page, and WebImporter
appends the entry page's failure reason to the raised message so the CLI
shows the Playwright install instructions.
* fix(web-crawler): surface render hints and enforce crawl limits
* fix(web-crawler): avoid rendering SSR app pages
* perf(web-crawler): bound render concurrency and cap networkidle wait
Playwright renders were dispatched from parse callbacks without any
concurrency limit, so a page with many child links could spawn dozens of
Chromium pages at once (observed peak 28 for a 20-page crawl), risking OOM
on large sites and starting ~2.3x more renders than needed before
max_pages stopped the crawl. Gate renders with a semaphore sized to
config.concurrency and re-check the success limit after acquiring a slot
so queued callbacks skip rendering once the crawl is already done.
Also cap the networkidle wait at 8s: pages with continuous background
activity (e.g. GraphiQL) never go idle and previously blocked until the
full render timeout, turning a ~3s page into ~38s. Content is ready after
domcontentloaded and _wait_past_challenge covers late-arriving text.
Bump default concurrency 5 -> 10.
* fix(web-crawler): route .html/.htm URLs through recursive WebImporter
An explicit .html/.htm URL is detected as DOWNLOAD_HTML via the extension
map, so access() previously only routed URLType.WEBPAGE to WebImporter and
these URLs fell through to single-file download, silently ignoring
depth/max_pages. Route DOWNLOAD_HTML through WebImporter too, treating a
single-page import as the depth=0 case.
* fix(web-crawler): improve HTML extraction and rendering heuristics
- Drop trafilatura favor_precision=True: it stripped the full body of
link-dense pages, keeping only headers.
- Only render __NEXT_DATA__ pages with Playwright when their static body
is too thin; SSR/SSG Next.js pages already ship full text.
- Disable Scrapy telnet console to avoid opening port 6023.
* fix(web-crawler): keep code-hosting single-file URLs off recursive crawler
GitHub/GitLab blob and GitHub raw URLs resolve to a single file, not a
site. Route them through the single-file download path instead of the
recursive WebImporter, which otherwise crawls the hosting UI shell.
* docs(resources): add recursive web crawler usage examples
Add depth/max_pages crawl examples to the HTTP, Python SDK, and CLI
blocks in both the zh and en resource API docs, plus path-prefix
filtering and skip_download_links variants.
* fix(server): restore identity resolution in /health endpoint
The /health endpoint was refactored in #2503 which removed the identity
resolution logic. This caused the frontend dashboard to show 'Usage/Audit
未初始化' because the role field was missing from the response.
Restored the identity resolution so that /health returns account_id,
user_id, and role when an API key is provided.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(server): update health endpoint tests for identity resolution
- Test that /health returns identity info when API key is provided
- Test that /health omits identity info when no API key is provided
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: wugj <wugj@g-bits.com>
Co-authored-by: Claude <noreply@anthropic.com>
