* feat(cli): download directories as zip archives
* fix(download): cap directory archives
* fix(download): bound directory archives while they are built
The archive size cap was only enforced by `os.path.getsize()` after the
whole ZIP had been written, and `actual_total` counts file payload bytes
only. A tree made of empty directories or empty files therefore adds
per-entry ZIP headers that no check sees until the temp file is already
complete: with the limit set to 1 KiB, a 20k-entry tree writes 1.9 MB to
disk before being rejected.
Check the live write offset after every member so the temp archive stays
within the limit as it grows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CneCiyRjLaeDRYSWKngcJ
* docs(download): make the KG snippet re-runnable and sync the API catalog
The new knowledge-graph snippet extracts with a plain `unzip`, but the
note above it only tells the reader to delete the archive. Re-running it
leaves the previously extracted `./journal-kg/` in place, so `unzip`
stops at an overwrite prompt — and in a non-interactive shell it exits 1
without extracting anything. Use `unzip -o` and say what the note
actually has to cover.
Also update the endpoint catalog in api/01-overview.md, which still
described /content/download as file-bytes only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CneCiyRjLaeDRYSWKngcJ
* fix(download): build directory archives in memory, not in a temp file
The temp archive is handed to FileResponse with a BackgroundTask that
unlinks it, but starlette runs `background` only after a successful
send. Both Range-header error branches (starlette/responses.py:370,373)
`return await PlainTextResponse(...)` before reaching it, so a malformed
or unsatisfiable Range leaks the archive permanently — 22 such requests
leak 22 files in a local repro, up to 10 MiB each, with nothing to
reclaim them. asyncio.CancelledError misses the `except Exception`
cleanup for the same reason.
Since the archive is capped at 10 MiB anyway, build it in a BytesIO and
return it as a plain Response, exactly like the single-file branch. That
drops the temp file, the cleanup callback, and the tempfile/os/
FileResponse/BackgroundTask imports, and gives both branches the same
`Content-Disposition: attachment; filename*=UTF-8''...` form instead of
two different ones.
Directory downloads no longer honour Range. They never usefully did:
the archive is rebuilt per request and zipfile stamps time.localtime()
into every member, so resuming a range spliced two different archives.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CneCiyRjLaeDRYSWKngcJ
* fix(download): return 413 for oversized directory archives
RESOURCE_EXHAUSTED maps to 429, which tells clients the request is
rate-limited and worth retrying after a backoff. An archive over the
10 MiB cap fails because of the directory's own size, so every retry
re-walks the tree and re-zips it before failing again.
Add PAYLOAD_TOO_LARGE / 413 and raise it from the archive size check.
The code is plumbed through both status<->code maps (server app and
utils), the client's code->exception table, and the Rust CLI's status
mapping, so an over-cap `ov get` still surfaces a typed error rather
than falling through to INTERNAL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CneCiyRjLaeDRYSWKngcJ
* feat(cli): write directory downloads as a named .zip in a target directory
`ov get viking://resources/myfolder ./myfolder` wrote the ZIP bytes to a
path named `myfolder` with no suffix: a regular file wearing a folder's
name, which `cd` rejects and `file` reports as ZIP data. The local path
was always used verbatim, so only the docs' hard-coded `./project.zip`
form produced a sane result.
Treat a target that is an existing directory — or omitted, meaning the
current directory — as the destination *directory*, and name the file
after the resource, appending `.zip` when the response came back as
`application/zip`. An explicit non-directory path is still used
verbatim, so `ov get <uri> ./explicit.zip` is unchanged. Nothing is
extracted; the archive is what lands.
get_bytes_with_type exposes the response Content-Type, which is how the
caller tells a raw file apart from a directory served as a ZIP;
get_bytes keeps its old signature for the TUI and its existing test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013CneCiyRjLaeDRYSWKngcJ
* fix(download): bound archive entries and preflight targets
* fix(cli): preflight existing symlink targets
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(memory): support event tag filtering
Add session-level default event tags, commit-time overrides, durable queue propagation, and first-write vector index tagging. Include config update APIs and coverage for serialization, concurrency, extraction, and HTTP behavior.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* feat(memory): expose event tags in SDKs and CLI
Add session default tag configuration, config updates, and commit-time event tag overrides across embedded Python, standalone Python, TypeScript, Go, and the Rust CLI. Preserve explicit empty-tag semantics and document each public interface.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(sdk): align legacy session tag APIs
Forward commit-time event tags through the legacy Python HTTP shims and align BaseClient session signatures without adding a new abstract-method requirement for existing subclasses.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* feat(session): allow updating auto-commit policy
Extend PATCH session config to atomically update event tags and auto-commit settings. Merge policy objects by field, use explicit null to disable automatic commits, preserve omitted fields, and expose the contract across SDKs and CLI.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(session): align session config interfaces
Replace the generic session create config JSON flag with explicit event-tag and auto-commit options. Preserve omitted, object, and null auto-commit semantics across HTTP, embedded clients, SDKs, and CLI, reject ambiguous null policy fields, and handle nullable event configuration consistently.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* test(session): trim redundant event tag tests
---------
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
* chore: clear unused files
* fix(tests): fix unit test
* refactor(auth): introduce plugin-based authentication architecture
Replace the monolithic `openviking/server/auth.py` with an extensible
plugin-based auth system. This refactor extracts the three built-in modes
(`dev`, `api_key`, `trusted`) into separate `AuthPlugin` implementations,
adds a registry for third-party plugins, and preserves all existing behavior
while enabling custom authentication backends (e.g. LDAP, OIDC, mTLS).
Key changes:
- **New public API**: `AuthPlugin` (ABC) and `register_auth_plugin` decorator.
- **New registry**: `AuthPluginRegistry` supports runtime registration.
- **Built-in plugins**: `DevAuthPlugin`, `ApiKeyAuthPlugin`, `TrustedAuthPlugin`.
- **Config change**: `auth_mode` widened from `Literal` to `str` for custom modes.
- **Validation delegated**: `validate_server_config()` now delegates to the active
plugin's `validate_config()`, preserving existing validation semantics.
- **Router compatibility**: All existing `require_*` decorators and `resolve_identity`
/ `get_request_context` dependencies remain unchanged. Routers import the same
symbols from `openviking.server.auth`.
- **Tests**: `conftest.py` manually wires the DevAuthPlugin in ASGI tests (lifespan
not triggered). `test_auth.py` expanded with plugin registration and validation tests.
- **Docs**: `04-authentication.md` (en/zh) updated with plugin registration examples.
Co-Authored-By: claude-sonnet-4-6 <noreply@anthropic.com>
* fix(tests): fix trusted mode test
* fix(tests): fix unit test
* fix(cli): remove unexisted transaction observer
* docs: update skills definition
* docs: update skills definition
* docs: update skills definition
* docs: update skills definition
* fix(skills): now we allow viking://agent/skills again, and optimize CLI for skills
* docs(skills): use -p instead of --parent in agent skills examples
Align the `ov skills add` examples in the context-types and viking-uri
docs with the short flag `-p` introduced for `ov skills list/find/show`,
so all four user-facing examples consistently demonstrate the short form
when targeting `viking://agent/skills`.
Co-Authored-By: claude-sonnet-4-6 <noreply@anthropic.com>
* fix(tests): error check for api key
* fix(tests): unit test wait until resource not busy
* fix(tests): unit test wait until resource not busy
* fix(sdk): args form in skills find
* fix(skills): pass target uri in request body
---------
Co-authored-by: claude-sonnet-4-6 <noreply@anthropic.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
* feat(grep): integrate VikingDB bm25 keyword search for grep engine
* fix(grep): address CI review feedback: max-size eviction to _count_cache, use Literal, Split regex alternation into individual keywords for bm25 (max 10)
* fix(schema): use dynamic __version__ for schema_version and handle dev suffixes in version comparison
* fix(schema): upsert data to vikingdb lack of content
* chore: add benchmark for retrieval
* fix(grep): vikingdb return 200 and no results means no matching content, not necessary to fallback to local fs
* fix(benchmark): sub uri args; add report
* refactor: code format by ruff
* optimize: move grep config (engine and switch_to_remote_threshold) to ov.conf
* optimize: auto adapt remote_return_limit by agg API; rm unnecessary params in keywords search
* fix: adjust benchmark scripts
* fix(grep): store full content for BM25; use PathScope depth; reduce redundant API calls
* refactor: new benchmark
* fix: step1 add resource by real code data
* feat(benchmark): split grep benchmark into effectiveness/performance suites with async reindex
* optimize (benchmark): adjust keywords and ground truth for testing
* fix: truncate 64KB for content field
* optimize: effectiveness add resource plainly
* optimize: change param use of SearchByKeywords from "keywords" to "query"
* optimize(benchmark): refactor effectiveness scripts
* optimize: ensure raw data for content field
* optimize: fulltext analyzer's stop-words only use symbols
* fix: adapt to new ov cli for benchmark
* optimize: reuse file content to avoid re-read AGFS file
* optimize: tune grep vikingdb defaults and refresh bm25 benchmark scripts
* optimize: benchmark client timeout
* update README
* fix: rm unused param
* fix: default values in docs
* optimize: increase truncate byte size to 1MB for content field for VikingDB
* fix(logger): harden queued stream logging (#2786)
* fix(logger): replace StreamHandler with QueueHandler+QueueListener to prevent thread deadlock
When log.output='stdout' (default) and the server is managed by systemd,
concurrent log writes can deadlock because logging.StreamHandler holds a
thread lock across stream.flush() which blocks on systemd-piped file I/O.
During session.commit() phase 2, multiple async coroutines (memory
extraction, summarization) concurrently call logger.info()/warning()
with large payloads. The first thread's flush() blocks on the pipe,
while all subsequent threads block on handler.acquire() forever.
This permanently silences the server log and prevents _write_done_file()
from executing, leaving phase 2 hanging without .done.
Fix: use QueueHandler + QueueListener from stdlib logging.handlers
(Python 3.2+). QueueHandler.emit() does queue.put(record) with no lock
or I/O, returning immediately. QueueListener has a dedicated single
thread as the sole consumer touching the real StreamHandler, making
lock contention impossible.
Changes in _create_log_handler(): stdout/stderr branches now create
a shared QueueListener with unbounded queue, returning QueueHandler
instances to callers. _build_standard_handler() delegates formatter
and filter setup to the real handler in the listener thread.
Closes: #2752
* fix(logger): harden queued stream logging
---------
Co-authored-by: njuboy11 <njuboy11@users.noreply.github.com>
---------
Co-authored-by: Qin Haojie <qinhaojie.exe@bytedance.com>
Co-authored-by: njuboy11 <njuboy11@users.noreply.github.com>
* fix: stabilize studio identity and streaming chat
* fix: hide unsupported studio terminal commands
* fix: remove unsupported terminal command copy
* fix: run selected terminal suggestion on enter
* fix: group supported terminal commands
* fix: add terminal quick start and history
* fix: scope session visibility by user
* fix: harden bot user scoping
* fix: forward request scoped bot identity
* fix: add terminal quick start translations
* fix: add terminal command group translations
* fix: simplify studio identity scoping
* fix: support api key copy on dev urls
* fix: stop passing agent id to ov http client
* fix: search follow-up memory questions
Previously, adding messages required one HTTP request per message,
making bulk operations (e.g. memory extraction, history migration)
very slow due to network round-trip overhead.
Changes:
- Add POST /api/v1/sessions/{id}/messages/batch endpoint
- Add BatchAddMessageRequest model with max_length=500 limit
- Extract _resolve_message_parts() helper to deduplicate part resolution
- Add _defer_meta_save parameter to Session.add_message() for batch optimization
- Add batch_add_messages method to Python SDK clients (base/http/sync)
- Add batch_add_messages to Session wrapper class
- Update LangChain integration to use batch API
- Update Rust CLI add_memory to use batch API
* feat(fs): add count API for directory entry counting
Adds a dedicated `count` endpoint that returns the exact number of files
and sub-directories under a directory by traversing the filesystem,
distinct from `stat`'s vector-index-based estimate. Wired through
VikingFS, FSService, HTTP router and sync/async/local SDK clients.
* feat(cli): add `ov count` command for directory entry counting
Wires the new fs.count HTTP endpoint into the Rust CLI. Adds
`-r/--recursive` and `-a/--all` flags. Documentation updated with
CLI usage examples.
* fix
---------
Co-authored-by: dingben.db@bytedance.com <dingben.db@bytedance.com@bytedance.com>
* feat(ovpack): add v2 manifest and conflict policy
Add a portable OVPack manifest for scalar metadata and make imports validate scope, derived files, and conflicts before writing.
* fix(ovpack): remove import vectorize option
Make OVPack imports always rebuild vectors in the target environment, keep legacy packages compatible, and reject unsupported manifest versions before writing.
* fix(ovpack): remove force import alias
Use on_conflict as the single OVPack import conflict policy and reject removed force inputs.
* fix(ovpack): regenerate runtime vector metadata
Keep type portable but stop exporting or applying created_at, updated_at, and active_count from OVPack manifests.
* fix(ovpack): validate manifest contents
* fix(ovpack): require manifests for imports
* fix(ovpack): close manifest validation gaps
* fix(ovpack): defer parent creation until validation passes
* fix(ovpack): remove export size guard
* fix(ovpack): support session and scope-root restores
* docs(ovpack): document full backup migration
* feat(ovpack): add backup restore workflow
* fix(ovpack): validate import scope compatibility
* feat(server): add operation telemetry for session create/add_message/commit APIs
Wrap session.create, session.add_message and session.commit HTTP handlers with
run_operation so callers can opt in via TelemetryRequest and receive a
telemetry summary in the response. Propagate the telemetry parameter through
the async/sync HTTP clients, the local client and the public SDK so all
client modes expose a consistent surface.
* refactor(client/local): move part imports into _add_message_impl where they are used
Extend the ``target_uri`` parameter from ``str`` to ``Union[str, List[str]]``
across the full find/search stack so callers can scope a single query to
multiple directories in one request:
- server: FindRequest / SearchRequest accept list[str] target_uri
- service: SearchService.find / .search forward list[str]
- storage: VikingFS.find normalizes list[str], canonicalizes each entry,
and forwards the full list as target_directories to the retriever
(matching the existing behaviour of VikingFS.search)
- clients: BaseClient, LocalClient, AsyncHTTPClient, SyncHTTPClient,
AsyncOpenViking and SyncOpenViking signatures updated; the HTTP client
gains a ``_normalize_target_uri`` helper that applies
``VikingURI.normalize`` to each non-empty entry
Single-string behaviour is fully preserved: a plain ``str`` is normalized
internally to a one-element list, and empty ``""`` keeps today's
no-target semantics.
zgy