Commit Graph
31 Commits
Author SHA1 Message Date
Hao Zhe c1d38eb47f feat(langchain): support request-scoped actor peers (#3626) 2026-08-01 22:55:33 +08:00
Qin Haojie fd42b1ad92 feat(tasks): support task cancellation (#3577)
* feat(tasks): support task cancellation

* refactor(tasks): scope cancellation to current user

* feat(cli): support task cancellation

* refactor(tasks): make cancellation queue-aware

* refactor(tasks): simplify cancellation bookkeeping

* test: remove task cancellation coverage

* refactor(tasks): trim cancellation coordination

* fix(tasks): contain cancellation to owned work

* feat(tasks): persist resource source metadata

* fix(tasks): handle cancelled work consistently

* refactor(tasks): make completion queue-aware

* fix(tasks): persist terminal state before queue ack

* test(tasks): remove added lifecycle tests

* docs(tasks): document task cancellation
2026-07-30 20:34:27 +08:00
zihengli e9c4cc97c3 refactor: extract Connector delegation and expose declarative add_type (#3591)
* refactor: delegate add_resource imports to external Connector

* refactor: delegate add_resource imports to external Connector

* refactor: extract Connector delegation and expose declarative add_type

* fix: merge main to refactor/connector_delegator

* fix: merge main to refactor/connector_delegator
2026-07-29 18:09:25 +08:00
Jiahui Zhou 34b5a88971 Feat/add resource tags (#3560)
* feat: allow tags during resource import

* feat: support uploaded resource watches with tags

* feat: add resource tag flags to CLI

* fix: reject uploaded resource watches with tags

* docs: untrack add resource tags design draft

* fix: write add_resource tags during ingest

* docs: move add_resource tags docs into resources api

* fix: tighten add_resource tag ingestion semantics

* fix: address add_resource tag review feedback

* fix: merge resource tags at vector upsert

* refactor: carry add_resource tags with ingest options
2026-07-29 15:43:06 +08:00
fujiajie666 c91b0d36f2 feat: implement Skill-driven knowledge compilation for ov compile (#3567)
* feat(compile): implement skill-driven ov compile

Require a Skill and run compile tasks through VikingBot AgentLoop with structured wiki bundle rendering and durable task state.

Add OpenViking batch-write and bot proxy APIs, Python SDK and Rust CLI support, shared link and memory helpers, tests, and a One-Page demo.

* fix(compile): refine defaults, links, and failure handling

* fix(compile): normalize skill tools and degrade gracefully

* feat(compile): support skill-defined artifact outputs

* feat(compile): improve artifact reliability and wiki navigation

* feat(compile): support generating and updating skill packages

* fix(compile): validate OKF frontmatter and catalog page types

* feat(compile): rank target catalog and validate updates lazily

* feat(compile): tag generated wiki files for search

* fix(compile): preserve generated skill artifacts in submissions

* fix(compile): enforce fixed toolset and workspace artifact submissions

* fix(skills): preserve nested metadata in skill frontmatter

* fix(compile): normalize wiki paths and citation line breaks

* docs(examples): remove outdated compile demos

* docs(api): document compile and batch-write endpoints

* fix(content): allow arbitrary resource files in batch writes

* fix(compile): harden task lifecycle, auth, and execution

* fix(compile): disable direct exec by default

* fix(compile): allow file-only tasks when exec is disabled

* fix(compile): prevent task lock leaks
2026-07-28 20:33:25 +08:00
Jiahui Zhou 5d1ba45be4 Feat/add resource processing mode (#3566)
* feat: add resource processing mode

* fix: keep semantic artifacts in vectors-only add resource

* test: support processing mode in api test client

* docs: document add resource processing mode

* fix: align processing mode after resource ingestion refactor

* feat: expose processing mode in TypeScript SDK

* fix: preserve add resource compatibility
2026-07-28 20:09:06 +08:00
Hao Zheandzhiheng.liu 797406c381 fix(sdk): consolidate model, client, and evaluation correctness (#3550)
* fix(models): route multimodal inputs to DashScope embed_content

Sweep findings: A-09. Preserve image parts through the shared embedding entrypoints.

(cherry picked from commit 50b20d7ca9)

* fix(models): constrain DashScope multimodal routing

Sweep findings: A-09. Preserve text mode and require a single fused vector for multipart input.

(cherry picked from commit 77784429d4)

* fix(models): respect DashScope Qwen fusion parameters

Sweep finding A-09

(cherry picked from commit aafc702b20)

* fix(review): preserve tongyi multipart compatibility

Addresses blocking review finding on #3400.

(cherry picked from commit 96844047c5)

* fix(eval): flush queued records on stop and adapt RAG pipeline to FindResult

Sweep findings: B-05, B-12. Drain recorder queues through the sentinel and consume current retrieval result objects.

(cherry picked from commit fecdbeb6d4)

* fix(sdk/python): support sync client inside a running event loop

Sweep findings: F-09. Run sync wrappers on one persistent worker loop with result and exception propagation.

(cherry picked from commit ba0faadc87)

* fix(sdk/python): preserve cancellation and fork safety

Sweep finding: F-09. Preserve original cancellation errors and reset worker synchronization after fork.

(cherry picked from commit a19e3c95e9)

* feat(sdk/go): add tags filter and relations API for parity

Sweep findings: F-11, F-12. Expose existing server capabilities consistently to Go callers.

(cherry picked from commit 8af0c2cb36)

* fix(sdk/python): runnable quickstarts, correct migrate payload, explicit timeout precedence

Sweep findings: F-01, F-02, F-14. Initialize documented clients and preserve Python SDK request/config semantics.

(cherry picked from commit d9a64f5665)

---------

Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
2026-07-28 18:11:30 +08:00
agent 8391d3a758 feat: add global Agent Evolution switch and HTTP usage sink (#3223)
* feat: add per-user agent evolution settings

* simplify Agent Evolution user settings

* fix: preserve agent evolution client compatibility

* feat(snapshot): add path diff API

* feat(snapshot): expose path diff in clients and CLI

* fix(agent-evolution): gate case memory production

* fix(agent-evolution): preserve configuration compatibility

* feat(usage): add built-in HTTP sink

* fix(agent-evolution): address PR review findings

* fix(usage): isolate HTTP outbox by destination

* docs(usage): define CountRecord HTTP mapping

* docs(usage): plan CountRecord HTTP implementation

* feat(usage): emit CountRecord over HTTP

* docs(agent-evolution): design global switch

* docs(agent-evolution): plan global switch migration

* feat(agent-evolution): make production switch global

* fix(agent-evolution): preserve embedded defaults

* test(agent-evolution): cover failed archive policy replay

* docs(agent-evolution): clarify embedded compatibility

* docs(agent-evolution): expose global switch in example config

* refactor(agent-evolution): align global setting terminology

* fix(agent-evolution): preserve session skill extraction

* fix(usage-reporter): capitalize count record keys
2026-07-27 20:09:38 +08:00
DuTao 0ab85f450a feat(session): add turn-aware retention and reliable archive recovery (#3380)
* 优化OpenViking的 session compact逻辑,active message 改为turn,压缩 assistant,保留完整user。
详见RFC:https://github.com/volcengine/OpenViking/discussions/3330

* Vikingbot 使用 ov turn session

* fix pr comment

* 更新文档

* fix pr issue
2026-07-24 14:26:46 +08:00
agent 27debfe7a8 feat(snapshot): add path diff API, SDK, and CLI (#3451)
* feat(snapshot): add path diff API

* feat(snapshot): expose path diff in clients and CLI

* fix(snapshot): pin path diff to resolved commits

* fix(snapshot): bound path diff resource usage

* fix(snapshot): bound diff blob reads

* fix(snapshot): harden path diff API

* fix(snapshot): bound path diff resource usage

* fix(build): sync native binding dependencies
2026-07-23 17:45:36 +08:00
Monday 49ca3cdfe8 feat(python-sdk): support event hooks and lifecycle reset (#3392)
* feat(python-sdk): support event hooks and lifecycle reset

* fix(python-sdk): limit client hooks change to event hooks

* docs(python-sdk): initialize clients in readme examples
2026-07-23 14:16:43 +08:00
ef4d97ebe3 feat(snapshot): support path-filtered commit history (#3271)
* feat(snapshot): support path-filtered commit history

* refactor(snapshot): reuse SDK git log implementation

* fix(snapshot): harden path-filtered log resource limits

* chore: remove stale SDK lock entry

---------

Co-authored-by: zhanghaoyu.la <zhanghaoyu.la@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-07-16 19:53:40 +08:00
Jiahui Zhou 1c46d44fbc Fix/reindex preserve owners (#3096)
* fix: preserve reindex content owners

feat: allow trusted admin role assertion

feat: prune orphan vectors during reindex

fix: harden reindex memory body reads

feat: expose reindex prune options in clients

fix(cli): prefer workspace sdk for compat clients

fix: harden reindex prune orphans

* test: align reindex expectations after rebase
2026-07-14 20:38:30 +08:00
Zayn Jarvis cc0281ac70 fix(studio): sort limited listings by mtime (#3212)
* fix(studio): sort limited listings by mtime

* fix(client): forward ls ordering options
2026-07-13 16:17:02 +08:00
yufeng 5bfa9b617e fix: align TypeScript SDK with server contracts (#3159)
* fix: align TypeScript SDK with server contracts

* fix: harden TypeScript SDK contracts

* fix: handle Node image references safely

* fix: make OVPack downloads atomic
2026-07-11 22:24:16 +08:00
DuTao cbfb387dc7 feat(bot): add unified VikingBot gateway routing and OpenViking auth (#3119)
* bot opt config\api check

* 优化vikingbot的启动链路

* 美化颜色

* docs: add VikingBot gateway routing diagram

* fix(bot): harden gateway auth and proxy routing
2026-07-10 17:45:41 +08:00
Qin Haojie 3003ed61d7 feat(retrieval): support image search (#3093)
Add multimodal image vectorization and image query support across the server, SDKs, and CLI.
2026-07-09 16:42:53 +08:00
baojun-zhang 882d0c16e3 Grep mem optimize (#3043)
* feat(rgafs): add rg command into dockerfile

* feat(glob): cap grep api default node_limit at 256

* feat(python-sdk): expose default node_limit forgrep, update docs
2026-07-07 11:00:11 +08:00
baojun-zhang 6a33ebb7ca Optimize glob walkdir (#3013)
* feat(storage): optimize glob func

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* fix(localfs): offload blocking fs operations to spawn_blocking

* feat(glob): cap glob api default node_limit at 256

* feat(sdk): add node_limit options for glob in python and go SDKs
2026-07-06 21:39:16 +08:00
17631bd353 feat: support ignoring certain files in the multi-version management function, similar to the git ignore feature (#2930)
* docs: add ovgitignore design spec

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add ovgitignore implementation plan

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(git): add ovgitignore matcher

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(git): apply ovgitignore during commits

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(pyagfs): expose ovgitignore commit results

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(vikingfs): manage account ovgitignore

Co-Authored-By: Claude <noreply@anthropic.com>

* docs(git): document ovgitignore semantics

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(git): enable ovgitignore restore assertion and document python api

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: fix gitignore

* fix: fix gitignore

* feat: 新增 git ignore 相关的接口

* fix: 修复 cli 命令渲染

* doc: 删除执行计划文档

* doc: 删除执行计划文档

* feat: 在 http client 中新增 git ignore 相关接口

* fix: 修改文档中关于 .ovgitignore 被版本化的内容

---------

Co-authored-by: zhanghaoyu.la <zhanghaoyu.la@bytedance.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-02 11:45:38 +08:00
Qin Haojie 8186cfbdeb feat(auth): support seeded API key generation (#2932)
* feat(auth): support seeded API key generation

Allow admin key issuance flows to accept an optional seed so clients can derive predictable user API keys when needed, while preserving random generation by default.

* fix(go-sdk): preserve explicit empty seed payloads

Use pointer seed options so callers can distinguish omitted seeds from explicit empty seeds, matching the Admin API behavior.
2026-07-02 10:47:18 +08:00
Qin Haojie 6223537a00 feat(fs): 新增 attrs 元信息接口 (#2906)
* feat(fs): add attrs metadata endpoint

* feat(fs): move set tags under attrs API
2026-07-01 11:57:04 +08:00
Qin Haojie d7b96d7715 feat(server): add user add target defaults (#2888)
* feat: add user add target defaults

Allow deployments and per-user settings to provide default add targets while keeping explicit request targets authoritative.

* test: remove low-value CLI config parsing test

* refactor: unify user config option naming
2026-06-29 20:32:00 +08:00
Qin Haojie f9c1a60658 fix: narrow legacy agent id compatibility (#2839)
Treat agent_id as an actor peer alias in new clients while keeping X-OpenViking-Agent handling only on server ingress for old clients.
2026-06-26 14:37:57 +08:00
t0saki 3d456ff442 fix(sdk): omit null/empty request fields so older instances accept them (#2834)
The Python SDK's find/search/add_resource unconditionally attach optional
fields to their request bodies — `args` as an empty `{}` and
`tags`/`score_threshold`/`filter`/`context_type`/`session_id` as `null` — even
when the caller never set them. OpenViking kernels use
`model_config = ConfigDict(extra="forbid")` on these routes, so any instance
that predates a field rejects the whole request with
`body.<field>: Extra inputs are not permitted`. This breaks the SDK (and every
SDK consumer, e.g. vikingbot via `ov.AsyncHTTPClient`) against older instances:
`add_resource` fails on `body.args` against a pre-#2549 instance, and `find`
fails on `body.tags` against a strict pre-#2706 instance.

This is the SDK-side counterpart of #2799, which fixed the same anti-pattern in
the Rust CLI but left the Python SDK untouched.

- Add `_compact_request_body()` and apply it in find/search/add_resource: drop
  null-valued keys and an empty `args` object before sending. Scoped to these
  read/create routes only, where a missing optional field and an explicit null
  are equivalent — not applied globally, since null may mean "clear" on a future
  update/PATCH route. Mirrors the CLI's `compact_request_body`.

Tests: unit tests for `_compact_request_body` and for find/search/add_resource
omitting unset optional fields while keeping explicitly-provided ones.
2026-06-25 21:26:31 +08:00
Qin Haojie cf70a95443 fix(cli): remove unrelated HTTP compat shim (#2826)
Keep the grep/BM25 PR scoped by dropping legacy HTTP compatibility behavior and unrelated SDK config typo guidance.
2026-06-25 17:26:36 +08:00
0102a48c2a fix(skills): now we allow viking://agent/skills again, and optimize CLI for skills (#2813)
* chore: clear unused files

* fix(tests): fix unit test

* refactor(auth): introduce plugin-based authentication architecture

Replace the monolithic `openviking/server/auth.py` with an extensible
plugin-based auth system. This refactor extracts the three built-in modes
(`dev`, `api_key`, `trusted`) into separate `AuthPlugin` implementations,
adds a registry for third-party plugins, and preserves all existing behavior
while enabling custom authentication backends (e.g. LDAP, OIDC, mTLS).

Key changes:
- **New public API**: `AuthPlugin` (ABC) and `register_auth_plugin` decorator.
- **New registry**: `AuthPluginRegistry` supports runtime registration.
- **Built-in plugins**: `DevAuthPlugin`, `ApiKeyAuthPlugin`, `TrustedAuthPlugin`.
- **Config change**: `auth_mode` widened from `Literal` to `str` for custom modes.
- **Validation delegated**: `validate_server_config()` now delegates to the active
  plugin's `validate_config()`, preserving existing validation semantics.
- **Router compatibility**: All existing `require_*` decorators and `resolve_identity`
  / `get_request_context` dependencies remain unchanged. Routers import the same
  symbols from `openviking.server.auth`.
- **Tests**: `conftest.py` manually wires the DevAuthPlugin in ASGI tests (lifespan
  not triggered). `test_auth.py` expanded with plugin registration and validation tests.
- **Docs**: `04-authentication.md` (en/zh) updated with plugin registration examples.

Co-Authored-By: claude-sonnet-4-6 <noreply@anthropic.com>

* fix(tests): fix trusted mode test

* fix(tests): fix unit test

* fix(cli): remove unexisted transaction observer

* docs: update skills definition

* docs: update skills definition

* docs: update skills definition

* docs: update skills definition

* fix(skills): now we allow viking://agent/skills again, and optimize CLI for skills

* docs(skills): use -p instead of --parent in agent skills examples

Align the `ov skills add` examples in the context-types and viking-uri
docs with the short flag `-p` introduced for `ov skills list/find/show`,
so all four user-facing examples consistently demonstrate the short form
when targeting `viking://agent/skills`.

Co-Authored-By: claude-sonnet-4-6 <noreply@anthropic.com>

* fix(tests): error check for api key

* fix(tests): unit test wait until resource not busy

* fix(tests): unit test wait until resource not busy

* fix(sdk): args form in skills find

* fix(skills): pass target uri in request body

---------

Co-authored-by: claude-sonnet-4-6 <noreply@anthropic.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-06-25 14:36:06 +08:00
Haoyu Zhangandzhanghaoyu.la aa53e7aede feat: 实现 commit、restore、show 文件系统多版本管理功能 (#2756)
* feat: 实现 commit、restore、show 文件系统多版本管理功能

fix: 修复commit时删除文件

fix: commit 的 fast path 1 添加 Racy-clean 机制

fix: 将 sdk 中的 git 命令改为 snapshot 命令,同步修改单测

fix: 多版本管理的文件存储目录改为 .ovgit

feat: snapshot cli 渲染

fix: 修复 restore 时将删除的文件回滚时,目录不存在的问题

feat: restore 命令的 project_dir 参数改为可选,不传时默认全目录回滚

feat: 更新文档

fix: 删除暂未使用的配置参数

feat: 新增示例脚本

fix: 修复示例代码

fix: 修复 restore 返回的 task id 任务完成状态

feat: 在 restore 修改文件系统时加锁

fix: fix openviking_sdk

* feat: 将git多版本管理功能改为默认打开,并复用agfs的配置参数作为默认值

* fix: restore 命令改为先完成 ref 一致性协议再写回 VFS;object store 并发改为使用唯一 temp path

* fix: 在 git 配置检验层去除未实现的cas_mode = "redis_lock"模式

* fix: 在 Rust GitService 边界统一校验 account

* fix: 当前commit不支持通过 path 传入目录,增加报错信息

* fix: 将git文件默认存储路径统一为 .ovgit

* fix: restore 时写入 VFS 失败时返回详细的报错,并继续触发 reindex

* fix: 校验 commit、restore、show 的路径

* feat: 实现 commit 时指定目录

---------

Co-authored-by: zhanghaoyu.la <zhanghaoyu.la@bytedance.com>
2026-06-25 11:28:04 +08:00
87329714dd feat(grep): integrate VikingDB bm25 keyword search for grep engine (#2144)
* feat(grep): integrate VikingDB bm25 keyword search for grep engine

* fix(grep): address CI review feedback: max-size eviction to _count_cache, use Literal, Split regex alternation into individual keywords for bm25 (max 10)

* fix(schema): use dynamic __version__ for schema_version and handle dev suffixes in version comparison

* fix(schema): upsert data to vikingdb lack of content

* chore: add benchmark for retrieval

* fix(grep): vikingdb return 200 and no results means no matching content, not necessary to fallback to local fs

* fix(benchmark): sub uri args; add report

* refactor: code format by ruff

* optimize: move grep config (engine and switch_to_remote_threshold) to ov.conf

* optimize: auto adapt remote_return_limit by agg API; rm unnecessary params in keywords search

* fix: adjust benchmark scripts

* fix(grep): store full content for BM25; use PathScope depth; reduce redundant API calls

* refactor: new benchmark

* fix: step1 add resource by real code data

* feat(benchmark): split grep benchmark into effectiveness/performance suites with async reindex

* optimize (benchmark): adjust keywords and ground truth for testing

* fix: truncate 64KB for content field

* optimize: effectiveness add resource plainly

* optimize: change param use of SearchByKeywords from "keywords" to "query"

* optimize(benchmark): refactor effectiveness scripts

* optimize: ensure raw data for content field

* optimize: fulltext analyzer's stop-words only use symbols

* fix: adapt to new ov cli for benchmark

* optimize: reuse file content to avoid re-read AGFS file

* optimize: tune grep vikingdb defaults and refresh bm25 benchmark scripts

* optimize: benchmark client timeout

* update README

* fix: rm unused param

* fix: default values in docs

* optimize: increase truncate byte size to 1MB for content field for VikingDB

* fix(logger): harden queued stream logging (#2786)

* fix(logger): replace StreamHandler with QueueHandler+QueueListener to prevent thread deadlock

When log.output='stdout' (default) and the server is managed by systemd,
concurrent log writes can deadlock because logging.StreamHandler holds a
thread lock across stream.flush() which blocks on systemd-piped file I/O.

During session.commit() phase 2, multiple async coroutines (memory
extraction, summarization) concurrently call logger.info()/warning()
with large payloads. The first thread's flush() blocks on the pipe,
while all subsequent threads block on handler.acquire() forever.
This permanently silences the server log and prevents _write_done_file()
from executing, leaving phase 2 hanging without .done.

Fix: use QueueHandler + QueueListener from stdlib logging.handlers
(Python 3.2+). QueueHandler.emit() does queue.put(record) with no lock
or I/O, returning immediately. QueueListener has a dedicated single
thread as the sole consumer touching the real StreamHandler, making
lock contention impossible.

Changes in _create_log_handler(): stdout/stderr branches now create
a shared QueueListener with unbounded queue, returning QueueHandler
instances to callers. _build_standard_handler() delegates formatter
and filter setup to the real handler in the listener thread.

Closes: #2752

* fix(logger): harden queued stream logging

---------

Co-authored-by: njuboy11 <njuboy11@users.noreply.github.com>

---------

Co-authored-by: Qin Haojie <qinhaojie.exe@bytedance.com>
Co-authored-by: njuboy11 <njuboy11@users.noreply.github.com>
2026-06-24 18:46:02 +08:00
Jiahui Zhou 5a433e5a75 docs: add release guide and align SDK tag format (#2765)
* docs: add release guide and align SDK tag format

* fix: restrict main package release tag discovery

* fix: constrain CLI version tag discovery
2026-06-22 17:15:31 +08:00
Jiahui Zhou 02f06488f3 feat: extract standalone python http sdk (#2736)
* feat: extract standalone python http sdk

* fix(python-sdk): restore ovcli.conf compatibility

fix(python-sdk): restore compatibility and lazy-load sdk

fix(python-sdk): restore legacy http client compatibility

docs(python-sdk): add chinese readme
2026-06-22 14:03:05 +08:00