Commit Graph
1920 Commits
Author SHA1 Message Date
Jiahui Zhou 4295dfdef0 docs(cli): update ov cli readme (#3388) 2026-07-22 00:01:59 +10:00
Yuanqing ZHAOandYuanqing Zhao 2d84a6f13c perf(cuvs): improve recovery scalability and filtered-search efficiency (#3311)
* perf(index): avoid materializing descendant path strings

* perf(cuvs): compact host vector shadow

* perf(storage): prune redundant tenant path scopes

* fix(cuvs): synchronize worker-thread searches

* perf(cuvs): stream dense shadow recovery

* test(storage): cover cross-user path scopes

* perf(storage): page candidate recovery scans

* perf(cuvs): accelerate adaptive filter routing

* perf(cuvs): reduce filtered-search host overhead

---------

Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
2026-07-21 15:59:58 +08:00
Jiahui Zhou 5390bd5e54 fix: require all retrieval tags in search (#3097) 2026-07-21 11:54:01 +08:00
yufeng 085aaf832d docs: correct context paths and refine navigation (#3385) 2026-07-21 11:45:17 +08:00
ShaoZegangByte 39dc01e2a5 feat(resource): support Feishu/Lark URL imports via UnderstandingAPI (#3320)
* feat(resource):add lark understand api

* fix(resource):add lark understand api env

* fix(resource):add lark understand api

* fix(resource):understand api pr

* fix(resource):add test

* fix(resource):handle deferred URIs for async Feishu imports

* fix(resource):fix Ruff issues in lark import

* fix: persist final resource URI for async UnderstandingAPI tasks

* fix: clean up cancelled async UnderstandingAPI scheduling

* fix: getattr defer_target_resolution
2026-07-21 11:08:13 +08:00
Yuanqing ZHAOandYuanqing Zhao e1cea0998b perf(vectordb): avoid hydrating unrequested vectors (#3375)
Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
2026-07-20 19:13:00 +08:00
Evo dd31e3a05a docs(resources): clarify uploaded snapshot watch limits (#3363) 2026-07-20 19:10:49 +08:00
Qin Haojie 370fe45f6f fix(auth): serialize API key registry writes (#3377)
Prevent concurrent account and user registry updates from racing in AGFS.
2026-07-20 17:51:07 +08:00
huangruitengandhuangruiteng f093fafd1b fix(feishu): preserve title prefixes in resource names (#3366)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-20 17:16:49 +08:00
Hao Zhe 3b2571b36e feat(openclaw): add catalog icon (#3362) 2026-07-20 14:55:30 +08:00
Qin Haojie 1d4bf132ed fix(task): 修复 add-resource 任务重启后无法恢复 (#3334)
* fix(task): recover add-resource jobs after restart

Persist asynchronous add-resource work in QueueFS so interrupted jobs can resume instead of leaving tasks running forever.

* fix(queue): omit parser args from prepared jobs

* fix(queue): fail when semantic source is missing
2026-07-20 14:07:27 +08:00
Yu Zhang 295ce577b1 fix(memory): isolate unresolved URI operations (#3299)
* fix(memory): isolate unresolved URI operations

* fix(memory): preserve deletes when URI resolution fails
2026-07-20 10:31:41 +08:00
huangruitengandhuangruiteng 810a22d881 docs: isolate OpenViking installs for Hermes (#3365)
* docs: isolate OpenViking installs for Hermes

* docs: keep install guidance scoped to Hermes

---------

Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-20 08:21:03 +08:00
黄云龙 c8fd73aaca fix(session): return in-flight archive messages in get_session_context (#3129) (#3149)
* fix(session): return in-flight archive messages in get_session_context (#3129)

Seed latest_completed_index from 0 instead of commit_count so that
archives whose Phase 1 has completed (messages written, commit_count
advanced) but whose Phase 2 is still running (.done not yet written)
are treated as pending rather than already completed.

The release/0.3.x implementation seeded from 0 and did not have
this bug; the regression was introduced when commit_count was
adopted as the seed value.

* test(session): deterministic regression for pending archive context (#3129)

Replace the monkey-patched commit_async test with a direct
filesystem-state test that sets up the post-Phase-1 archive
(messages.jsonl present, commit_count advanced, no .done marker)
and asserts get_session_context still surfaces the archived
messages.

This is deterministic regardless of the queue-worker architecture
because it creates the archive state directly via the mock AGFS
and loads a fresh session from that state, never calling
commit_async or touching the session compressor.
2026-07-20 08:18:25 +08:00
kujomyandkujomy 5ca69cb5d8 fix(resource): reject watch_interval on uploaded one-shot sources instead of silently watching a frozen snapshot (#3324)
Co-authored-by: kujomy <kujomy@users.noreply.github.com>
2026-07-19 21:37:45 +08:00
Kchenandchenpengfei 5f945fa8e6 fix(path_lock): (#3353)
# 并发精确路径锁下父目录幂等创建设计

## 问题

两个并发写请求分别写入不同文件时,可能需要在同一个父目录中创建精确路径锁文件。如果该父目录尚不存在,两个加锁流程都可能先观察到目录不存在,随后递归创建同一个目录。其中一个 `mkdir` 成功,另一个收到 `EEXIST`/already-exists;此时所需目录其实已经存在。

`PathLockEngine._ensure_directory_exists_async` 当前将所有 `mkdir` 异常都视为失败。因此,竞争失败一方的加锁结果为 `False`,内容写入链路最终向调用方返回 HTTP 409 `resource is busy`。

## 预期行为

并发场景下,目录创建应具有幂等语义:

- 如果 `mkdir` 抛出异常,但重新执行 `stat` 后确认目标已经是目录,则目录准备成功。
- 如果目标仍不存在、无法查询,或者目标是非目录条目,则保持原有失败行为。
- 本次修改只影响路径锁所需父目录的准备流程,不改变锁冲突、等待超时和 HTTP 错误语义。

## 实现方案

修改 `openviking/storage/transaction/path_lock.py` 中的 `PathLockEngine._ensure_directory_exists_async`:

1. 保留现有的首次 `stat` 和递归创建父目录流程。
2. 调用 `mkdir(path)` 创建当前目录。
3. 如果 `mkdir` 抛出异常,立即通过 `_is_existing_directory_async(path)` 重新查询目录状态。
4. 如果重新查询确认目标是目录,则返回成功。这样既能处理明确的 `EEXIST`,也能兼容存储后端对该错误的包装;只有在所需文件系统状态已经成立时才忽略异常。
5. 如果重新查询未确认目标是目录,则记录原始 `mkdir` 异常并返回失败。

不增加通用重试循环。如果竞争方在该路径创建的是文件而不是目录,不能将其视为成功。

## 回归测试

在 `tests/transaction/test_exact_path_lock.py` 中增加一个可稳定复现竞态的异步测试:

- 初始文件系统中存在 `/local/default/resources`,但不存在其下的共享子目录。
- 并发为共享子目录下两个不同文件获取精确路径锁,例如 `shared/a.md` 和 `shared/b.md`。
- 通过测试 AGFS 的同步点,保证两个加锁流程首次对 `shared` 执行 `stat` 时都看到目录不存在,然后才允许任一方执行 `mkdir(shared)`。
- 允许一个 `mkdir(shared)` 创建目录,另一个抛出 already-exists 异常。
- 断言两个精确路径锁都获取成功,并且创建了两个不同的锁文件。

该测试在现有实现上必须失败:竞争失败一方的 `mkdir` 异常会进入 `_ensure_directory_exists_async` 的失败分支。

## 非目标

- 不修改调用方 `upsertTextWithConflictFallback` 的行为。
- 不调整 `resource is busy` 的可重试分类。
- 不修改精确路径锁与目录树锁之间的冲突规则。
- 不为其他 AGFS 异常增加通用重试。

Co-authored-by: chenpengfei <chenpengfei@bytedance.com>
2026-07-19 19:17:11 +08:00
huangruitengandhuangruiteng 379c19f66e fix(codex): preserve recall on Windows spawn failure (#3308)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-19 13:35:41 +08:00
huangruitengandhuangruiteng d16a92c63f fix(memory): restore vectorization after redo recovery (#3331)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-18 00:25:30 +08:00
DuTao 55a9d12cd6 1. 优化评测参数化; (#3332)
2. 优化评测显示;
3. 修复gpt-5.6 api返回 无 choices时bot兼容问题。
2026-07-17 17:46:57 +08:00
Hao Zhe ebe7fa2a7d fix(plugins): align OpenCode and OpenClaw contracts (#3232) 2026-07-17 17:06:27 +08:00
chenjw 8fd8fd1b0b fix(lock): extend default lock expiration to 30 minutes (#3321) 2026-07-17 13:59:55 +08:00
huangruitengandhuangruiteng f0d241e4a0 fix(bot): enable logs for config-started gateway (#3319)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-17 10:56:16 +08:00
ef4d97ebe3 feat(snapshot): support path-filtered commit history (#3271)
* feat(snapshot): support path-filtered commit history

* refactor(snapshot): reuse SDK git log implementation

* fix(snapshot): harden path-filtered log resource limits

* chore: remove stale SDK lock entry

---------

Co-authored-by: zhanghaoyu.la <zhanghaoyu.la@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
v0.4.10
2026-07-16 19:53:40 +08:00
Yuanqing ZHAOandYuanqing Zhao fa19ac0a75 perf(vectordb): coalesce auto cuVS rebuilds during bulk ingest (#3277)
* perf(vectordb): coalesce auto cuVS rebuilds during bulk ingest

Add an opt-in bulk-ingest maintenance scope that coalesces Auto cuVS background rebuilds across multiple write batches.

- defer derived GPU maintenance until the outermost bulk scope exits while keeping native writes and persistence visible per call
- harden the background worker against debounce, generation, shutdown, and stale-candidate races
- preserve suspension across index replacement and retire replaced workers
- wait for the final Auto GPU snapshot before vectordb_perf records search QPS
- document that the scope is non-transactional and only schedules readiness on exit

Auto cuVS and background rebuild remain disabled by default. Native CPU and remote backends use no-op hooks, so their existing behavior and dtype are unchanged.

* fix(vectordb): reject stale index replacements

* fix(vectordb): harden bulk rebuild lifecycle

---------

Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
2026-07-16 18:59:16 +08:00
Qin Haojie 4723f055bd fix(memory): restore event content rendering (#3300)
Restore the ExtractContext helpers required by the events template so extracted event memories retain their rendered body.
2026-07-16 18:04:44 +08:00
MaojiaShengandqin-ctx 3c70f8d370 feat(parse): add large image processing for image parser (#3265)
* feat(parse): add large image processing for image parser

- Add large_image_processor.py: detect large images (>10MB or >4096px),
  create low-res previews, split into grid tiles, and generate grid
  overlay images with tile labels
- Refactor ImageParser.parse() to integrate large image processing pipeline
- Enable SVG-to-PNG conversion in utils.py (cairosvg/wand)
- Rename ImageConfig.max_dimension to preview_max_dimension and add new
  config fields: max_file_size_mb, max_tile_size_mb, max_tile_dimension_px,
  tile_overlap_px, large_image_threshold_dimension
- Update ov.conf.example with new image config options

* fix(parse): correct tile dimension comment from 1024px to 2048px

* fix(parse): fix tile label path in grid overlay to include tiles/ directory

* fix(parse): register missing image extensions for ImageParser

TIFF, ICO, DIB, ICNS, SGI, JP2 were not in IMAGE_EXTENSIONS, causing
them to fallback to TextParser. All are supported by PIL.

* fix(parse): preserve PNG format for tiles instead of always converting to JPEG

* fix(parse): address review feedback for large image processing

- Wire config.image to ImageParser in ParserRegistry (was missing)
- Remove unnecessary preview creation for small images (broke LA mode PNG)
- Enforce max_tile_size_mb on tiles with quality reduction and resize fallback
- Remove 64-tile hard cap that conflicted with max_tile_dimension_px
- Add comment explaining why original file is not saved for large images

* refactor(parse): remove max_tile_size_mb as it is a soft suggestion

max_tile_size_mb was a soft constraint that was not enforced
consistently. Remove it from config, constants, and all enforcement
logic. Tile dimension (max_tile_dimension_px) remains the sole constraint.

* fix(parse): use CJK-capable font for grid overlay labels

The old font loading only tried macOS-specific paths and fell back to
PIL's default bitmap font, which cannot render CJK characters in
filenames. Add a cross-platform CJK font lookup that covers Linux
(Noto/Droid/WQY/DejaVu), macOS (PingFang), and Windows (MSYH/SimSun).

* fix(parse): convert non-VLM-supported image formats to PNG on save

Image formats like TIFF, ICO, DIB, ICNS, SGI, JP2 are not recognized
by VLM backends (OpenAI/LiteLLM/VolcEngine only support PNG/JPEG/GIF/
WebP/BMP) or by embedding_utils for image vectorization. When a file
with one of these extensions is parsed, convert it to PNG and use a
.png extension so that downstream pipelines see consistent data.

SVG files (already PNG-converted via cairosvg) also get the .png
extension for the same reason.

* fix(parse): import io for SVG conversion

---------

Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-07-16 17:37:05 +08:00
19ca274a24 fix(retrieve): bound reranker input size (#3289)
* fix(retrieve): bound reranker input size

* fix(retrieve): make rerank input limit opt-in

---------

Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-07-16 17:19:38 +08:00
michaeltarletonandMichael Tarleton 3b7c61d266 fix(rerank): support Voyage via litellm (plain-string documents + dict results) (#3291)
LiteLLMRerankClient.rerank_batch wrapped each document as {"text": d} and read
result items via getattr(item, ...). That works for Cohere-style object results
but breaks Voyage through litellm: Voyage's rerank API rejects dict-wrapped
documents (400: 'documents' is not a valid string), and litellm returns Voyage
results as plain dicts, so getattr(item, "index") misses and rerank silently
falls back to a no-op.

- Pass documents as plain strings (litellm.rerank expects List[str]).
- Add _result_field() to read index/relevance_score from dict- or object-shaped
  result items.

Verified end-to-end against voyage/rerank-2.5 (scores now applied). Adds
tests/unit/models/rerank/test_litellm_rerank.py covering the plain-string
documents contract and both dict- and object-shaped results.

Co-authored-by: Michael Tarleton <mtarleton@istation.com>
2026-07-16 15:34:55 +08:00
agent ccc271ff27 feat: add extensible usage reporting (#3222)
* feat: add extensible usage reporting

* fix: harden usage reporter lifecycle

* fix: scope experience usage events correctly

* refactor: generalize usage event schema

* docs: design Codex experience memory tools

* docs: plan Codex experience memory tools

* feat: add Codex experience memory tools

* docs: remove temporary Codex implementation plans

* fix: capture Codex MCP tool parts

* fix: harden experience usage reporting

* fix: reload credentials for local MCP tools

* fix: preserve MCP tool-level errors

* fix: bound synchronous sink shutdown

* fix: enforce sink shutdown timeout

* fix: enforce experience tool contracts

* fix(usage-reporter): keep tool schemas and replay ids stable

* fix(usage-reporter): reject unidentifiable tool events
2026-07-16 15:33:27 +08:00
t0saki db6c91fa89 fix(mcp): advertise plain-typed tool schemas for strict function-calling APIs (#3288)
FastMCP derives tool input schemas from Python type hints, so Optional/
Union parameters become anyOf nodes with no top-level type, and nested
models become $ref/$defs. Valid JSON Schema, but clients that forward
these schemas verbatim to strict function-calling APIs break: Gemini's
OpenAPI 3.0 subset rejects the whole request (400: schema didn't specify
the schema type field), and n8n's JSON-schema-to-Zod conversion can
silently fall back and drop tool arguments entirely.

Rewrite the advertised schemas after registration: drop null branches,
collapse unions to their most general branch, inline $refs, and ensure
every node carries an explicit type. Runtime argument validation still
uses the original function signatures, so union parameters keep
accepting every branch (e.g. read still takes a bare URI string even
though the schema advertises an array).

Also type recall's other_peer_penalty honestly as
Optional[Union[float, Dict[str, float]]] instead of Optional[Any],
which produced an empty {} schema node.
2026-07-16 12:24:21 +08:00
Qin Haojie b04342798a feat(session): make generated IDs readable (#3287) 2026-07-16 11:30:24 +08:00
Yuanqing ZHAOandYuanqing Zhao 546da35cc8 fix(benchmark): load WIKI-Dir path mapping (#3279)
Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
2026-07-16 11:11:15 +08:00
huangruitengandhuangruiteng 0cf36f483e fix(deps): align bot requests with chardet 7 (#3282)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-16 11:09:20 +08:00
Qin Haojie d47f2106ee refactor: remove unused and deprecated APIs (#3272)
Delete dead compatibility paths and test-only helpers so unsupported APIs do not remain as accidental contracts.
2026-07-16 10:49:56 +08:00
chenjw 1b4534effd docs: 本地 trace 文件路径改名 + 补充产生/上传 trace 排查文档 (#3263)
* docs: 本地 trace 文件路径改名 + 补充产生/上传 trace 排查文档

- 将默认本地 trace 路径从 ~/.openviking/data/traces/offline-traces.jsonl
  改为 ~/.openviking/logs/traces.jsonl,不再建 traces 子目录
- 新增 upload_offline_trace.py 脚本,支持上传 JSONL trace 到远端 OTLP
- 上传脚本上传成功后收集并打印 trace_id 列表
- 在中英文 observability guide 中新增「产生本地 Trace 并提交排查」一节,
  覆盖用户开启 local trace、复现问题、提交 JSONL 给管理员、管理员上传的完整流程
- 更新 test_server_config_loader.py 断言

* fix: return non-zero exit code when main file missing or no batches uploaded

Address review feedback: when --file points to a non-existent path, or
when uploaded_batches == 0 (empty file / all lines invalid), the script
now returns exit code 1 instead of silently returning 0.
2026-07-16 10:16:12 +08:00
Yuanqing ZHAOandYuanqing Zhao 0080f94bdc perf(vectordb): batch benchmark upserts (#3264)
Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
2026-07-15 19:49:06 +08:00
t0saki 6f10e26361 perf(plugins): batch add-message writes and shared async detach across memory plugins (#3261)
* Batch add-message writes across memory plugins

* fix(plugins): stop batch enqueue at first failure and bump plugin versions

Keep pending-queue entries a contiguous prefix when a mid-stream enqueue
fails: consumers mark the first sent+queued payloads as captured, so a
queued entry after a gap could silently drop the gapped message.

Bump claude-code 0.4.3, codex 0.7.3, opencode 0.2.3, cursor/trae 0.1.2 so
existing installs pick up the batch write path.
2026-07-15 18:20:10 +08:00
Qin Haojie b27e326e4d fix(openclaw): restore peer_role routing (#3252)
* fix(openclaw): restore peer_role routing

The plugin refactor stopped applying peer_role to message attribution and data-plane actor headers, causing assistant mode to tag user messages and route every request as the agent. Restore role-aware routing across context and tool flows.

* fix(openclaw): close peer routing isolation gaps

* fix(openclaw): separate message peers from actor scope
2026-07-15 17:39:13 +08:00
Qin Haojie ca70bc0649 refactor(embedding): remove unused batch APIs (#3260) 2026-07-15 16:44:57 +08:00
zihengli 540139091f feat(connector): delegate add_resource imports to external Connector (#3084)
* feat(connector): delegate add_resource imports to external Connector

Opt-in integration that routes add_resource data fetching and parsing
to external Connector service; the Connector stages source data and
calls back into OV through the standard add_resource pipeline.

- add ConnectorClient wrapping the control plane's inner doc/add and
  task/info endpoints
- add [connector] config section: enable, connector/tracker endpoint
  URLs, timeout_seconds, poll_interval_ms, allowed_add_types
- route add_resource via Connector when enabled and args.add_type is
  in allowed_add_types; otherwise fall back to the standard pipeline
  with an info log
- track imports as connector_import TaskRecords and poll Connector
  task status in the background until terminal state or timeout

* fix(connector): delegate add_resource imports to external Connector
2026-07-15 16:34:23 +08:00
Qin Haojie 4a54dbeccf fix(session): resume commits after restart (#3254) 2026-07-15 16:01:17 +08:00
huangruitengandhuangruiteng 32acc6a32f fix(codex): compress type-quota recall results (#3248)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-15 14:44:21 +08:00
pppobear 5a6a1a0a03 fix(pi): avoid blocking user message rendering on recall (#3233) 2026-07-15 14:33:46 +08:00
fujiajie666 4847ffa378 模板优化 (#3242) 2026-07-15 11:37:47 +08:00
huangruitengandhuangruiteng abc325826b fix(auth): allow root key on trusted admin targets (#3249)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-15 11:04:22 +08:00
huangruitengandhuangruiteng 6de1cd2c6e test(oc2ov): fail fast on empty OpenClaw output (#3250)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-15 11:02:37 +08:00
Jiahui Zhou 1c46d44fbc Fix/reindex preserve owners (#3096)
* fix: preserve reindex content owners

feat: allow trusted admin role assertion

feat: prune orphan vectors during reindex

fix: harden reindex memory body reads

feat: expose reindex prune options in clients

fix(cli): prefer workspace sdk for compat clients

fix: harden reindex prune orphans

* test: align reindex expectations after rebase
2026-07-14 20:38:30 +08:00
huangruitengandhuangruiteng d14c0673ca fix(recall): hide memory fields metadata (#3240)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-14 17:41:30 +08:00
DuTao d2670274a4 feat(bot): Opt doc add workspace desc, change the type of messages appended to ov (#3236)
* 1. 默认开启OpenViking session处理;
2. 调整bot传给ov 的message类型

* 增加Agent workspace介绍

* fix pr
2026-07-14 17:37:20 +08:00
Haoyu Zhangandzhanghaoyu c665399cc7 fix: sync media vectorization extensions (#3241)
Co-authored-by: zhanghaoyu <zhanghaoyu.la@bytedance.com>
2026-07-14 17:36:56 +08:00