Commit Graph
30 Commits
Author SHA1 Message Date
txfangandchrisfang e7f58639e3 feat(cache): add DynamicProvider support through a versioned C ABI refactor (#4568)
* feat(ragfs): unify cache runtime providers

* refactor(ragfs): remove bundled external cache providers

* refactor(ragfs): unify cache runtime and redis backend

* chore: sync contributing guide with main

* fix(cache): harden Redis runtime compatibility

* refactor(cache): gate memory mock behind test feature

* fix(cache): harden Redis runtime integration

* feat(cache): add dynamic provider C ABI

* fix(cache): harden dynamic provider FFI

---------

Co-authored-by: chrisfang <chrisfang@noreply.gitcode.com>
2026-09-02 16:48:40 +08:00
txfangandchrisfang 3123e8d8ff feat(ragfs): unify CacheRuntime and Redis-backed CacheFS/QueueFS (#4353)
* feat(ragfs): unify cache runtime providers

* refactor(ragfs): remove bundled external cache providers

* refactor(ragfs): unify cache runtime and redis backend

* chore: sync contributing guide with main

* fix(cache): harden Redis runtime compatibility

* refactor(cache): gate memory mock behind test feature

* fix(cache): harden Redis runtime integration

---------

Co-authored-by: chrisfang <chrisfang@noreply.gitcode.com>
2026-08-31 11:10:08 +08:00
baojun-zhang 8c9c2282a6 feat(queuefs): support redis as queufs backend ,redis mode support singleton 、cluster 、 sentinel (#3741) 2026-08-05 17:19:42 +08:00
baojun-zhang df1557901e feat(ragfs): add structured lock tracing logs (#3693)
* feat(ragfs): add structured lock tracing logs

* feat(ragfs): add log async writer

* fix(ragfs): add CRITICAL trans && add rust hook to receive rotate log file
2026-08-03 15:58:55 +08:00
Hao Zheandzhiheng.liu 4237c66031 fix(bot): harden scheduling, sessions, sandbox, and packaging (#3549)
* fix(bot): make SRT sandbox constructible and surface startup failures

Sweep findings: C-02, C-04. Read SRT settings from the correct config and never cache failed startups.

(cherry picked from commit 0d3798399a)

* fix(bot): reject unschedulable cron jobs and honest manual runs

Sweep findings: C-07, C-08. Reject impossible schedules and fail no-op manual execution.

(cherry picked from commit 47413fdffe)

* build: ship VikingBot workspace and bridge assets in wheels

Sweep findings: F-07. Copy required bot assets into the package and resolve installed workspace templates correctly.

(cherry picked from commit 85e9ff2d06)

* fix(bot): OpenAPI channel concurrency, session deletion, and context handling

Sweep findings: C-09, C-10, C-11. Reject ambiguous requests and make session deletion durable.

(cherry picked from commit 68fd70c5be)

* docs(bot): correct channel setup config paths and license metadata

Sweep findings: C-13, C-15. Point channel setup at ov.conf and align package metadata with MIT.

(cherry picked from commit b15c241037)

* fix(review): align bot license metadata

Addresses blocking review finding on #3426.

(cherry picked from commit 001e0fa7ed)

* fix(bot): preserve cron scheduler liveness

Record callback-less scheduled runs as job errors so recurring timers persist and rearm, while manual runs still fail without mutating state. Document the OpenAPI context rejection and same-session concurrency contract.

* fix(ragfs): use stable Windows file identity APIs

---------

Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
2026-07-31 15:57:43 +08:00
baojun-zhang 2f9451231e refactor(pathlock):using rust implement instead python (#3602)
* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):optimize unit test code

* refactor(pathlock):optimize encryption create func

* refactor(pathlock):avoid releasing handoffed pathlock on enqueue errors

* fix(pathlock): use owned lease capability and handle S3 create-new 409 as conflict

* fix(ragfs): keep original FsContext for multi-write metadata

* fix(pathlock): resolve lease coverage and CAS handling issues

- detect S3 conditional conflicts from structured service errors
- pass transaction leases when deleting skill roots
- let temp cleanup acquire locks for temp paths
- disambiguate cache and pathlock providers in cache tests
- update temp cleanup lease assertions

* fix(ragfs): bypass pathlock for multi-write metadata

* fix(ragfs): revert pathlock fail-fast design

* fix(ragfs):fix(ragfs): use non-blocking fcntl locks for localfs CAS

* fix(ragfs): serialize heartbeat lease refresh with release and report real conflict kind

* fix(ragfs): preserve conflict kind snapshot and drop unused test scaffolding

* fix(ragfs): preserve conflict kind snapshot and drop unused test scaffolding
2026-07-29 19:45:34 +08:00
baojun-zhang 1841dfed81 Revert "refactor(pathlock):using rust implement instead python (#3557)" (#3597)
This reverts commit 6b538db569.
2026-07-29 11:31:41 +08:00
baojun-zhang 6b538db569 refactor(pathlock):using rust implement instead python (#3557)
* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):optimize unit test code

* refactor(pathlock):optimize encryption create func

* refactor(pathlock):avoid releasing handoffed pathlock on enqueue errors

* fix(pathlock): use owned lease capability and handle S3 create-new 409 as conflict

* fix(ragfs): keep original FsContext for multi-write metadata

* fix(pathlock): resolve lease coverage and CAS handling issues

- detect S3 conditional conflicts from structured service errors
- pass transaction leases when deleting skill roots
- let temp cleanup acquire locks for temp paths
- disambiguate cache and pathlock providers in cache tests
- update temp cleanup lease assertions

* fix(ragfs): bypass pathlock for multi-write metadata

* fix(ragfs): revert pathlock fail-fast design
2026-07-29 11:08:42 +08:00
agent 27debfe7a8 feat(snapshot): add path diff API, SDK, and CLI (#3451)
* feat(snapshot): add path diff API

* feat(snapshot): expose path diff in clients and CLI

* fix(snapshot): pin path diff to resolved commits

* fix(snapshot): bound path diff resource usage

* fix(snapshot): bound diff blob reads

* fix(snapshot): harden path diff API

* fix(snapshot): bound path diff resource usage

* fix(build): sync native binding dependencies
2026-07-23 17:45:36 +08:00
Qin Haojie 3003ed61d7 feat(retrieval): support image search (#3093)
Add multimodal image vectorization and image query support across the server, SDKs, and CLI.
2026-07-09 16:42:53 +08:00
baojun-zhang 6a33ebb7ca Optimize glob walkdir (#3013)
* feat(storage): optimize glob func

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* feat(rgafs): implement paged glob traversal without full tree materialization

* fix(localfs): offload blocking fs operations to spawn_blocking

* feat(glob): cap glob api default node_limit at 256

* feat(sdk): add node_limit options for glob in python and go SDKs
2026-07-06 21:39:16 +08:00
Haoyu Zhangandzhanghaoyu.la aa53e7aede feat: 实现 commit、restore、show 文件系统多版本管理功能 (#2756)
* feat: 实现 commit、restore、show 文件系统多版本管理功能

fix: 修复commit时删除文件

fix: commit 的 fast path 1 添加 Racy-clean 机制

fix: 将 sdk 中的 git 命令改为 snapshot 命令,同步修改单测

fix: 多版本管理的文件存储目录改为 .ovgit

feat: snapshot cli 渲染

fix: 修复 restore 时将删除的文件回滚时,目录不存在的问题

feat: restore 命令的 project_dir 参数改为可选,不传时默认全目录回滚

feat: 更新文档

fix: 删除暂未使用的配置参数

feat: 新增示例脚本

fix: 修复示例代码

fix: 修复 restore 返回的 task id 任务完成状态

feat: 在 restore 修改文件系统时加锁

fix: fix openviking_sdk

* feat: 将git多版本管理功能改为默认打开,并复用agfs的配置参数作为默认值

* fix: restore 命令改为先完成 ref 一致性协议再写回 VFS;object store 并发改为使用唯一 temp path

* fix: 在 git 配置检验层去除未实现的cas_mode = "redis_lock"模式

* fix: 在 Rust GitService 边界统一校验 account

* fix: 当前commit不支持通过 path 传入目录,增加报错信息

* fix: 将git文件默认存储路径统一为 .ovgit

* fix: restore 时写入 VFS 失败时返回详细的报错,并继续触发 reindex

* fix: 校验 commit、restore、show 的路径

* feat: 实现 commit 时指定目录

---------

Co-authored-by: zhanghaoyu.la <zhanghaoyu.la@bytedance.com>
2026-06-25 11:28:04 +08:00
Zayn Jarvis f869731485 Support gzip responses in Rust CLI (#2783) 2026-06-23 12:12:47 +08:00
tuofangandfang 2d56bd71f4 feat(ragfs): add CachedFileSystem and Redis/Mooncake/Yuanrong cache providers (#2520)
* add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

* Handle poisoned known_keys mutexes in cache providers

* Add configurable cache support to ragfs python

* Add Redis cache provider support

* Prune native cache providers from default build

* Add RAGFS cache guides in Chinese and English

* delete .cargo/config.toml

* Fix stale cache invalidation across shared wrappers

* Document Yuanrong native concurrency limits

* add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

* Handle poisoned known_keys mutexes in cache providers

* Add configurable cache support to ragfs python

* Add Redis cache provider support

* Prune native cache providers from default build

* Add RAGFS cache guides in Chinese and English

* delete .cargo/config.toml

* Fix stale cache invalidation across shared wrappers

* Document Yuanrong native concurrency limits

* Limit directory cache entries and add regression test

* docs: add TOS s3fs backend test design

* Add runtime cache config override

* add build guides for mooncake and yuanrong

* fix _FakeConfig test with cache

* fix(ragfs): cache encrypted data below the encryption layer

- pass runtime cache config to the Rust binding
- cache ciphertext instead of decrypted content
- disable cache for encrypted multi-write mounts
- update tests and provider build documentation

* fix(ragfs): invalidate cache after same-mount raw copy

Ensure copy_within_mount invalidates the cached destination file and
parent directory after the raw backend fast path writes data directly.
This prevents stale reads and stale directory metadata when cache is
enabled and Python cp() uses the same-mount copy optimization.

Add a regression test covering overwrite copy followed by read/list.

* fix(ragfs): preserve multi-write discovery through cache layer

Allow MountableFS::as_multiwrite() to unwrap CachedFileSystem when
discovering the underlying MultiWriteWrappedFS. This preserves
multi-write admin paths and same-mount copy behavior for cache-enabled,
unencrypted multi-write mounts.

Add a regression test covering sync status, sync retry, same-mount copy,
and unmount behavior for cached unencrypted multi-write mounts.

* feat(ragfs): add cache-aware tree traversal mode

- add configurable tree traversal mode to cache policy
- keep default tree behavior delegated to backend
- allow cached traversal to reuse read_dir directory cache
- bypass cached traversal for multi-write backends
- add regression coverage for tree cache behavior and fallbacks

* docs: design cache-aware grep traversal

* feat(ragfs): add cache-aware grep traversal

Introduce a shared cache traversal mode for recursive APIs and use it to
optionally run grep through CachedFileSystem.

- add CacheTraversalMode with backend and cached_traversal modes
- keep CacheTreeMode as a compatibility alias
- route tree and grep through cached traversal only when explicitly enabled
- reuse cached read_dir entries and full-file reads during grep traversal
- keep multi-write traversal on the backend path
- expose storage.agfs.cache.traversal_mode in Python config
- raise max cached directory entries threshold to 4096
- add regression tests for grep cache traversal and traversal config

* Optimize cached grep generation validation

* Parallelize cached grep file scanning

---------

Co-authored-by: fang <fang@fangMacBook-Air.local>
2026-06-17 16:01:52 +08:00
baojun-zhang eff7e67037 feat(storage): support content-type auto-detecting in S3 case (#2668)
* feat(storage): support content-type auto-detecting in S3 case

* feat(storage): update code doc
2026-06-16 18:24:17 +08:00
Hao Zhe 402d51f7c0 fix(cli): render filesystem modtimes locally (#2576) 2026-06-12 18:36:54 +08:00
DuTao 78471b6485 feat(skill):Add skills cli and api, support add skills from git, support update skills (#2453)
* add skills cli and api

* add api source

* fix agent space

* fix bug

* fix bug

* fix pr bug

* fix pr bug

* fix pr new bug
2026-06-09 15:29:06 +08:00
baojun-zhang e492cbd16f refactor(encryption): using rust refactor encryption (#2444) 2026-06-05 17:26:26 +08:00
baojun-zhang 7e43428f04 refactor(storage): deprecated and remove agfs http mode client (#2425)
* refactor(storage): delete agfs http mode client

* refactor(storage): refactor protocol to support diff return type

* refactor(storage): refactor code format from lint
2026-06-04 17:36:43 +08:00
baojun-zhang 86f61ee16e perf(storage): optimize s3 backend grep performance (#2078)
* perf(storage): optimize s3 backend grep performance

* feat(storage): optimize git pr suggestion
2026-05-20 16:53:51 +08:00
zgy 79389ee474 feat(retriever): add level filter to find with retriever-level filtering, preserving recursive navigation (#1988)
* Feat(retriever): add level filter to find with retriever-level filtering, preserving recursive navigation

Add level: Optional[List[int]] parameter to find API/CLI/SDK to filter
results by L0 (abstract), L1 (overview), L2 (original file).

Key design: filter at two result collection points inside
HierarchicalRetriever (global search pool + recursive traversal pool),
NOT at vector search layer or post-filter layer. This preserves L0/L1
directory waypoints in dir_queue for recursive navigation, avoiding the
quality regression that merge_level_filter (PR #1980) causes.

Changes:
- Python: FindRequest, SearchService, VikingFS, LocalClient,
  AsyncOpenViking, SyncOpenViking, MCP endpoint all pass level through
- Retriever: initial_candidates and collected_by_uri filtered by level;
  dir_queue navigation unchanged
- Fix variable shadowing: rename debug loop 'level' to 'result_level'
- Add stagnation detection to convergence check when level filter
  prevents reaching limit
- CLI: --level / -L flag with Option<Vec<i32>> + value_delimiter
- Remove merge_level_filter from find route (breaks recursive navigation)
- Tests: 8 new test cases covering param passthrough, backward compat,
  single/mixed level filtering, and edge cases

* feat(search): 为 search 方法添加 level 过滤,与 find 保持一致

- HTTP 路由层:FindRequest.level 改为 Union[int, str, List[int]],search 路由传递 level 参数
- Service 层:SearchService.search 增加 level 参数
- 存储层:VikingFS.search 增加 level 参数,传递给 retriever.retrieve
- SDK 层:LocalClient/AsyncOpenViking/SyncOpenViking.search 增加 level 参数
- MCP 端点:search 工具增加 level 参数
- CLI 层:ov search --level 从 Option<String> 改为 Option<Vec<i32>>,与 find 一致
- 删除 append_level_filter_params,改用内联格式化
- 路由层用 _resolve_levels() 统一将 Union 类型转为 List[int]
- 新增 9 个测试:7 个 search level 测试 + 2 个 find Union 类型输入测试
2026-05-14 18:06:03 +08:00
MaojiaShengandopenviking 9dc98aa133 feat: ov CLI support (#1916)
* feat: support ov config switch and ov config setup-cli

* feat: support ov config switch and ov config setup-cli

* feat: add ov tui image preview

---------

Co-authored-by: openviking <openviking@example.com>
2026-05-08 18:51:31 +08:00
Jiahui Zhouandqin-ctx ac3346422a feat(rebuild): add rebuild api scaffold (#1592)
* feat(admin): add rebuild api scaffold

feat: add admin rebuild API

fix: harden admin rebuild execution

feat(cli): add rebuild command support

fix(rebuild): support namespace rebuild routing

refactor(rebuild): unify memory semantic rebuild mode

refactor(rebuild): move http endpoint to content route

fix(rebuild): skip root namespace vectorization

fix(rebuild): harden namespace classification

refactor: rename rebuild api to reindex

refactor: rename reindex executor module

refactor(reindex): remove unused reason field

* fix(reindex): tighten namespace URI handling

Share segment-based Viking URI classification across context inference and reindex execution, add skill namespace support, and require root reindex requests to select an account.

* refactor: reuse indexing pipeline in reindex

* Revert "refactor: reuse indexing pipeline in reindex"

This reverts commit 2725fe6733.

* fix(reindex): respect semantic vectorization skips

Avoid scheduling semantic DAG vectorization work during semantic_and_vectors reindex, and keep resource vector text selection aligned with normal vectorize_file handling for non-text files.

---------

Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-05-08 17:48:33 +08:00
baojun-zhang c8ff9e3575 perf(storage): optimize VikingFS grep implementation (#1731)
* perf(storage): optimize VikingFS grep implementation
- add ripgrep support and native local grep fallback for LocalFS mode
- normalize grep path handling and improve regression test coverage

* perf(storage): optimize VikingFS grep implementation
- add ripgrep support and native local grep fallback for LocalFS mode
- normalize grep path handling and improve regression test coverage

* perf(storage): format code

* perf(storage): support async grep

* feat(encryption): Push `exclude_uri` and `level_limit` down to the backend to ensure that the limits take effect after the same set of filtering conditions. &&  The default implementation also follows the same path contract as LocalFS, returning the path relative to the query root

* feat(encryption): Push `exclude_uri` and `level_limit` down to the backend to ensure that the limits take effect after the same set of filtering conditions. &&  The default implementation also follows the same path contract as LocalFS, returning the path relative to the query root

* fix(storage): relativize localfs exclude_path against grep query root && disable parent .gitignore inheritance in localfs rg fast path && add HTTP AGFS grep support for exclude_path and level_limit
2026-05-07 19:41:33 +08:00
MaojiaShengandopenviking 3ed8669248 feat: ov --progress (#1872)
* feat: rust CLI:   - 默认不显示进度条,保持静默操作
  - 使用 ov --progress add-resource ... 开启进度显示
  - 在 ~/.openviking/ovcli.conf 中设置 show_progress = true 永久开启
  - 配合 -v, --verbose 可以看到更详细的信息(比如正在添加哪些文件)

* feat: rust CLI:   - 默认不显示进度条,保持静默操作
  - 使用 ov --progress add-resource ... 开启进度显示
  - 在 ~/.openviking/ovcli.conf 中设置 show_progress = true 永久开启
  - 配合 -v, --verbose 可以看到更详细的信息(比如正在添加哪些文件)

* feat: rust CLI:   - 默认不显示进度条,保持静默操作
  - 使用 ov --progress add-resource ... 开启进度显示
  - 在 ~/.openviking/ovcli.conf 中设置 show_progress = true 永久开启
  - 配合 -v, --verbose 可以看到更详细的信息(比如正在添加哪些文件)

---------

Co-authored-by: openviking <openviking@example.com>
2026-05-06 21:01:46 +08:00
Jiahui Zhou b174deb410 Fix ci (#1307)
* fix(ci): fix build ci

* build: move ragfs-python packaging into setup.py
2026-04-09 00:05:18 +08:00
MaojiaShengandopenviking dc052ce3e3 reorg: Rewrite agfs to ragfs with rust (#1221)
* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* fix: grep level limit

* fix: grep root

* fix: import error

* fix: rust code optimazation

* fix: CI error

* fix: CI go mod cache

* fix: grep level limit

* fix: CI

---------

Co-authored-by: openviking <openviking@example.com>
2026-04-05 21:17:45 +08:00
baojun-zhang 8cdecd9163 feat: Add multi-tenant file encryption capability (#828)
* feat: Add multi-tenant file encryption capability

* feat: move cli command `ov crypto` to `ov system crypto` && ruff format && add encryption config example

* feat: reformat code

* feat: adjust crypto.rs to support diff platform

* feat: reformat code
2026-03-21 13:35:46 +08:00
MaojiaShengandopenviking 0fb8f13a13 fix: windows zip path, code repo indexing, search retrieval, account id, rust cli version... (#577)
* fix: windows zip path norm

* fix: account id in vector db

* fix: add some log

* fix: add some log, and fixed search

* fix: add some log, and fixed search

* fix: add some log, and fixed search

* fix: add some log, and fixed search

* fix: add some log, and fixed search

* fix: add some log, and fixed search

* fix: add some log, and fixed search

* fix: add some log, and fixed search

---------

Co-authored-by: openviking <openviking@example.com>
2026-03-15 10:04:51 +08:00
zhoujiahui 8f1fef9cdf fix(Dockerfile): add rust ov (#570) 2026-03-13 16:03:42 +08:00