* feat(pathlock): pathlock support cache runtime
* perf(pathlock): shard Redis lock hashes by account
Route global, system, and account paths to separate HASH keys while retaining namespace slot affinity. Reject cross-scope batches and preserve lock lifecycle behavior within each scope.
* fix(pathlock): fix batch Redis stale token deletions too many results to unpack problem
* fix(test): update pathlock cache provider docs
* fix(pathlock): After the Redis lock times out, the incremental update reverts to an incorrect lock scope.
The api_key_watch_interval mechanism reloads account info across replicas
by polling compute_store_signature() (a (path, size, modTime) signature
over accounts.json + users.json). On S3FS this never detected writer-side
changes: S3FS stat() serves from a sliding-TTL StatCache (default 60s,
re-armed on every get()), so the watcher's 30s poll kept the entry alive
forever and the signature never moved -> reload() never fired. localfs
stats live, so it worked there.
Thread a bypass_cache flag from the watcher's stat call down to S3FS so
signature stats read fresh backend metadata:
- FsContext(View): add bypass_cache field + builder/getter
- ragfs-python build_fs_context: parse ctx["bypass_cache"]
- S3FS stat(): skip stat_cache.get() when bypass_cache is set
- AsyncAGFSClient.stat(bypass_cache=...): inject ctx flag
- legacy _stat_signature: stat with bypass_cache=True
Co-authored-by: TRAE CLI <traecli@bytedance.com>
- keep handed-off leases in the registry with pending_handoff so auto-refresh continues
- include lease_ref in PathLockHandoffRef for same-process adopt fast path
- rotate both lease_ref and ownership_ref on adopt to invalidate stale producer capabilities
- reject stale capability use in release, release_selected and refresh
- validate owner_id, lock_paths and covered_paths before local adopt
- reject replayed fallback adopt when the same owner/path token is already held
- add tests for pending handoff refresh, retryable adopt race, forged coverage and replay rejection
- add storage.agfs.pathlock.lock_timeout_secs
- use pathlock default timeout instead of hardcoded zero in wrapper
- map legacy storage.transaction.lock_timeout when new config is unset
- remote redolog by using persistent `session_commit` queue.
* feat(storage): optimize glob func
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* feat(rgafs): implement paged glob traversal without full tree materialization
* fix(localfs): offload blocking fs operations to spawn_blocking
* feat(glob): cap glob api default node_limit at 256
* feat(sdk): add node_limit options for glob in python and go SDKs
* add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide
add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide
* Handle poisoned known_keys mutexes in cache providers
* Add configurable cache support to ragfs python
* Add Redis cache provider support
* Prune native cache providers from default build
* Add RAGFS cache guides in Chinese and English
* delete .cargo/config.toml
* Fix stale cache invalidation across shared wrappers
* Document Yuanrong native concurrency limits
* add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide
add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide
* Handle poisoned known_keys mutexes in cache providers
* Add configurable cache support to ragfs python
* Add Redis cache provider support
* Prune native cache providers from default build
* Add RAGFS cache guides in Chinese and English
* delete .cargo/config.toml
* Fix stale cache invalidation across shared wrappers
* Document Yuanrong native concurrency limits
* Limit directory cache entries and add regression test
* docs: add TOS s3fs backend test design
* Add runtime cache config override
* add build guides for mooncake and yuanrong
* fix _FakeConfig test with cache
* fix(ragfs): cache encrypted data below the encryption layer
- pass runtime cache config to the Rust binding
- cache ciphertext instead of decrypted content
- disable cache for encrypted multi-write mounts
- update tests and provider build documentation
* fix(ragfs): invalidate cache after same-mount raw copy
Ensure copy_within_mount invalidates the cached destination file and
parent directory after the raw backend fast path writes data directly.
This prevents stale reads and stale directory metadata when cache is
enabled and Python cp() uses the same-mount copy optimization.
Add a regression test covering overwrite copy followed by read/list.
* fix(ragfs): preserve multi-write discovery through cache layer
Allow MountableFS::as_multiwrite() to unwrap CachedFileSystem when
discovering the underlying MultiWriteWrappedFS. This preserves
multi-write admin paths and same-mount copy behavior for cache-enabled,
unencrypted multi-write mounts.
Add a regression test covering sync status, sync retry, same-mount copy,
and unmount behavior for cached unencrypted multi-write mounts.
* feat(ragfs): add cache-aware tree traversal mode
- add configurable tree traversal mode to cache policy
- keep default tree behavior delegated to backend
- allow cached traversal to reuse read_dir directory cache
- bypass cached traversal for multi-write backends
- add regression coverage for tree cache behavior and fallbacks
* docs: design cache-aware grep traversal
* feat(ragfs): add cache-aware grep traversal
Introduce a shared cache traversal mode for recursive APIs and use it to
optionally run grep through CachedFileSystem.
- add CacheTraversalMode with backend and cached_traversal modes
- keep CacheTreeMode as a compatibility alias
- route tree and grep through cached traversal only when explicitly enabled
- reuse cached read_dir entries and full-file reads during grep traversal
- keep multi-write traversal on the backend path
- expose storage.agfs.cache.traversal_mode in Python config
- raise max cached directory entries threshold to 4096
- add regression tests for grep cache traversal and traversal config
* Optimize cached grep generation validation
* Parallelize cached grep file scanning
---------
Co-authored-by: fang <fang@fangMacBook-Air.local>
* feat(storage): optimize tree func
* feat(storage): add test case && fix tree show parent dir problem
* feat(storage): format code
* feat(storage): fix check problem
* feat(storage): remove redundant unit test
* feat(storage): fix code review issue
* feat(storage): fix code review issue
* feat: ov add-resource (spec -L --level), ov stat (return count for dir)
* feat: ov add-resource (spec -L --level), ov stat (return count for dir)
* feat: Add VLM backup configuration for automatic failover
- Add backup field to VLMConfig with recursive backup prevention
- Implement FailoverVLM wrapper class for automatic failover
- Support rate limit, timeout, server error triggers
- Add comprehensive unit tests
* feat: Add VLM backup configuration for automatic failover
* feat: Add VLM backup configuration for automatic failover
* ov observer filesystem
* ov observer filesystem
* perf(storage): optimize VikingFS grep implementation
- add ripgrep support and native local grep fallback for LocalFS mode
- normalize grep path handling and improve regression test coverage
* perf(storage): optimize VikingFS grep implementation
- add ripgrep support and native local grep fallback for LocalFS mode
- normalize grep path handling and improve regression test coverage
* perf(storage): format code
* perf(storage): support async grep
* feat(encryption): Push `exclude_uri` and `level_limit` down to the backend to ensure that the limits take effect after the same set of filtering conditions. && The default implementation also follows the same path contract as LocalFS, returning the path relative to the query root
* feat(encryption): Push `exclude_uri` and `level_limit` down to the backend to ensure that the limits take effect after the same set of filtering conditions. && The default implementation also follows the same path contract as LocalFS, returning the path relative to the query root
* fix(storage): relativize localfs exclude_path against grep query root && disable parent .gitignore inheritance in localfs rg fast path && add HTTP AGFS grep support for exclude_path and level_limit
* fix(ci): fix ragfs-python native extension build in CI pipelines
* fix(ci): skip sessions/ tests when no VLM/Embedding secrets available
Sessions tests (add_message, commit, used) depend on storage backend
that requires proper initialization with valid API keys. When running
without secrets (PR from fork), these tests fail with 400 because the
storage layer cannot fully initialize. Skip them in no-secrets mode;
they will still run on main branch pushes where secrets are available.
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* reorg: rewrite agfs with rust, and named with ragfs, keep License
* fix: grep level limit
* fix: grep root
* fix: import error
* fix: rust code optimazation
* fix: CI error
* fix: CI go mod cache
* fix: grep level limit
* fix: CI
---------
Co-authored-by: openviking <openviking@example.com>
