Commit Graph
7 Commits
Author SHA1 Message Date
txfangandchrisfang 3123e8d8ff feat(ragfs): unify CacheRuntime and Redis-backed CacheFS/QueueFS (#4353)
* feat(ragfs): unify cache runtime providers

* refactor(ragfs): remove bundled external cache providers

* refactor(ragfs): unify cache runtime and redis backend

* chore: sync contributing guide with main

* fix(cache): harden Redis runtime compatibility

* refactor(cache): gate memory mock behind test feature

* fix(cache): harden Redis runtime integration

---------

Co-authored-by: chrisfang <chrisfang@noreply.gitcode.com>
2026-08-31 11:10:08 +08:00
tuofangandfang 2d56bd71f4 feat(ragfs): add CachedFileSystem and Redis/Mooncake/Yuanrong cache providers (#2520)
* add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

* Handle poisoned known_keys mutexes in cache providers

* Add configurable cache support to ragfs python

* Add Redis cache provider support

* Prune native cache providers from default build

* Add RAGFS cache guides in Chinese and English

* delete .cargo/config.toml

* Fix stale cache invalidation across shared wrappers

* Document Yuanrong native concurrency limits

* add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

add CachedFileSystem + CacheProvider trait and Mooncake/Yuanrong Provide

* Handle poisoned known_keys mutexes in cache providers

* Add configurable cache support to ragfs python

* Add Redis cache provider support

* Prune native cache providers from default build

* Add RAGFS cache guides in Chinese and English

* delete .cargo/config.toml

* Fix stale cache invalidation across shared wrappers

* Document Yuanrong native concurrency limits

* Limit directory cache entries and add regression test

* docs: add TOS s3fs backend test design

* Add runtime cache config override

* add build guides for mooncake and yuanrong

* fix _FakeConfig test with cache

* fix(ragfs): cache encrypted data below the encryption layer

- pass runtime cache config to the Rust binding
- cache ciphertext instead of decrypted content
- disable cache for encrypted multi-write mounts
- update tests and provider build documentation

* fix(ragfs): invalidate cache after same-mount raw copy

Ensure copy_within_mount invalidates the cached destination file and
parent directory after the raw backend fast path writes data directly.
This prevents stale reads and stale directory metadata when cache is
enabled and Python cp() uses the same-mount copy optimization.

Add a regression test covering overwrite copy followed by read/list.

* fix(ragfs): preserve multi-write discovery through cache layer

Allow MountableFS::as_multiwrite() to unwrap CachedFileSystem when
discovering the underlying MultiWriteWrappedFS. This preserves
multi-write admin paths and same-mount copy behavior for cache-enabled,
unencrypted multi-write mounts.

Add a regression test covering sync status, sync retry, same-mount copy,
and unmount behavior for cached unencrypted multi-write mounts.

* feat(ragfs): add cache-aware tree traversal mode

- add configurable tree traversal mode to cache policy
- keep default tree behavior delegated to backend
- allow cached traversal to reuse read_dir directory cache
- bypass cached traversal for multi-write backends
- add regression coverage for tree cache behavior and fallbacks

* docs: design cache-aware grep traversal

* feat(ragfs): add cache-aware grep traversal

Introduce a shared cache traversal mode for recursive APIs and use it to
optionally run grep through CachedFileSystem.

- add CacheTraversalMode with backend and cached_traversal modes
- keep CacheTreeMode as a compatibility alias
- route tree and grep through cached traversal only when explicitly enabled
- reuse cached read_dir entries and full-file reads during grep traversal
- keep multi-write traversal on the backend path
- expose storage.agfs.cache.traversal_mode in Python config
- raise max cached directory entries threshold to 4096
- add regression tests for grep cache traversal and traversal config

* Optimize cached grep generation validation

* Parallelize cached grep file scanning

---------

Co-authored-by: fang <fang@fangMacBook-Air.local>
2026-06-17 16:01:52 +08:00
MaojiaShengandopenviking a7e5417ef2 reorg: remove golang depends (#1339)
* docs: fix docker deployment

* reorg: remove third_party/agfs

* feat(s3fs): add disable_batch_delete option for OSS compatibility

Port of PR #1333 from Go version to Rust:

- Add disable_batch_delete config option to S3Client
- When enabled, use sequential single-object deletes instead of DeleteObjects
- This is for S3-compatible services like Alibaba Cloud OSS that require
  Content-MD5 for DeleteObjects but AWS SDK v2 does not send it by default
- Add documentation and config example for OSS

* fix(s3fs): pass disable_batch_delete config from Python to Rust

Add disable_batch_delete to the s3_plugin_config dict in _generate_plugin_config
so that the Python config can properly control the Rust S3FS plugin's behavior.

* reorg: remove third_party/agfs

* reorg: remove third_party/agfs

* change some docs

* change some docs

---------

Co-authored-by: openviking <openviking@example.com>
2026-04-10 15:16:29 +08:00
MaojiaShengandopenviking dc052ce3e3 reorg: Rewrite agfs to ragfs with rust (#1221)
* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* reorg: rewrite agfs with rust, and named with ragfs, keep License

* fix: grep level limit

* fix: grep root

* fix: import error

* fix: rust code optimazation

* fix: CI error

* fix: CI go mod cache

* fix: grep level limit

* fix: CI

---------

Co-authored-by: openviking <openviking@example.com>
2026-04-05 21:17:45 +08:00
zhoujiahui 1f1df6ea7e fix(packaging): sdist 排除运行时二进制 (#447)
- MANIFEST.in 中 prune openviking/bin、openviking/lib,并排除 *.so/*.dylib/*.dll/*.exe
- sdist 构建前在 CI 定点清理可能残留的本地二进制
2026-03-05 20:20:14 +08:00
kkkwjx ee5cfd0181 fix: fix_build (#6) 2026-01-30 11:17:08 +08:00
qin-ctx f98dc0ed1c first commit 2026-01-29 20:29:19 +08:00