Commit Graph
23 Commits
Author SHA1 Message Date
MaojiaShengandopenviking a7e5417ef2 reorg: remove golang depends (#1339)
* docs: fix docker deployment

* reorg: remove third_party/agfs

* feat(s3fs): add disable_batch_delete option for OSS compatibility

Port of PR #1333 from Go version to Rust:

- Add disable_batch_delete config option to S3Client
- When enabled, use sequential single-object deletes instead of DeleteObjects
- This is for S3-compatible services like Alibaba Cloud OSS that require
  Content-MD5 for DeleteObjects but AWS SDK v2 does not send it by default
- Add documentation and config example for OSS

* fix(s3fs): pass disable_batch_delete config from Python to Rust

Add disable_batch_delete to the s3_plugin_config dict in _generate_plugin_config
so that the Python config can properly control the Rust S3FS plugin's behavior.

* reorg: remove third_party/agfs

* reorg: remove third_party/agfs

* change some docs

* change some docs

---------

Co-authored-by: openviking <openviking@example.com>
2026-04-10 15:16:29 +08:00
yuan7heandyuanqianhe f72055d973 feat(s3fs): add disable_batch_delete option for OSS compatibility (#1330) (#1333)
Co-authored-by: yuanqianhe <yuanqianhe@lattebank.com>
2026-04-09 20:25:51 +08:00
Jiahui Zhou 2ec8558155 Add non-empty S3 directory marker compatibility (#997)
Add non-empty

Add S3 directory marker modes
2026-03-26 16:37:44 +08:00
Qin HaojieandClaude Opus 4.6 1823a7c4f7 feat(storage): add path locking and selective crash recovery for write operations (#431)
* feat(storage): add transaction support with journal, undo, and crash recovery

Implement a full transaction system for VikingFS storage operations including
write-ahead journal, path locking, undo/rollback, context manager API, and
crash recovery. Includes comprehensive tests and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(transaction): add e2e rollback tests for mv and multi-step operations

Add end-to-end tests covering rollback scenarios that were missing:
- mv rollback: file moved back to original location on failure
- mv commit: file persists at new location
- Multi-step rollback: mkdir + write + mkdir all reversed in order
- Partial step rollback: only completed entries are reversed
- Nested directory rollback: child removed before parent
- Best-effort rollback: single step failure does not block others

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(storage): add transaction support with path locking and journal

Implement transaction system for VikingFS with ACID-like guarantees:
- TransactionManager with configurable lock timeout and journal-based recovery
- PathLock supporting point, subtree, and mv lock modes
- Refactor VikingFS mv to use cp+rm to prevent lock files from being carried
- Fix stale lock detection returning false for missing lock files
- Update ragas eval to use LangchainLLMWrapper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: tests

* fix(transaction): fix rollback and race condition bugs

- Reconstruct RequestContext from undo params for vectordb_delete/update_uri
  rollback (previously skipped silently due to missing ctx)
- Serialize ctx fields into undo params in rm/mv operations
- Fix Phase 1 undo path to target archive dir instead of session root
- Remove Phase 2 fs_write_new undo (overwrites are idempotent, checkpoint
  handles recovery)
- Add ancestor SUBTREE recheck after lock creation in acquire_subtree
- Move _collect_uris inside TransactionContext in rm/mv to close race window
- Log journal persistence failures instead of silently swallowing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(transaction): make TransactionManager required and rewrite tests with real backends

Remove all optional/fallback code paths where tx_manager could be None. get_transaction_manager()
now raises RuntimeError if not initialized. Fix undo rollback to reconstruct ctx for vectordb_upsert
and use correct agent_id default. Replace mock-based transaction tests with integration tests using
real AGFS and VectorDB backends.

* refactor(transaction): make rollback fully async and unify session commit path

- Convert execute_rollback/rollback_entry to async, removing sync run_async wrappers
- Unify Session.commit() to delegate to commit_async(), removing duplicate phase methods
- Fix SUBTREE lock to conflict with ancestor SUBTREE locks (was previously missing)
- Fix mv lock mode: directory moves now use SUBTREE on both source and destination
- Replace deprecated asyncio.get_event_loop() with get_running_loop()
- Remove max_parallel_locks config option
- Update docs (en/zh) and tests to match new async rollback signatures

* fix: tests

* refactor(transaction): simplify session commit and add redo-based crash recovery

Session commit no longer wraps archive phase in a transaction. Phase 2 uses
redo semantics so crashed memory-extraction can be replayed from archive.
PathLock stale-lock cleanup no longer redundantly re-checks timeout.
Semantic processor vectorization runs concurrently via asyncio.gather.

* fix: transaction

* fix: UserIdentifier

* refactor(transaction): replace undo-based transaction manager with lightweight lock + redo-log

Remove the heavyweight TransactionManager/Journal/UndoEntry system (~4000 lines) and
replace it with a simpler architecture: LockManager for path locking, LockContext as
the async context manager, LockHandle/LockOwner protocol, and a RedoLog for crash
recovery of session_memory operations. VikingFS rm/mv now use inline error handling
instead of rollback semantics. Updated docs, observers, and tests accordingly.

Co-Authored-By: Claude Opus 4.6

* fix(transaction): remove checkpoint dead code, fix TOCTOU race, clarify mv lock param

- Remove unused _write_checkpoint/_write_checkpoint_async/_read_checkpoint
  from Session (superseded by redo-log)
- Re-resolve URI inside lock in resource_processor Phase 3.5 to prevent
  concurrent add_resource calls from resolving to the same final_uri
- Rename acquire_mv dst_path to dst_parent_path with docstring to clarify
  that callers pass the destination parent directory

* fix: path

* fix: resource lock

* fix: test

* docs: update

* fix: tests

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 14:50:19 +08:00
chuanbao666 84d206dd7a fix(agfs): refactor Makefile for cross-platform compatibility (#571)
* fix(agfs): refactor Makefile for cross-platform compatibility

* build: support python 3.14
2026-03-14 17:45:11 +08:00
qin-ptrandqin-ctx 4f32d3a8a7 [feat]: openclaw-memory-plugin: 全面改进内存插件功能 (#479)
* openclaw-memory-plugin: 改进内存插件功能

* Update README_CN.md

---------

Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-03-08 23:13:20 +08:00
Maojia ShengandMaojia Sheng 50323dfa6c fix: grep for binding-client (#474)
Co-authored-by: Maojia Sheng <shengmaojia@bytedance.com>
2026-03-08 00:59:12 +08:00
Qin HaojieandClaude Opus 4.6 ee2786cd98 feat(agfs): make binding client optional, add server bootstrap, tune logging and CI (#451)
- Make AGFSBindingClient import optional so pure-HTTP AGFSClient works without native lib
- Add lightweight server entry point that pre-parses --config before openviking imports
- Downgrade noisy info/exception logs to debug/warning level
- Replace FileNotFoundError with NotFoundError for consistent error handling
- Docker image build now only triggers on release tags and manual dispatch

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:25:35 +08:00
chuanbao666 8c740fd565 chore: 编译子命令失败报错, golang版本最低要求1.22+ (#444)
* chore: 编译子命令失败报错, golang版本最低要求1.22+

* fix: agfs默认启用binding-client相关改造
2026-03-05 20:52:38 +08:00
yangxinxin-7 42cced0f95 feat(agfs): add ripgrep-based grep acceleration and fix vimgrep line parser (#432)
* feat(agfs): add ripgrep-based grep acceleration and fix vimgrep line parser

- Add ripgrep fallback in Grep/GrepStream handlers when `rg` is available and the underlying FS is a local path resolver
- Expose `ResolvePath` on LocalFS to satisfy the `localPathResolver` interface
- Fix `parseRipgrepLine` to use the first `:line:col:` separator match instead of the last, so content containing `:digit:digit:` patterns is parsed correctly
- Add unit tests for `parseRipgrepLine`

* feat: fix tree -L n
2026-03-05 17:03:44 +08:00
MaojiaShengandopenviking c4209da6cc chore: downgrade golang version limit, update vlm version to seed 2.0 (#425)
* feat: define a system path for future deployment

* feat: define a system path for future deployment

* fix: golang downgrade to 1.19

* fix: golang downgrade to 1.19, and change doc

* docs: change model recommendation

* fix: golang downgrade to 1.19, and change doc

* fix: mv test files

* fix: loguru

---------

Co-authored-by: openviking <openviking@example.com>
2026-03-05 14:34:02 +08:00
MaojiaShengandopenviking e4b1ffed10 Feat: CLI optimization (#389)
* feat: remove python cli and disable python -m openviking

* feat: ls, tree, find, search, grep all use --node-limit as the result limiting arg

* feat: add ls -n

* feat: update agfs to support grep -n

* feat: update agfs to support grep -n

* docs: cancel modify

---------

Co-authored-by: openviking <openviking@example.com>
2026-03-03 13:23:37 +08:00
chuanbao666 e418870220 fix(agfs): 修复agfs binding-client安装问题, 清理agfs lib文件 (#337)
* fix(agfs): 修复agfs binding-client安装问题, 清理agfs lib文件

* fix(agfs): 暂时去掉文档说明
2026-02-27 23:32:55 +08:00
chuanbao666 4c9340fc10 feat(agfs): agfs新增binding client (#304)
* feat(agfs): agfs新增binding client

fix: viking_fs适配binding client, 简化agfs相关参数

* feat(agfs): agfs binding-client支持windows

* docs: 补充agfs binding-client用法

* fix(agfs): 默认带上agfs binding-client依赖库; setup.py支持编译该库

* fix(agfs): add agfs binding-client linux so

* fix(agfs): code format
2026-02-26 20:27:00 +08:00
Qin Haojie 6781df5090 fix/tos_vikingdb (#294) 2026-02-25 22:13:04 +08:00
SeanZ 5e44a0e7dd fix(storage): idempotent rm/mv operations with vector index sync (#284)
* fix(storage): idempotent rm/mv operations with vector index sync

- AGFS Server (Go): Use filesystem.NewNotFoundError() instead of
  fmt.Errorf() so mapErrorToStatus() returns 404 instead of 500
  - Affected plugins: localfs, memfs, queuefs, serverinfofs

- AGFS SDK (Python): rm() defaults to force=True (like rm -f)
  - 404 returns success instead of raising exception
  - Use AGFSHTTPError with status_code for HTTP errors

- VikingFS: rm() and mv() are now idempotent
  - Deleting non-existent file succeeds and cleans orphan index
  - mv() cleans all orphan indices when source not found (404)

This follows industry standards (Linux rm -f, AWS S3 DeleteObject)
where delete operations are idempotent by default.

* fix(agfs-server): unify 404 status for all file operations

Extend NewNotFoundError usage to Read, ReadDir, Open, and Grep
operations for consistent HTTP 404 responses across all plugins.

* fix(pyagfs): always use AGFSHTTPError to preserve status_code

When server returns JSON body with error message, the old code would
raise AGFSClientError (without status_code) instead of AGFSHTTPError.
This broke the force=True logic in rm() which checks status_code == 404.

Now all HTTP errors use AGFSHTTPError consistently, preserving the
status_code for callers that need to handle specific HTTP status codes.

Root cause: Original upstream code had this design flaw - JSON parsing
success path would lose status_code. This was not exposed before because
no code depended on status_code until we added force=True for rm().
2026-02-25 19:56:02 +08:00
Mingjian Queandmijamind719 498d560fd1 feat(vectordb): integrate KRL for ARM Kunpeng vector search optimization (#256)
- Add third_party/krl: Kunpeng Retrieval Library (KRL) source with ARM NEON/SIMD-optimized L2 and inner-product distance routines
- vector_base.h: add ARM platform macros OV_PLATFORM_ARM, OV_SIMD_NEON, OV_SIMD_SVE
- space_l2.h: on ARM use krl_L2sqr in l2_sqr_neon instead of scalar path
- space_ip.h: on ARM use krl_ipdis in inner_product_neon instead of scalar path
- CMakeLists.txt: enable OV_PLATFORM_ARM on aarch64, build and link KRL static library

On ARM, vectordb uses KRL-optimized paths; on x86 the existing AVX/SSE implementations are unchanged.

Co-authored-by: mijamind719 <mijamind@163.com>
2026-02-24 17:41:28 +08:00
baojun-zhang 66fdea27af fix: When the AGFS backend is s3, the error "pyagfs.exceptions.AGFSClientError: parent directory does not exist" occurs. (#254) 2026-02-23 07:27:08 +08:00
baojun-zhang 052535e0e1 feat: support tos oss (#255) 2026-02-23 07:25:46 +08:00
bot777andOpenClaw Integration 5b4a28fdc3 fix: invalid Go version format in agfs-server go.mod (#207)
go.mod specifies 'go 1.25.1' but Go only supports two-component
version directives (MAJOR.MINOR). The three-component format causes
'go mod tidy' and builds to fail with:

  go: invalid go version '1.25.1': must match format 1.N

This blocks pip install of OpenViking on any system that needs to
compile the AGFS server component.

Fix: Change 'go 1.25.1' to 'go 1.25' to match Go's version spec.

Co-authored-by: OpenClaw Integration <openclaw@example.com>
2026-02-19 09:23:03 +08:00
kkkwjx bbbdee90a0 fix: fix agfs-server for windows (#16) 2026-01-31 13:30:06 +08:00
kkkwjx 3acc6884fa Remove agfs (#13)
* feat: add github workflow ignore

* feat: remove agfs-server binary
2026-01-30 12:19:40 +08:00
qin-ctx f98dc0ed1c first commit 2026-01-29 20:29:19 +08:00