mirror of
https://github.com/volcengine/OpenViking.git
synced 2026-09-28 11:43:00 +08:00
chenjwandTRAE CLI
a09a9d20a8
* feat(compile): add memory consolidation mode via `--skill memory` Add an in-process memory consolidation mode to `ov compile`. When `--skill` is the sentinel value `memory`, CompileService runs the existing memory framework (ConsolidationExtractContextProvider -> ExtractLoop -> MemoryUpdater) directly inside OpenViking core to dedup/merge/split/compact an existing memory-type directory in place, conforming to that type's schema. The default skill path (VikingBot Wiki compile) is unchanged. Highlights: - New ConsolidationExtractContextProvider: agentic exploration with ls/search/read tools seeded by a recursive listing; single schema inferred from --to; space (self/peer) taken from the canonical --to URI so listing never depends on an empty ctx.user_id. - MemoryCompileRunner: session.commit-lite task shape (task_tracker + one in-process asyncio.Task, no QueueFS re-delivery), bound to a root span so a trace_id is recorded; result reports adds/updates/deletes (file URIs only, memory_diff.json semantics) classified via read_file_contents. - MemoryLsTool: add recursive listing (relative paths, 500-node cap with truncation) and stop hiding subdirectories so subfoldered dirs are not misreported as empty. Only the compile provider exposes ls, so session.commit is unaffected. - CLI: --from optional (required for normal mode, rejected for memory mode); help gains a memory example. - Fix a syntax regression in crates/ragfs/src/lock/provider.rs test module that blocked `make build` (unrelated to compile; introduced by #4908). - Docs: document the memory mode in ov-compile-design.md. - Tests: unit tests for provider/runner/request validation; integration script test_compile_memory_xiaomei.py with merge/split/dedup/preferences cases. Co-authored-by: TRAE CLI <traecli@bytedance.com> * docs(compile): document memory consolidation mode Add a dedicated user-facing page (zh + en) for `ov compile --skill memory` covering when to use it, usage, parameters, behavior, and the adds/updates/ deletes result. Link it from the context-compilation overview. The VitePress sidebar picks the new page up automatically from the directory listing. Co-authored-by: TRAE CLI <traecli@bytedance.com> * fix(compile): honor language and cancellation Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: migrate files when URI fields change Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: materialize URI moves at apply time Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: inherit source links on explicit merge Co-authored-by: TRAE CLI <traecli@bytedance.com> * test(memory): cover rename conflict and streaming migration Co-authored-by: TRAE CLI <traecli@bytedance.com> * test(compile): assert URI migration diff semantics Co-authored-by: TRAE CLI <traecli@bytedance.com> * docs(compile): clarify cross-type link migration Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: fail rename on target read errors Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: preserve omitted URI identity fields Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: rebase URI moves on latest source Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: protect occupied empty rename targets Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: reject case-only URI moves Co-authored-by: TRAE CLI <traecli@bytedance.com> * docs(compile): document URI migration semantics Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: validate explicit replacement targets Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: reject case-only replacement moves Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: rerender managed links after URI moves Co-authored-by: TRAE CLI <traecli@bytedance.com> * test(compile): add repeatable Chinese URI rename case Co-authored-by: TRAE CLI <traecli@bytedance.com> * test(compile): cover bidirectional URI renames Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: remove empty directories after URI moves Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory(prompts): lowercase filename identity segments Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: consolidate memories-root in one extract loop Co-authored-by: TRAE CLI <traecli@bytedance.com> * docs(compile): document memory-root consolidation and failure semantics Co-authored-by: TRAE CLI <traecli@bytedance.com> * test(compile): add memory_root and deterministic rename cases Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory(compile): honor Account templates and preserve merged duplicate links - MemoryCompileRunner._consolidate now resolves the Account-level memory template snapshot via resolve_account_memory_registry(), matching the session.commit extract path, so consolidation no longer overwrites operator-customized content templates with deployment defaults. - _inherit_deleted_link_relations now tracks the deleted source URI for each inherited link. Implicit rename targets only exclude contributions copied from their own migration source, so links unique to a duplicate merged into the same target (delete_replacements) are folded in and the neighbor backlinks match. - Regression tests cover the Account-template snapshot and a same-batch rename+merge where only the duplicate holds a link to a third file. Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: preserve rename sources through session commit queue - ExtractLoop._updated_uri_for_existing_operation only considers an identity-field change a rename when the merged value actually differs from the current one, and returns the source URI unchanged when the regenerated candidate matches it. Legacy paths whose new template differs only in case (e.g. preferences user "Alice") no longer trip the case-only migration guard on plain content updates. - clone_operation_for_uri no longer drops old_memory_file_content when the target URI differs from the source. The upstream _materialize_uri_migrations still detects the mismatch and generates a write-new + delete-old migration, but the queue clone keeps the source content so the migration can inherit prior body and links instead of turning a rename into an empty new record. - python_protocol contract preamble drops the misleading "Unknown business fields are ignored" clause; unknown fields raise at parse time, so the note was inaccurate. - Regression tests cover legacy case-only preferences updates going through ExtractLoop and split_request_by_merge_group preserving the rename source for the add + delete pair. Co-authored-by: TRAE CLI <traecli@bytedance.com> * vikingbot(cli): pin openviking log stream to stderr before eager imports vikingbot chat --eval / -e commits to a stdout JSON contract. Downstream consumers like benchmark/locomo/vikingbot/run_eval.py parse stdout with json.loads, so any log line on stdout breaks the parse and drops token_usage and iteration to defaults (0). That is what caused Total prompt tokens=0 and Avg iteration=0 in the LoCoMo summary. Move the openviking / openviking_cli log redirection into a module-level _preimport_redirect_openviking_logs_to_stderr() that runs before any vikingbot.agent.* or openviking.* imports. Those imports call get_logger() at module load time, which loads ov.conf and can emit warnings (e.g. "Ignoring unknown config field") through openviking_cli's shared QueueListener whose default output is stdout. Force the "stdout" listener plus real StreamHandler pair into existence up front and rebind its stream to stderr. Also move the in-chat() redirect ahead of ensure_config and extend it to swap the shared stdout handler as a second-line guard. Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory(python-protocol): silently drop unknown-field edits Extraction DSL programs occasionally reference a field name that does not exist in the memory schema. Failing the whole program on this is brittle: unrelated valid statements in the same commit are lost. Match the tolerance kwargs already have on create()/set()/update() and treat unknown-field attribute access as a compile-time no-op: return a _FieldHandle flagged is_noop=True, and skip any .update()/.edit()/.drop() chained on it plus the final _apply_field_handle. Sibling operations on real fields keep applying. Update the corresponding regression tests: replace the literal-`field` placeholder rejection test with two new cases asserting the whole program still commits and a real content.edit() still lands when a bogus field appears in the same batch. Co-authored-by: TRAE CLI <traecli@bytedance.com> * memory: tree-lock delete parents so overview cleanup covers empty dir rm `generate_overview` recursively removes a memory directory once the last file is deleted, but `_operation_tree_lock_paths` only tree-locked rename source parents. Batches that plain-deleted the last file in a directory ran the follow-up `rm -r` under a lease that did not cover the parent, and RAGFS rejected the request with "pathlock lease ref does not cover the requested operation". Add each delete's parent directory (unless a same-directory rename replaces it) to the tree-lock set. Co-authored-by: TRAE CLI <traecli@bytedance.com> --------- Co-authored-by: TRAE CLI <traecli@bytedance.com>