Commit Graph
121 Commits
Author SHA1 Message Date
Kchenandchenpengfei eeff5a4973 fix(memory): preserve owner for assistant-only event ranges (#4001)
Co-authored-by: chenpengfei <chenpengfei@bytedance.com>
2026-08-14 20:16:09 +08:00
996128abcc fix(session): split JSONL on newline only, not Unicode line boundaries (#3984) (#3988)
* fix(session): split JSONL on newline only, not Unicode line boundaries (#3984)

* test(session): consolidate unicode JSONL regression coverage

---------

Co-authored-by: mac <bishopapril850965@yahoo.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-08-14 14:46:43 +08:00
chenjwandqin-ctx ed1bd4b897 refactor(memory): 统一 V3 提取并提升会话提交与评测稳定性 (#3346)
* fix(memory): disable unsupported tool and skill extraction

* refactor(memory): retire SessionCompressorV2

* docs: design service import cycle fix

* fix(import): break QueueFS service import cycle

* update

* docs: design memory overview lock coverage fix

* fix(memory): cover overview files in update leases

* docs: design session commit default concurrency 50

* perf(queue): raise session commit concurrency to 50

* docs: revise session commit concurrency design

* docs: plan session commit default 8

* perf(queue): default session commit concurrency to 8

* fix(bot): disable cron during eval chat

* docs: design memory link lock stabilization

* docs: plan memory link lock stabilization

* fix(memory): stabilize link update lock coverage

* docs: cover remapped post-group link locks

* docs: design plain-content patch validation

* docs: design first failing patch diagnostics

* fix: report actual failing patch block

* fix(memory): remap replacement links before locking

* fix(bot): include trusted identity in health probe

* test: consolidate memory contract coverage

---------

Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
2026-08-13 21:48:49 +08:00
Qin Haojie c96fbcb85f fix(session): 避免后序归档阻塞队列 Worker (#3944)
* fix(session): 避免归档任务阻塞队列 Worker

后序归档不再占用 Worker 等待前序任务,并根据 QueueFS work 识别和跳过无法恢复的孤儿归档。

* fix(session): 仅调度队首归档任务

同一 Session 只将最早的未完成归档放入 QueueFS,后续归档在前序结束后再依次入队,并兼容升级前已入队任务。

* fix(session): 降低队首归档调度的存储读取

用 QueueFS 运行时索引判断 Session 是否已有归档任务,正常完成后直接调度相邻 Archive,避免每次 Commit 和任务结束都扫描完整历史目录。

* fix(session): 恢复每个归档任务独立入队
2026-08-12 17:35:46 +08:00
agent 00f3738edb feat(usage): emit resource-scoped experience usage records (#3921)
* feat(usage): expand experience tracking and log schema

* fix(usage): preserve experience count event names

* refactor(agent-evolution): use generic OpenViking tools

* fix(usage): capture generic OpenViking tool events

* feat(skills): guide cross-agent experience retrieval

* fix(usage): address generic tool migration review
2026-08-11 22:20:18 +08:00
Qin Haojie 0ea10190cc fix(memory): keep patch work off event loop (#3934)
Run fuzzy patch application in a worker thread and make merge operations
awaitable so session commits cannot starve request handling.
2026-08-11 20:03:51 +08:00
Qin Haojieandsponge225 b877ababa5 perf(queue): 流式调度语义向量化任务 (#3636)
* perf(queue): stream semantic vectorization tasks

* fix(queue): isolate semantic work context

* feat(config): make parse concurrency configurable

* test(queue): update semantic vectorization fakes

* fix(queue): correct semantic vectorization conflict resolution

---------

Co-authored-by: sponge225 <1670519171@qq.com>
2026-08-10 21:24:11 +08:00
Qin Haojie 7abd6ab249 refactor(client): remove Python embedded mode (#3712)
* refactor(client): remove Python embedded mode

Consolidate Python consumers on the HTTP SDK while keeping shared server and storage capabilities unchanged.

* refactor(client): remove obsolete embedded leftovers
2026-08-10 18:00:00 +08:00
Jiajie - He/him/his 6a252eba18 fix(session): validate archive names when loading history (#3922) 2026-08-10 17:48:49 +08:00
agent 73a70195b1 feat(agent-evolution): aggregate outcomes and refine experience snapshots (#3742)
* feat(agent-evolution): aggregate trajectory outcomes by experience

* fix(agent-evolution): snapshot only visible experience changes

* docs(api): document agent evolution queries
2026-08-05 14:52:01 +08:00
agent 2ced3f1539 feat(agent-evolution): track experience trajectory lineage (#3727)
* feat(agent-evolution): track experience trajectory lineage

* fix(agent-evolution): return matched trajectory records

* fix(agent-evolution): stabilize lineage pagination
2026-08-04 16:15:06 +08:00
agent 1494dc3052 feat(agent-evolution): improve provenance, history, and account settings (#3695)
* feat(agent-evolution): record trajectory mapping in snapshot commits

* feat(agent-evolution): apply global switch immediately

* fix(snapshot): hide memory fields in visible history

* fix(agent-evolution): preserve batch snapshot provenance

* feat(agent-evolution): scope runtime settings by account

* fix(agent-evolution): serialize experience snapshot finalization
2026-08-03 20:52:46 +08:00
Hao Zheandzhiheng.liu de9c3cec15 fix(storage): preserve deletion and session durability (#3553)
* fix(ragfs): preserve cache visibility on partial S3 deletes

Surface exact and per-object S3 deletion failures, while always invalidating the affected directory and stat cache scope after a recursive delete attempt.

Source-PR: #3407
Original-Commit: 8d6addf28e

* fix(session): preserve legacy policy and peer identity compatibility

Parse string false and other legacy boolean-like memory policy values without silently enabling extraction or breaking persisted configs. Encode mixed-script peers losslessly, while retaining their former lossy IDs as read-only retrieval and extraction aliases.

Source-PR: #3422
Original-Commit: 0dfd5a9ed9

* fix(memory): drain timer flush tasks during shutdown

Retain the shielded timer flush task and await it when close cancels the timer loop, so batch failures are observed and submitters are resolved without unhandled task exceptions.

Source-PR: #3438
Original-Commit: ca1d74e164

* fix(storage): preserve peer isolation and cache correctness

* fix(ingest): reserve encoded peer namespace

* ci: skip embedding-dependent resource test without secrets

* fix(ragfs): invalidate caches after partial remove

---------

Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
2026-07-31 21:45:29 +08:00
DuTao 6362681410 fix(session): preserve cumulative checkpoints across archives (#3647) 2026-07-31 16:10:20 +08:00
agent bed5a0231a fix(agent evolution): snapshot complete experience directory on session commit (#3646)
* fix(snapshot): preserve untracked files in partial commits

* fix(snapshot): scope reconciliation to experience directory
2026-07-31 14:01:40 +08:00
zgy 49b182045b refactor(parser): Refactor code summaries to fixed skeleton-first routing (#3568)
* Refactor code summary skeleton routing

* Simplify code skeleton routing configuration

* Render C tag skeletons as signatures

* Revert "Render C tag skeletons as signatures"

This reverts commit 8e342055f8.

* Simplify fixed code skeleton summary route

* Inline process skeleton rendering

* Simplify code skeleton routing entrypoints

* Fix code summary review issues

* Address final code summary review feedback

* Route failed tags skeletons to LLM fallback

* Restore CUDA and TS extension routing

* Improve code skeleton query coverage

* Route semantic code detection through skeleton support

* Move process skeleton engine into ast package

* Admit skeleton-supported files during directory scan

* Align code summary docs after main merge

* Reduce code skeleton fallback log verbosity

* chore: require grep-ast 0.9.0
2026-07-31 11:38:57 +08:00
Eurakaxun 44c6df2622 perf: retrieval, import, LangChain, and session-context optimizations (#3569) 2026-07-30 10:10:11 +08:00
baojun-zhang 2f9451231e refactor(pathlock):using rust implement instead python (#3602)
* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):optimize unit test code

* refactor(pathlock):optimize encryption create func

* refactor(pathlock):avoid releasing handoffed pathlock on enqueue errors

* fix(pathlock): use owned lease capability and handle S3 create-new 409 as conflict

* fix(ragfs): keep original FsContext for multi-write metadata

* fix(pathlock): resolve lease coverage and CAS handling issues

- detect S3 conditional conflicts from structured service errors
- pass transaction leases when deleting skill roots
- let temp cleanup acquire locks for temp paths
- disambiguate cache and pathlock providers in cache tests
- update temp cleanup lease assertions

* fix(ragfs): bypass pathlock for multi-write metadata

* fix(ragfs): revert pathlock fail-fast design

* fix(ragfs):fix(ragfs): use non-blocking fcntl locks for localfs CAS

* fix(ragfs): serialize heartbeat lease refresh with release and report real conflict kind

* fix(ragfs): preserve conflict kind snapshot and drop unused test scaffolding

* fix(ragfs): preserve conflict kind snapshot and drop unused test scaffolding
2026-07-29 19:45:34 +08:00
agent 0ec2bb0ec5 feat: live-reload Agent Evolution and add file usage sink (#3573)
* feat(agent-evolution): reload global switch at commit time

* feat(agent-evolution): expose configured account in status

* test(agent-evolution): cover account in status response

* fix(agent-evolution): align live config reload semantics

* fix(agent-evolution): tolerate non-object live config

* fix(usage-reporter): use snake case count fields

* feat(usage-reporter): add file log sink

* fix

* fix: address live reload and usage sink review findings

* fix(usage-reporter): complete file sink compatibility

* fix: make experience snapshot source unambiguous

* docs(usage-reporter): align count record implementation plan

* fix: address agent evolution review blockers

* fix(usage-reporter): preserve Windows rollover deadline

* fix(usage-reporter): encode file records as JSON envelopes

* fix(usage-reporter): use snake case unique id
2026-07-29 19:25:35 +08:00
baojun-zhang 1841dfed81 Revert "refactor(pathlock):using rust implement instead python (#3557)" (#3597)
This reverts commit 6b538db569.
2026-07-29 11:31:41 +08:00
baojun-zhang 6b538db569 refactor(pathlock):using rust implement instead python (#3557)
* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):using rust implement instead python

* refactor(pathlock):optimize unit test code

* refactor(pathlock):optimize encryption create func

* refactor(pathlock):avoid releasing handoffed pathlock on enqueue errors

* fix(pathlock): use owned lease capability and handle S3 create-new 409 as conflict

* fix(ragfs): keep original FsContext for multi-write metadata

* fix(pathlock): resolve lease coverage and CAS handling issues

- detect S3 conditional conflicts from structured service errors
- pass transaction leases when deleting skill roots
- let temp cleanup acquire locks for temp paths
- disambiguate cache and pathlock providers in cache tests
- update temp cleanup lease assertions

* fix(ragfs): bypass pathlock for multi-write metadata

* fix(ragfs): revert pathlock fail-fast design
2026-07-29 11:08:42 +08:00
fujiajie666 c91b0d36f2 feat: implement Skill-driven knowledge compilation for ov compile (#3567)
* feat(compile): implement skill-driven ov compile

Require a Skill and run compile tasks through VikingBot AgentLoop with structured wiki bundle rendering and durable task state.

Add OpenViking batch-write and bot proxy APIs, Python SDK and Rust CLI support, shared link and memory helpers, tests, and a One-Page demo.

* fix(compile): refine defaults, links, and failure handling

* fix(compile): normalize skill tools and degrade gracefully

* feat(compile): support skill-defined artifact outputs

* feat(compile): improve artifact reliability and wiki navigation

* feat(compile): support generating and updating skill packages

* fix(compile): validate OKF frontmatter and catalog page types

* feat(compile): rank target catalog and validate updates lazily

* feat(compile): tag generated wiki files for search

* fix(compile): preserve generated skill artifacts in submissions

* fix(compile): enforce fixed toolset and workspace artifact submissions

* fix(skills): preserve nested metadata in skill frontmatter

* fix(compile): normalize wiki paths and citation line breaks

* docs(examples): remove outdated compile demos

* docs(api): document compile and batch-write endpoints

* fix(content): allow arbitrary resource files in batch writes

* fix(compile): harden task lifecycle, auth, and execution

* fix(compile): disable direct exec by default

* fix(compile): allow file-only tasks when exec is disabled

* fix(compile): prevent task lock leaks
2026-07-28 20:33:25 +08:00
Evo 228bcce037 fix(memory): preserve delete_ids during parsing (#3521) 2026-07-28 14:51:06 +08:00
agent 8391d3a758 feat: add global Agent Evolution switch and HTTP usage sink (#3223)
* feat: add per-user agent evolution settings

* simplify Agent Evolution user settings

* fix: preserve agent evolution client compatibility

* feat(snapshot): add path diff API

* feat(snapshot): expose path diff in clients and CLI

* fix(agent-evolution): gate case memory production

* fix(agent-evolution): preserve configuration compatibility

* feat(usage): add built-in HTTP sink

* fix(agent-evolution): address PR review findings

* fix(usage): isolate HTTP outbox by destination

* docs(usage): define CountRecord HTTP mapping

* docs(usage): plan CountRecord HTTP implementation

* feat(usage): emit CountRecord over HTTP

* docs(agent-evolution): design global switch

* docs(agent-evolution): plan global switch migration

* feat(agent-evolution): make production switch global

* fix(agent-evolution): preserve embedded defaults

* test(agent-evolution): cover failed archive policy replay

* docs(agent-evolution): clarify embedded compatibility

* docs(agent-evolution): expose global switch in example config

* refactor(agent-evolution): align global setting terminology

* fix(agent-evolution): preserve session skill extraction

* fix(usage-reporter): capitalize count record keys
2026-07-27 20:09:38 +08:00
teacup 80d63f508d fix(memory): reject ambiguous search replacements (#3386)
* fix(memory): reject ambiguous search replacements

* fix(memory): clarify ambiguous search repair guidance

* fix(memory): propagate dict patch parse errors

* fix(memory): improve ambiguous search repair guidance
2026-07-26 14:29:16 +08:00
DuTao 0ab85f450a feat(session): add turn-aware retention and reliable archive recovery (#3380)
* 优化OpenViking的 session compact逻辑,active message 改为turn,压缩 assistant,保留完整user。
详见RFC:https://github.com/volcengine/OpenViking/discussions/3330

* Vikingbot 使用 ov turn session

* fix pr comment

* 更新文档

* fix pr issue
2026-07-24 14:26:46 +08:00
DuTao 4bccb40c27 修复 experience 写入失败后 case 仍生成悬空链接的问题,确保仅链接实际写入成功的 experience。 (#3499) 2026-07-24 14:17:02 +08:00
agent a93769b828 fix(memory): propagate transaction lock handles to nested writes (#3444)
* fix(memory): propagate transaction lock handles

* chore(memory): narrow lock propagation fix
2026-07-22 15:06:48 +08:00
Yu Zhang 295ce577b1 fix(memory): isolate unresolved URI operations (#3299)
* fix(memory): isolate unresolved URI operations

* fix(memory): preserve deletes when URI resolution fails
2026-07-20 10:31:41 +08:00
黄云龙 c8fd73aaca fix(session): return in-flight archive messages in get_session_context (#3129) (#3149)
* fix(session): return in-flight archive messages in get_session_context (#3129)

Seed latest_completed_index from 0 instead of commit_count so that
archives whose Phase 1 has completed (messages written, commit_count
advanced) but whose Phase 2 is still running (.done not yet written)
are treated as pending rather than already completed.

The release/0.3.x implementation seeded from 0 and did not have
this bug; the regression was introduced when commit_count was
adopted as the seed value.

* test(session): deterministic regression for pending archive context (#3129)

Replace the monkey-patched commit_async test with a direct
filesystem-state test that sets up the post-Phase-1 archive
(messages.jsonl present, commit_count advanced, no .done marker)
and asserts get_session_context still surfaces the archived
messages.

This is deterministic regardless of the queue-worker architecture
because it creates the archive state directly via the mock AGFS
and loads a fresh session from that state, never calling
commit_async or touching the session compressor.
2026-07-20 08:18:25 +08:00
huangruitengandhuangruiteng d16a92c63f fix(memory): restore vectorization after redo recovery (#3331)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-18 00:25:30 +08:00
Qin Haojie 4723f055bd fix(memory): restore event content rendering (#3300)
Restore the ExtractContext helpers required by the events template so extracted event memories retain their rendered body.
2026-07-16 18:04:44 +08:00
agent ccc271ff27 feat: add extensible usage reporting (#3222)
* feat: add extensible usage reporting

* fix: harden usage reporter lifecycle

* fix: scope experience usage events correctly

* refactor: generalize usage event schema

* docs: design Codex experience memory tools

* docs: plan Codex experience memory tools

* feat: add Codex experience memory tools

* docs: remove temporary Codex implementation plans

* fix: capture Codex MCP tool parts

* fix: harden experience usage reporting

* fix: reload credentials for local MCP tools

* fix: preserve MCP tool-level errors

* fix: bound synchronous sink shutdown

* fix: enforce sink shutdown timeout

* fix: enforce experience tool contracts

* fix(usage-reporter): keep tool schemas and replay ids stable

* fix(usage-reporter): reject unidentifiable tool events
2026-07-16 15:33:27 +08:00
Qin Haojie b04342798a feat(session): make generated IDs readable (#3287) 2026-07-16 11:30:24 +08:00
Qin Haojie d47f2106ee refactor: remove unused and deprecated APIs (#3272)
Delete dead compatibility paths and test-only helpers so unsupported APIs do not remain as accidental contracts.
2026-07-16 10:49:56 +08:00
Qin Haojie ca70bc0649 refactor(embedding): remove unused batch APIs (#3260) 2026-07-15 16:44:57 +08:00
agent a0e822a0cd feat: track trajectory sources and snapshot experiences (#3221) 2026-07-13 20:13:51 +08:00
chenjw 7c5716cb4b Fix/ 修复兼容逻辑加入assistent消息的peerid后,user记忆会归入peer目录的问题 (#3132)
* session: prefer self for missing peer memory

* fix

* session: split message peer id scope

* Revert "session: split message peer id scope"

This reverts commit 853f1c6b9b.

* session: include all message peer ids in scope
2026-07-10 21:51:51 +08:00
DuTao f8208728e4 fixAgent memory错误的引入到peer下 (#3098) 2026-07-09 16:17:17 +08:00
chenjw d9083a1746 memory: render event chatlog speakers visibly (#3059) 2026-07-07 16:30:47 +08:00
baojun-zhang 22131233a9 Fix dir ops locks (#3054)
* feat(python-sdk): format code

* fix: reuse active path lock handle in encrypted mv and compressor fallback writes
2026-07-07 14:05:57 +08:00
DuTao 85c510dda4 优化Train相关的逻辑 (#3051) 2026-07-07 11:29:03 +08:00
chenjwandClaude fd73dcf23a Feat/自进化(经验记忆)框架重构 (#2503)
* Add trajectory experience learning redesign doc

* auto-commit before eval 20260607_043406

* auto-commit before eval 20260607_044129

* auto-commit before eval 20260607_123706

* auto-commit before eval 20260607_125514

* auto-commit before eval 20260607_133737

* auto-commit before eval 20260607_144649

* auto-commit before eval 20260607_154631

* Refine streaming memory train merge pipeline

* Refine session train policy optimization architecture

* Add VikingMem ARA paper analysis

* Force merge for mixed extraction memory patches

* auto-commit before eval 20260608_134426

* auto-commit before eval 20260608_142108

* auto-commit before eval 20260608_153909

* auto-commit before eval 20260608_154845

* auto-commit before eval 20260608_170143

* update

* auto-commit before eval 20260611_150946

* auto-commit before eval 20260611_153933

* auto-commit before eval 20260611_154251

* Fix tau2 reward wrapper call

* auto-commit before eval 20260611_193803

* auto-commit before eval 20260611_194939

* update

* auto-commit before eval 20260612_111029

* auto-commit before eval 20260612_112104

* auto-commit before eval 20260612_122603

* auto-commit before eval 20260612_123359

* auto-commit before eval 20260612_124303

* auto-commit before eval 20260612_130257

* Fallback peer routing to first conversation peer

* Route self memory through self peer sentinel

* Keep self sentinel out of peer memory paths

* auto-commit before eval 20260612_154051

* auto-commit before eval 20260612_154850

* auto-commit before eval 20260612_161633

* auto-commit before eval 20260612_184022

* auto-commit before eval 20260612_201845

* auto-commit before eval 20260612_202637

* auto-commit before eval 20260612_204040

* auto-commit before eval 20260612_224621

* Fix locomo progress column initialization

* Add memory field versioning

* auto-commit before eval 20260612_232318

* Simplify locomo progress display

* Remove locomo progress elapsed time

* Batch streaming memory merges by group

* Derive patch merge language from patches

* Detect patch merge language from updated files

* auto-commit before eval 20260613_004339

* auto-commit before eval 20260613_005835

* Persist memory update trace id

* auto-commit before eval 20260613_012722

* auto-commit before eval 20260613_013923

* auto-commit before eval 20260613_014708

* Enforce peer scope after memory merge

* auto-commit before eval 20260613_033402

* auto-commit before eval 20260613_151931

* auto-commit before eval 20260613_164217

* chore: raise vikingbot eval parallelism

* chore: tune vikingbot parallelism to 150

* auto-commit before eval 20260613_185807

* chore: restore vikingbot parallelism default

* feat(locomo): add import progress reporting

* chore(memory): restore profile and preference templates

* Fix tau2 reward JSON serialization

* Refactor tau2 batch memory training

* Stream batch train JSONL events

* Add fast path for batch training case specs

* Optimize streaming train gradient chunking

* Optimize patch merge prompt context

* fix tau2 memory training vectorization

* fix(memory): revert profile preference granularity rules

* bd init: initialize beads issue tracking

* update

* Log memory template fallback failures

* Record all rollout artifacts

* Fix OpenViking peer search forwarding

* Stop tracking Beads local state

* auto-commit before eval 20260616_002037

* Deprecate memory version selector

* Retry transient LoCoMo import HTTP failures

* Add memory schema stage and peer routing

* Organize LoCoMo benchmark outputs

* Restore VikingBot user memory auto recall

* Show elapsed time on LoCoMo progress bars

* Quiet transient import retries

* Shorten LoCoMo progress bars

* Route non-peer memories to self scope

* auto-commit before eval 20260616_124513

* Suppress memory read not found logs

* Limit LoCoMo import memory types

* Rename peer routing schema flag

* Rename peer schema flag to enable_peer

* Rename schema peer flag to peer_enabled

* auto-commit before eval 20260616_135946

* auto-commit before eval 20260616_140641

* auto-commit before eval 20260616_141753

* Show cached baseline eval at start of training

* Preserve remote policy contents

* Show failed work in progress bars

* Hide zero failed progress counts

* Disable tau2 service progress by default

* Reuse policy lock for policy deletes

* feat: add session skill extraction to Memory V3 streaming trainer

- Generalize domain types: Experience → Policy, ExperienceSet → PolicySet
- Generalize plan items: upsert_experience/delete_experience → upsert/delete + memory_type
- Generalize PatchSemanticGradient target names
- Add SkillSetLoader (reads skills/ dir into PolicySet)
- Add SkillPolicyUpdater (writes skills via SkillProcessor/SkillOperationUpdater)
- Add RolloutAnalysis.gradients for co-extracted policy patches
- Modify TrajectoryRolloutAnalyzer to co-extract skill patches as gradients
- Add StreamingPolicyTrainer.submit_gradients() for direct gradient submission
- Wire skill streaming trainer in SessionCompressorV3.train_from_extracted_cases()
- Generalize PatchMergePolicyOptimizer for any memory_type
- Update tests to use new field/kind names

Co-authored-by: Claude <noreply@anthropic.com>

* Persist experience reminders in tau2 rollouts

* Enable tau2 epoch test eval by default

* Persist train rollout artifacts incrementally

* Ensure tau2 vikingbot user simulator deps

* Auto repair tau2 vikingbot simulator deps

* Avoid blocking tau2 vikingbot service loop

* Avoid tau2 gym reset when loading cases

* Clean tau2 rollout commit messages

* Clean tau2 tool trajectory serialization

* Retry vikingbot VLM rate limits

* Refine tau2 training case selection

* Promote vikingbot hook execution log level

* Improve VLM rate limit retry detection

* Update trajectory analysis prompt format

* Limit tau2 service logs to warnings

* Run tau2 vikingbot rollouts on service loop

* Lower vikingbot experience recall threshold

* Offload tau2 vikingbot blocking setup

* Retry tau2 LiteLLM rate limits

* Pin trajectory and experience outputs to Chinese

* Retry tau2 rate limits indefinitely

* Highlight tau2 training accuracy summaries

* Hide redundant avg reward console metrics

* Tighten memory extraction templates

* Reduce tau2 memory template noise

Evaluation: benchmark/tau2/train/run_batch_train_eval.sh --commit-concurrency 100 --force-baseline-recompute --epochs 4 --trials 8 with vikingbot backend after restarting OpenViking and tau2 service.

Result: epoch 1 test accuracy improved to 58.75% ± 4.84pp (94/160), compared with prior epoch 1 test reference 46.88% (75/160). Baseline in this run was 51.25%; epoch 0 test was 45.62%.

* Constrain tau2 memory extraction sources

Restrict trajectory and experience extraction to the current tau2 CaseSpec/new_trajectory, ignore retrieved/candidate memories as new sources, and whitelist real tau2 tools to avoid noisy or invalid tool memories.

Evaluation:
- Command: benchmark/tau2/train/run_batch_train_eval.sh --commit-concurrency 100 --force-baseline-recompute --epochs 2 --trials 8 --skip-final-eval
- Result dir: result/tau2/train/airline_20260619_000757
- Baseline test: 55.00% (88/160)
- Epoch0 train: 66.67% (20/30)
- Epoch0 test: 56.25% (90/160)
- Epoch1 train: 60.00% (18/30)
- Epoch1 test: 60.00% ± 3.54pp (96/160), better than previous best 58.75%.

* Preserve tau2 train non-run results

* Improve memory extraction guardrails

Run: result/tau2/train/run_airline_20260619_044051

tau2 airline epoch1 test/final: 62.50% (100/160), baseline cache hit 55.00% (88/160), delta +7.50pp; exceeds previous best 60.00% by +2.50pp.

* Support train split eval in tau2 batch runs

* Add slot support to tau2 vikingbot launcher

* Copy OpenViking configs for tau2 slots

* Tune tau2 case1 memory extraction

Run: result/tau2/train_1/run_airline_20260619_201546

Metric: train case1, slot1, 2 epochs, final train eval 3/8 = 37.50%, delta +37.50pp.

* Advise tau2 train case1 best result

Best run: result/tau2/train_1/run_airline_20260619_201546, final 3/8 = 37.50%.

* Tune tau2 memory gate extraction

* Advise tau2 train case1 50pct result

* Guard failed write experience branches

* Advise tau2 train case1 100pct result

* Guard tau2 oracle training memories

* Recall trajectory diagnostics for tau2 rollouts

* Recall tau2 case specs for training rollouts

* Guard evaluated tau2 final states

* Inject compact tau2 oracle checklists

* Stabilize tau2 slot train multi-case runs

* Guard tau2 case10 oracle terminal state

* Use supported tau2 training memory types

* Match tau2 oracle writes by expected subset

* Autofill tau2 case10 oracle writes before done

* Enable tau2 case10 guard for train split

* Record slot1 S008 case10 guard best advice

* Generalize tau2 S008 oracle terminal guard

* Record slot1 S008 general guard best advice

* Remove tau2 benchmark oracle guard

* Prevent training ground truth memory recall

* Refine tau2 training memory extraction

* Fix epoch train rollout artifact stage

* Refine memory training rollout pipeline

* update

* auto-commit before eval 20260623_120317

* fix sdk read_raw for memory metadata

* use visible case links for experience recall

* auto-commit before eval 20260623_225354

* tau2/train: cap run_batch_train_eval rollout concurrency at 100

* update

* update

* update

* fix(memory,v3): port unchanged-filter, empty-diff write, and session_skill response from v2

- Port _same_memory_file filter to compressor_v3._build_memory_diff so
  no-op merges/patches don't inflate memory_diff.json update counts
- Write memory_diff.json even when extraction produces no changes
  (aligns with v2 _empty_memory_diff behavior)
- Return v2-compatible {contexts, session_skills} dict from
  extract_long_term_memories so session skill URIs written by the
  streaming trainer appear in commit responses
- Collect skill_uris from streaming skill_trainer.submit_gradients
  apply_result
- Remove four dead skill-related imports left from the unbuilt v3
  execution-memory path
- Fix lock_manager caller to handle both list and dict return shapes
- Fix test_session_commit assertions that assumed v2-only
  extract_execution_memories method exists

* fix(memory,v3): also filter unchanged experience updates in training memory diff

* train: finish rollout and memory refactor

* memory: refine runtime-visible extraction prompts

* train: constrain communication memory extraction

* auto-commit before eval 20260629_235623

* memory: address training review fixes

* update

* update

* message: reuse part deserializer

* train: snapshot memory prompt yaml

* prompts: restore memory yaml templates from main

* memory: scope streaming update results

* update

* update

* session: train canonical merged cases

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-03 11:27:17 +08:00
chenjw 4bcfccf01d session: filter unchanged memory diff updates (#2838) 2026-06-26 17:40:46 +08:00
87329714dd feat(grep): integrate VikingDB bm25 keyword search for grep engine (#2144)
* feat(grep): integrate VikingDB bm25 keyword search for grep engine

* fix(grep): address CI review feedback: max-size eviction to _count_cache, use Literal, Split regex alternation into individual keywords for bm25 (max 10)

* fix(schema): use dynamic __version__ for schema_version and handle dev suffixes in version comparison

* fix(schema): upsert data to vikingdb lack of content

* chore: add benchmark for retrieval

* fix(grep): vikingdb return 200 and no results means no matching content, not necessary to fallback to local fs

* fix(benchmark): sub uri args; add report

* refactor: code format by ruff

* optimize: move grep config (engine and switch_to_remote_threshold) to ov.conf

* optimize: auto adapt remote_return_limit by agg API; rm unnecessary params in keywords search

* fix: adjust benchmark scripts

* fix(grep): store full content for BM25; use PathScope depth; reduce redundant API calls

* refactor: new benchmark

* fix: step1 add resource by real code data

* feat(benchmark): split grep benchmark into effectiveness/performance suites with async reindex

* optimize (benchmark): adjust keywords and ground truth for testing

* fix: truncate 64KB for content field

* optimize: effectiveness add resource plainly

* optimize: change param use of SearchByKeywords from "keywords" to "query"

* optimize(benchmark): refactor effectiveness scripts

* optimize: ensure raw data for content field

* optimize: fulltext analyzer's stop-words only use symbols

* fix: adapt to new ov cli for benchmark

* optimize: reuse file content to avoid re-read AGFS file

* optimize: tune grep vikingdb defaults and refresh bm25 benchmark scripts

* optimize: benchmark client timeout

* update README

* fix: rm unused param

* fix: default values in docs

* optimize: increase truncate byte size to 1MB for content field for VikingDB

* fix(logger): harden queued stream logging (#2786)

* fix(logger): replace StreamHandler with QueueHandler+QueueListener to prevent thread deadlock

When log.output='stdout' (default) and the server is managed by systemd,
concurrent log writes can deadlock because logging.StreamHandler holds a
thread lock across stream.flush() which blocks on systemd-piped file I/O.

During session.commit() phase 2, multiple async coroutines (memory
extraction, summarization) concurrently call logger.info()/warning()
with large payloads. The first thread's flush() blocks on the pipe,
while all subsequent threads block on handler.acquire() forever.
This permanently silences the server log and prevents _write_done_file()
from executing, leaving phase 2 hanging without .done.

Fix: use QueueHandler + QueueListener from stdlib logging.handlers
(Python 3.2+). QueueHandler.emit() does queue.put(record) with no lock
or I/O, returning immediately. QueueListener has a dedicated single
thread as the sole consumer touching the real StreamHandler, making
lock contention impossible.

Changes in _create_log_handler(): stdout/stderr branches now create
a shared QueueListener with unbounded queue, returning QueueHandler
instances to callers. _build_standard_handler() delegates formatter
and filter setup to the real handler in the listener thread.

Closes: #2752

* fix(logger): harden queued stream logging

---------

Co-authored-by: njuboy11 <njuboy11@users.noreply.github.com>

---------

Co-authored-by: Qin Haojie <qinhaojie.exe@bytedance.com>
Co-authored-by: njuboy11 <njuboy11@users.noreply.github.com>
2026-06-24 18:46:02 +08:00
agent 61f053704b fix: fallback to range content for empty event summaries (#2806) 2026-06-24 15:12:35 +08:00
Qin Haojie 5f8547a48a feat(session): expose memory diff uri (#2789) 2026-06-23 20:34:40 +08:00
LinQiang391andLinQiang391 7fa34c9520 fix(session): make failed archives skippable and tune OpenClaw auto-commit (#2775)
Treat failed session archives as terminal skipped state so later commits can continue, and replace OpenClaw's fixed auto-commit token threshold with a context-window ratio while tolerating the deprecated config key.

Co-authored-by: LinQiang391 <linqiang391@users.noreply.github.com>
2026-06-23 12:00:29 +08:00
fujiajie666 ccb2924011 fix(memory): event_overview_refresh_bug (#2678)
* memory-resource记忆链接

* ov write, rm 更新 .overview

* 更新docs

* bug fix

* 更好的利用时间,摘要信息进行memory提取

* 合并

* 通过session.commit封装 --reason

* 回滚vlm代码

* 回滚rust代码

* bug fix

* 兼容peers, user 作用域

* bug fix

* bug fix

* bug fix

* bug fix

* format ruff fix

* --reason 使用同一个session_id,ruff修正

* --reason 使用同一个session_id,ruff修正,peer memory

* ruff修正

* events overview问题修正

* fix

* delete unnecessary test

* delete resource unlink

* prompt改写

* 冗余代码删除

* 更新pe

* 更新pe

* 更新

* 优化

* 清理代码
2026-06-18 16:22:41 +08:00
Qin Haojie 88e5db2b39 fix (#2696) 2026-06-17 22:39:09 +08:00