mirror of
https://github.com/volcengine/OpenViking.git
synced 2026-10-01 17:57:49 +08:00
python-sdk@0.1.7
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Haoyu ZhangandQin Haojie
|
3f3554256b |
feat: 支持基于火山方舟的音视频多模态理解 (#3563)
* feat: add audio and video understanding via VLM * docs: design media resource guards * fix: bound media staging concurrency * fix: cap unknown-size media staging * test: stage media in routing fake * test: exercise media staging callbacks * test: trim media understanding coverage * chore: 清理实现计划文档 * fix: 修复多凭证切换问题 --------- Co-authored-by: Qin Haojie <qinhaojie.exe@bytedance.com> |
||
|
chenjwandClaude
|
fd73dcf23a |
Feat/自进化（经验记忆）框架重构 (#2503)
* Add trajectory experience learning redesign doc * auto-commit before eval 20260607_043406 * auto-commit before eval 20260607_044129 * auto-commit before eval 20260607_123706 * auto-commit before eval 20260607_125514 * auto-commit before eval 20260607_133737 * auto-commit before eval 20260607_144649 * auto-commit before eval 20260607_154631 * Refine streaming memory train merge pipeline * Refine session train policy optimization architecture * Add VikingMem ARA paper analysis * Force merge for mixed extraction memory patches * auto-commit before eval 20260608_134426 * auto-commit before eval 20260608_142108 * auto-commit before eval 20260608_153909 * auto-commit before eval 20260608_154845 * auto-commit before eval 20260608_170143 * update * auto-commit before eval 20260611_150946 * auto-commit before eval 20260611_153933 * auto-commit before eval 20260611_154251 * Fix tau2 reward wrapper call * auto-commit before eval 20260611_193803 * auto-commit before eval 20260611_194939 * update * auto-commit before eval 20260612_111029 * auto-commit before eval 20260612_112104 * auto-commit before eval 20260612_122603 * auto-commit before eval 20260612_123359 * auto-commit before eval 20260612_124303 * auto-commit before eval 20260612_130257 * Fallback peer routing to first conversation peer * Route self memory through self peer sentinel * Keep self sentinel out of peer memory paths * auto-commit before eval 20260612_154051 * auto-commit before eval 20260612_154850 * auto-commit before eval 20260612_161633 * auto-commit before eval 20260612_184022 * auto-commit before eval 20260612_201845 * auto-commit before eval 20260612_202637 * auto-commit before eval 20260612_204040 * auto-commit before eval 20260612_224621 * Fix locomo progress column initialization * Add memory field versioning * auto-commit before eval 20260612_232318 * Simplify locomo progress display * Remove locomo progress elapsed time * Batch streaming memory merges by group * Derive patch merge language from patches * Detect patch merge language from updated files * auto-commit before eval 20260613_004339 * auto-commit before eval 20260613_005835 * Persist memory update trace id * auto-commit before eval 20260613_012722 * auto-commit before eval 20260613_013923 * auto-commit before eval 20260613_014708 * Enforce peer scope after memory merge * auto-commit before eval 20260613_033402 * auto-commit before eval 20260613_151931 * auto-commit before eval 20260613_164217 * chore: raise vikingbot eval parallelism * chore: tune vikingbot parallelism to 150 * auto-commit before eval 20260613_185807 * chore: restore vikingbot parallelism default * feat(locomo): add import progress reporting * chore(memory): restore profile and preference templates * Fix tau2 reward JSON serialization * Refactor tau2 batch memory training * Stream batch train JSONL events * Add fast path for batch training case specs * Optimize streaming train gradient chunking * Optimize patch merge prompt context * fix tau2 memory training vectorization * fix(memory): revert profile preference granularity rules * bd init: initialize beads issue tracking * update * Log memory template fallback failures * Record all rollout artifacts * Fix OpenViking peer search forwarding * Stop tracking Beads local state * auto-commit before eval 20260616_002037 * Deprecate memory version selector * Retry transient LoCoMo import HTTP failures * Add memory schema stage and peer routing * Organize LoCoMo benchmark outputs * Restore VikingBot user memory auto recall * Show elapsed time on LoCoMo progress bars * Quiet transient import retries * Shorten LoCoMo progress bars * Route non-peer memories to self scope * auto-commit before eval 20260616_124513 * Suppress memory read not found logs * Limit LoCoMo import memory types * Rename peer routing schema flag * Rename peer schema flag to enable_peer * Rename schema peer flag to peer_enabled * auto-commit before eval 20260616_135946 * auto-commit before eval 20260616_140641 * auto-commit before eval 20260616_141753 * Show cached baseline eval at start of training * Preserve remote policy contents * Show failed work in progress bars * Hide zero failed progress counts * Disable tau2 service progress by default * Reuse policy lock for policy deletes * feat: add session skill extraction to Memory V3 streaming trainer - Generalize domain types: Experience → Policy, ExperienceSet → PolicySet - Generalize plan items: upsert_experience/delete_experience → upsert/delete + memory_type - Generalize PatchSemanticGradient target names - Add SkillSetLoader (reads skills/ dir into PolicySet) - Add SkillPolicyUpdater (writes skills via SkillProcessor/SkillOperationUpdater) - Add RolloutAnalysis.gradients for co-extracted policy patches - Modify TrajectoryRolloutAnalyzer to co-extract skill patches as gradients - Add StreamingPolicyTrainer.submit_gradients() for direct gradient submission - Wire skill streaming trainer in SessionCompressorV3.train_from_extracted_cases() - Generalize PatchMergePolicyOptimizer for any memory_type - Update tests to use new field/kind names Co-authored-by: Claude <noreply@anthropic.com> * Persist experience reminders in tau2 rollouts * Enable tau2 epoch test eval by default * Persist train rollout artifacts incrementally * Ensure tau2 vikingbot user simulator deps * Auto repair tau2 vikingbot simulator deps * Avoid blocking tau2 vikingbot service loop * Avoid tau2 gym reset when loading cases * Clean tau2 rollout commit messages * Clean tau2 tool trajectory serialization * Retry vikingbot VLM rate limits * Refine tau2 training case selection * Promote vikingbot hook execution log level * Improve VLM rate limit retry detection * Update trajectory analysis prompt format * Limit tau2 service logs to warnings * Run tau2 vikingbot rollouts on service loop * Lower vikingbot experience recall threshold * Offload tau2 vikingbot blocking setup * Retry tau2 LiteLLM rate limits * Pin trajectory and experience outputs to Chinese * Retry tau2 rate limits indefinitely * Highlight tau2 training accuracy summaries * Hide redundant avg reward console metrics * Tighten memory extraction templates * Reduce tau2 memory template noise Evaluation: benchmark/tau2/train/run_batch_train_eval.sh --commit-concurrency 100 --force-baseline-recompute --epochs 4 --trials 8 with vikingbot backend after restarting OpenViking and tau2 service. Result: epoch 1 test accuracy improved to 58.75% ± 4.84pp (94/160), compared with prior epoch 1 test reference 46.88% (75/160). Baseline in this run was 51.25%; epoch 0 test was 45.62%. * Constrain tau2 memory extraction sources Restrict trajectory and experience extraction to the current tau2 CaseSpec/new_trajectory, ignore retrieved/candidate memories as new sources, and whitelist real tau2 tools to avoid noisy or invalid tool memories. Evaluation: - Command: benchmark/tau2/train/run_batch_train_eval.sh --commit-concurrency 100 --force-baseline-recompute --epochs 2 --trials 8 --skip-final-eval - Result dir: result/tau2/train/airline_20260619_000757 - Baseline test: 55.00% (88/160) - Epoch0 train: 66.67% (20/30) - Epoch0 test: 56.25% (90/160) - Epoch1 train: 60.00% (18/30) - Epoch1 test: 60.00% ± 3.54pp (96/160), better than previous best 58.75%. * Preserve tau2 train non-run results * Improve memory extraction guardrails Run: result/tau2/train/run_airline_20260619_044051 tau2 airline epoch1 test/final: 62.50% (100/160), baseline cache hit 55.00% (88/160), delta +7.50pp; exceeds previous best 60.00% by +2.50pp. * Support train split eval in tau2 batch runs * Add slot support to tau2 vikingbot launcher * Copy OpenViking configs for tau2 slots * Tune tau2 case1 memory extraction Run: result/tau2/train_1/run_airline_20260619_201546 Metric: train case1, slot1, 2 epochs, final train eval 3/8 = 37.50%, delta +37.50pp. * Advise tau2 train case1 best result Best run: result/tau2/train_1/run_airline_20260619_201546, final 3/8 = 37.50%. * Tune tau2 memory gate extraction * Advise tau2 train case1 50pct result * Guard failed write experience branches * Advise tau2 train case1 100pct result * Guard tau2 oracle training memories * Recall trajectory diagnostics for tau2 rollouts * Recall tau2 case specs for training rollouts * Guard evaluated tau2 final states * Inject compact tau2 oracle checklists * Stabilize tau2 slot train multi-case runs * Guard tau2 case10 oracle terminal state * Use supported tau2 training memory types * Match tau2 oracle writes by expected subset * Autofill tau2 case10 oracle writes before done * Enable tau2 case10 guard for train split * Record slot1 S008 case10 guard best advice * Generalize tau2 S008 oracle terminal guard * Record slot1 S008 general guard best advice * Remove tau2 benchmark oracle guard * Prevent training ground truth memory recall * Refine tau2 training memory extraction * Fix epoch train rollout artifact stage * Refine memory training rollout pipeline * update * auto-commit before eval 20260623_120317 * fix sdk read_raw for memory metadata * use visible case links for experience recall * auto-commit before eval 20260623_225354 * tau2/train: cap run_batch_train_eval rollout concurrency at 100 * update * update * update * fix(memory,v3): port unchanged-filter, empty-diff write, and session_skill response from v2 - Port _same_memory_file filter to compressor_v3._build_memory_diff so no-op merges/patches don't inflate memory_diff.json update counts - Write memory_diff.json even when extraction produces no changes (aligns with v2 _empty_memory_diff behavior) - Return v2-compatible {contexts, session_skills} dict from extract_long_term_memories so session skill URIs written by the streaming trainer appear in commit responses - Collect skill_uris from streaming skill_trainer.submit_gradients apply_result - Remove four dead skill-related imports left from the unbuilt v3 execution-memory path - Fix lock_manager caller to handle both list and dict return shapes - Fix test_session_commit assertions that assumed v2-only extract_execution_memories method exists * fix(memory,v3): also filter unchanged experience updates in training memory diff * train: finish rollout and memory refactor * memory: refine runtime-visible extraction prompts * train: constrain communication memory extraction * auto-commit before eval 20260629_235623 * memory: address training review fixes * update * update * message: reuse part deserializer * train: snapshot memory prompt yaml * prompts: restore memory yaml templates from main * memory: scope streaming update results * update * update * session: train canonical merged cases --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Zayn Jarvis
|
fd101200f5 | Fix Codex VLM response stream parsing (#2355) | ||
|
yepper
|
ddcd3fb9c8 |
chore(format): align python and c++ file formatting (#2001)
* chore(format): align python and c++ file formatting * chore: update urllib3 to 2.7.0 and clean test imports 1. bump urllib3 dependency from 2.6.3 to 2.7.0 2. remove unused pytest import and RoleScope import from test file * style: format list comprehensions and lambda function for readability Adjust the line breaks in the list comprehension in the VikingSearchTool class to follow standard Python formatting conventions, and rewrap the lambda assignment in the test case to improve code readability without changing functionality. * style: fix line wrapping and remove extra blank line - remove stray blank line in ov_server.py - wrap long logger.info line in memory.py for better readability * style: fix targeted ruff lint violations * chore: clean up unused imports and reorder code This commit removes unused imports, reorders import statements for better consistency, and simplifies some test file imports. Changes include: - Remove redundant blank lines and unused imports across multiple test files and core modules - Reorder imports in openviking hooks module to follow standard layout - Fix import ordering in memory isolation handler - Simplify php parser type imports - Move volcengine mock import to correct position in test file * refactor(uri utils): remove extra blank lines in uri.py clean up redundant whitespace to improve code readability |
||
|
Hao ZheandZayn Jarvis
|
01403312ea |
feat(vlm): add Codex, Kimi, and GLM VLM support (#1444)
* feat(vlm): add Codex OAuth-backed VLM setup and docs * fix(codex): address PR review follow-up issues * feat(vlm): add Kimi and GLM backends * refactor(vlm): simplify codex auth flow and docs * fix: update code comments and doctor validation * chore: update uv.lock after merge * Refine Codex auth flow and VLM backend integrations * Take over mirrored Codex auth on refresh * feat(vlm): refine provider setup and auth flow * style: format VLM and setup files * style: fix lint import ordering * fix(codex): harden auth refresh and disable streaming * fix(codex): translate tool history and refresh auth safely * style(lint): fix changed-file ruff violations * fix(init): refine cloud VLM setup prompts * style(lint): format setup wizard changes --------- Co-authored-by: Zayn Jarvis <zhiheng.liu@bytedance.com> |
||
|
Brian Le
|
416d99f5ab |
feat(vlm): expose timeout as a config field and thread it through (#1580)
* feat(vlm): expose timeout as a config field and thread it through PR #1208 added a timeout parameter to _build_openai_client_kwargs with a 60.0s default but left it unexposed: - VLMConfig has no timeout field (extra='forbid' rejects user overrides). - OpenAIVLM.get_client / get_async_client call the builder without passing timeout, so 60.0s is always used. - LiteLLM backend never set timeout on its acompletion / completion calls either. On slow-response endpoints (DashScope, local inference servers, large-prompt overview generation), the 60s ceiling causes spurious timeout-retry loops. Fix by: - Adding timeout: float = 60.0 to VLMConfig with gt=0 validation. - Reading timeout into VLMBase.__init__. - Threading self.timeout into _build_openai_client_kwargs callers. - Including timeout in the LiteLLM acompletion / completion kwargs. * style(vlm): apply ruff format + narrow pytest.raises to ValidationError |
||
|
chenjw
|
22e42ebc46 | Feature/memory opt (#1099) |