Commit Graph
20 Commits
Author SHA1 Message Date
huangruitengandhuangruiteng 6de1cd2c6e test(oc2ov): fail fast on empty OpenClaw output (#3250)
Co-authored-by: huangruiteng <huangruiteng@bytedance.com>
2026-07-15 11:02:37 +08:00
Qin Haojie ff258768c2 feat(memory): 引入 User/Peer 记忆隔离模型 (#2236)
* feat(memory): introduce user and peer memory isolation

Unify agent-scoped memory behavior into user-owned memory spaces, add peer_id compatibility for session and retrieval paths, and wire memory_policy through session commit flows.

* feat(memory): align session identity around peer IDs

* feat(search): pass peer id through retrieval

* refactor(memory): remove agent identity from integrations

* fix(memory): isolate peer identity from self extraction

* fix(tau2): provision benchmark user configs

* fix(auth): allow admin keys to access data APIs

* fix(openclaw): enable peer memory policy for peer roles

* fix(openclaw): resolve sender for peer recall

* refactor(session): simplify memory extraction routing

* refactor(ov-cli): reduce formatting-only diff

* refactor(message): remove unused message helpers

* refactor(retrieval): simplify peer target resolution

* refactor(namespace): remove deprecated agent namespace policy

* fix(agent): propagate peer id through integrations

* fix(auth): align integration clients with api-key mode
2026-06-05 10:55:48 +08:00
kaisongli 59c70bdbf1 feat: add new API test cases and fix memory v2 test reliability (#2097) 2026-05-18 11:28:05 +08:00
kaisongli d7ef4a04a1 feat: add CLI integration tests, deduplicate oc2ov_test, and extend CI pipeline (#2061)
- Add CLI integration tests (10 test files under tests/cli/)
- Extend api_test.yml with CLI install + test steps
- Run filesystem + scenarios/resources_retrieval serially to avoid 409 conflicts
- Other tests parallel with -n 4
- Add release prereleased trigger to api_test.yml and api_test_effect.yml
- Deduplicate oc2ov_test P0 cases (20→12, ~30-55min saved):
  - Delete test_memory_write.py (covered by V2 suite)
  - Remove events/tools from V2 suite (structurally identical to entities/skills)
  - Remove test_memory_read_verify (covered by V2 suite)
  - Remove test_cross_session_recall (overlaps with recall_explicit_search)
- Add ensure_resources_dir fixture to prevent NOT_FOUND on fresh environments
- Add retry logic for 429/500/403 rate-limit in api_client.py
- Add retry for commit when task_id is None in test_memory_v2_full_suite.py
- Add exponential backoff retry for GitHub platform test 5xx errors
2026-05-15 14:05:26 +08:00
kaisongli b66d51c8d0 Fix/oc2ov v2 session 404 (#1824)
* fix: handle OV session reuse in memory v2 tests

When OV reuses existing sessions instead of creating new ones,
find_new_session_id returns None. Now tries commit on all existing
sessions to find one with pending messages. Also handles 404 and
task_id=None gracefully, falling back to checking memory files
directly.

* fix: add retry with wait for OV auto-capture session creation

OV auto-capture may take a few seconds to create a new session after
OpenClaw sends a message. Increased initial wait from 5s to 8s, and
added up to 3 retries with 5s intervals when session not found.
Falls back to committing all existing sessions if still not found.
2026-05-06 11:39:11 +08:00
kaisongli fa19814d91 Fix/oc2ov remote ov adapt (#1815)
* fix: handle None values in commit response for memory v2 tests

commit_resp.get('data') can return None when OV API returns unexpected
format, causing 'NoneType object has no attribute get' error.
Use 'or {}' to ensure dict type.

* perf: skip retry on subprocess timeout to reduce test duration

Subprocess timeout (180s) means auto-recall context is too large,
retrying is pointless and wastes 180s*3=540s. Now:
- subprocess timeout: return immediately, no retry
- LLM idle timeout/empty/tool-result: still retry up to 3 times
- Default timeout back to 180s (was 300s), removed explicit timeout=300
- _is_llm_unstable_response still skips assertions on timeout
2026-04-30 11:06:53 +08:00
kaisongli 574e3dea03 fix: correct HTTP status code assertions in API tests for error responses (#1780)
- test_build_media_resources_slow: assert 500 for SVG parse failure
- test_build_platform_wikipedia: assert 500 for Wikipedia URL fetch failure
- test_build_error_handling_slow: assert 500 for corrupted ZIP (align with non-slow version)
- test_memory_v2_full_suite: add find_session_by_id fallback for CI deterministic session IDs
2026-04-28 21:28:36 +08:00
kaisongli 34ab56705d Fix/oc2ov timeout tolerance (#1726)
* fix: increase CLI timeout and handle subprocess timeout for auto-recall context bloat

- openclaw_cli_client.py: increase default timeout 180s→300s, add timeout param
- base_cli_test.py: _is_llm_timeout detects subprocess timeout format,
  send_and_log/send_and_retry_on_timeout support timeout param
- test_context_engine.py: all cross-session and post-commit queries use
  timeout=300 to handle auto-recall injecting large context
- test_memory_crud.py: all query operations use timeout=300

Root cause: auto-recall injects duplicate/verbose memories into context,
causing LLM to exceed 180s subprocess timeout when processing.

* fix: handle subprocess timeout in LLM instability detection and improve memory v2 session handling

- base_cli_test.py: _is_llm_unstable_response detects subprocess timeout
  format, so assertions skip when auto-recall context bloat causes timeout
- test_memory_v2_full_suite.py: wait and retry for OV session creation,
  add follow-up dialog retry when commit returns task_id=None, avoid
  committing wrong session via find_session_with_most_messages fallback
- openclaw_cli_client.py: default timeout 180s→300s, support custom timeout
- test_context_engine.py/test_memory_crud.py: query operations use timeout=300

* style: ruff format

* style: fix ruff F541 lint errors (remove f-string prefix without placeholders)
2026-04-28 17:30:43 +08:00
kaisongli cadf027957 [fix] Fix/oc2ov test stability (#1669)
* fix(oc2ov-test): 优化流水线配置和用例稳定性

- oc2ov_test.yml: 定时任务改为一天两次(10:00/18:00北京时间),超时改为100分钟
- api_test_effect.yml: 已有限制只有主仓库运行定时任务
- TestMemoryRead: 查询措辞优化+关键词容错,兼容旧记忆文件残留

* ci: restrict scheduled/release triggers to main repo only

Change workflow job conditions from exclusion-based to inclusion-based:
- oc2ov_test.yml: only workflow_dispatch or main repo can run
- api_test_effect.yml: only workflow_dispatch or main repo can run
- api_test.yml: only workflow_dispatch, pull_request, or main repo can run

This prevents fork repos from triggering resource-consuming scheduled
and release workflows while still allowing manual dispatch and PRs.
2026-04-23 17:06:03 +08:00
kaisongli c7df2971db [feat] Test/add resource ci validation and fix session api case 400 error (#1599)
* fix(oc2ov-test): 修复P0用例失败问题并增强Context Engine测试覆盖

主要变更:
- base_cli_test: smart_wait_for_sync/wait_for_sync 新增 session_id 参数,修复独立session检查错误session的Bug
- base_cli_test: 新增 send_and_retry_on_timeout 方法,处理LLM超时和空响应自动重试
- assertions: extract_response_text 拼接所有payloads而非只取第一个;空payloads时返回空字符串而非str(response)
- test_context_engine: 新增6个Context Engine核心交互测试(assemble/compact/recall/expand/isolation)
- test_memory_crud: TestMemoryRead增加确认步骤和关键词容错;使用独立session_id
- conftest: 更新测试报告描述,覆盖Context Engine交互场景
- run_tests: 更新测试路径配置

* feat: add-resource CI validation tests + effect patrol + session fix

Add 43 add_resource effect test cases in resources_retrieval_slow/:
- TC-B: 21 file build tests (text/document/archive/media)
- TC-P: 7 platform routing tests (GitHub/Wikipedia/arXiv/general web)
- TC-E: 15 error handling tests (HTTP/DNS/SSH/duplicate/corrupted)
- All cases have strict business logic assertions (L4-L6)

CI workflow improvements:
- Add api_test_effect.yml for daily patrol (cron 2am UTC)
- Update api_test.yml: schedule/workflow_dispatch runs on ubuntu+mac+windows
- Filter slow directory in all api_test.yml branches
- Fix Windows ragfs build (backslash escaping, integer comparison)

Bug fixes:
- Add parent parameter to add_resource() in client.py
- Add default user registration in conftest.py for CI environment
- Handle LLM tool-result-only responses in oc2ov_test
- Strip tool result prefix in assertions.py
- Fix ruff B011: assert False -> raise AssertionError

* style: ruff format 6 files

* fix: add conftest.py for build_test_helpers import path

- Add conftest.py in resources_retrieval/ and resources_retrieval_slow/
- Fixes ModuleNotFoundError when pytest runs from tests/api_test/ root
2026-04-22 11:03:08 +08:00
kaisongli fe19ba56b6 fix(oc2ov-test): 修复P0用例失败问题并增强Context Engine测试覆盖 (#1586)
主要变更:
- base_cli_test: smart_wait_for_sync/wait_for_sync 新增 session_id 参数,修复独立session检查错误session的Bug
- base_cli_test: 新增 send_and_retry_on_timeout 方法,处理LLM超时和空响应自动重试
- assertions: extract_response_text 拼接所有payloads而非只取第一个;空payloads时返回空字符串而非str(response)
- test_context_engine: 新增6个Context Engine核心交互测试(assemble/compact/recall/expand/isolation)
- test_memory_crud: TestMemoryRead增加确认步骤和关键词容错;使用独立session_id
- conftest: 更新测试报告描述,覆盖Context Engine交互场景
- run_tests: 更新测试路径配置
2026-04-20 11:24:35 +08:00
kaisongli 8d39834795 fix(ci): fix ragfs-python native extension build in CI pipelines (#1532)
* fix(ci): fix ragfs-python native extension build in CI pipelines

* fix(ci): skip sessions/ tests when no VLM/Embedding secrets available

Sessions tests (add_message, commit, used) depend on storage backend
that requires proper initialization with valid API keys. When running
without secrets (PR from fork), these tests fail with 400 because the
storage layer cannot fully initialize. Skip them in no-secrets mode;
they will still run on main branch pushes where secrets are available.
2026-04-17 17:13:23 +08:00
kaisongli 6e18da6578 fix: block CI when OpenViking server fails to start (#1478)
- Server startup failure now exits with error instead of just warning
- Use python -u for unbuffered output to capture crash traceback
- Print clear message about data compatibility issues
- Do NOT auto-clean data - let CI surface compatibility problems
2026-04-15 17:02:25 +08:00
kaisongli 793808cb96 fix: support CI environment in upgrade_openviking.sh (#1467)
- Use GITHUB_WORKSPACE as PROJECT_DIR in CI
- Skip backup (cp -r) in CI to avoid disk/time issues
- Skip git fetch/reset in CI since checkout already done
2026-04-15 14:35:47 +08:00
kaisongli d18d9d771f fix: resolve OpenClaw session lock conflicts & fix session commit assertion (#1441)
- Add pre-start cleanup in upgrade_openviking.sh: remove stale .lock
  and .jsonl.lock files, kill all residual openclaw processes before restart
- Add session lock awareness in openclaw_cli_client.py: wait for lock
  release before sending requests and after subprocess returns
- Optimize wait_for_sync in base_cli_test.py: enforce minimum 5s wait,
  check lock release before proceeding, raise poll interval to 3s minimum
- Add Chinese descriptions for Memory V2 test cases in conftest.py
- Fix test_session_commit assertion: remove pre_archive_abstracts check
  since get_session_context no longer returns this field
2026-04-14 20:24:01 +08:00
kaisongli f1a8954943 feat: memory v2 test suite with ov-install plugin update & split test cases (#1354) 2026-04-14 16:02:48 +08:00
kaisongli 006593ff2e fix(security): remove leaked token from settings.py (#1319)
- Remove tests/oc2ov_test/config/settings.py containing exposed auth token
- Add settings.py to .gitignore to prevent future leaks
- Users should copy settings.example.py to settings.py and fill in their own tokens
2026-04-09 11:44:51 +08:00
kaisongli 83e04d8cd8 fix: 优化测试关键词匹配和移除 Release Approval Gate (#1313)
- 添加中英文关键词匹配:删除、无、deleted、expired、no longer
- 移除 workflow 中的 release-approval job
2026-04-09 11:13:31 +08:00
kaisongli 9d1ea4e632 feat: 巻加测试优化功能 (#1280)
- Session ID 自动管理
- 智能等待策略
- 重试机制
- 测试数据管理
2026-04-08 19:18:18 +08:00
kaisongli 06ae06c3dc feat: Add OpenClaw2OpenViking integration tests with CI pipeline (#1168)
* feat: add OpenClaw2OpenViking integration tests with CI pipeline

- Add P0 memory tests for basic write/read operations
- Add session persistence tests for cross-session memory validation
- Add CI pipeline with auto-upgrade functionality
- Add upgrade script for development mode installation
- Support virtual environment detection and activation
- Generate HTML test reports and upload artifacts
- Ensure OpenClaw reloads latest OpenViking after upgrade
- Add cleanup mechanism for interrupted builds
- Improve OpenViking version detection with multiple fallback methods
- Fix code formatting issues (ruff format)

* fix: resolve ruff check errors

- Add noqa comment for F403 in config/__init__.py
- Remove unused variables (F841)
- Fix import sorting (I001)
- Remove unnecessary f-string prefixes (F541)
- All ruff check errors resolved
2026-04-02 15:26:44 +08:00