mirror of
https://github.com/volcengine/OpenViking.git
synced 2026-10-01 09:48:03 +08:00
t0sakiandbaobaodae
72d04cd488
* feat(ingest): replay local agent-harness logs into OV sessions / 本地 agent harness 日志重放入库
Add openviking/ingest/: parse Claude Code / Codex / OpenCode / Hermes / OpenClaw conversation logs into normalized messages and replay them through OpenViking's existing session pipeline (create_session -> batch_add_messages -> commit -> async memory extraction), instead of a bespoke ETL.
Supports one-shot backfill ("存量") and cursor-driven incremental polling ("新增", WatchScheduler-style, no fs-event dependency), per-harness enable/mode/paths config, and meaningful peer_id on every turn (assistant = {harness}/{model}; user = git identity for single-user harnesses, original username for group-chat harnesses). Cursor IDE is a registered but deferred stub.
Read-position cursors persist under ~/.openviking/ingest/state.db for crash-safe, idempotent resume. New openviking-ingest CLI (backfill/watch/run/status/list-sources) and an "ingest" section on OpenVikingConfig. Verified end-to-end against a local server: 3-message fixture -> session commit -> 10 memories extracted -> idempotent re-run.
Inspired by / supersedes volcengine/OpenViking#2674.
Co-authored-by: baobaodae <2014596548@qq.com>
* docs(ingest): bilingual guide + ov.conf.example for openviking-ingest / 本地日志入库双语文档与配置示例
Add docs/{zh,en}/agent-integrations/09-log-ingestion.md (auto-registered in the VitePress sidebar) and an `ingest` section in examples/ov.conf.example (off by default).
* fix(ingest): address review — gating, crash-safe batch replay, commit recovery, single-instance lock / 修复评审问题
Fixes the merge-blockers from the adversarial review:
- master switch ingest.enabled now actually gates enabled_harnesses();
- idempotent per-batch append with a durable pending-intent reconciled against the server message count on restart (no duplicate imports after a mid-append crash);
- bounded reads (<=100 msgs/call) so huge sessions don't materialize at once;
- needs_commit flag + commit_if_needed so appended-but-uncommitted sessions still get extracted (commit even when no new source rows);
- poller keeps dirty sessions until a commit actually succeeds;
- OpenCode advances its SQLite cursor only past complete rows (late part text no longer skipped);
- single-instance file lock guards concurrent ingest processes;
- positive-value config validation (no poll busy-loop); malformed ov.conf surfaces instead of silently defaulting.
Adds 6 tests (config gating/validation, crash reconcile both ways, commit recovery).
* refactor(ingest): expose as 'openviking-server ingest' subcommand; English-only code/docs
- Route the ingest CLI through 'openviking-server ingest ...' (same dispatch as 'init'/'doctor') and drop the separate 'openviking-ingest' console_script.
- Remove mixed-in Chinese terms (存量/新增) from source docstrings, CLI help, and the English doc; the Chinese doc keeps them.
* style(ingest): ruff format + import sort (isort I)
Run ruff 0.15.16 (from the uv cache) with the repo config: fixes 5 I001 import-order errors in tests and reformats 9 files. 'ruff check' and 'ruff format --check' now pass on all added/edited files.
---------
Co-authored-by: baobaodae <2014596548@qq.com>
25 lines
867 B
Python
25 lines
867 B
Python
# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.
|
|
# SPDX-License-Identifier: AGPL-3.0
|
|
"""Test isolation for the ingest suite.
|
|
|
|
``resolve_git_human_peer`` shells out to ``git config``, which otherwise reads the
|
|
developer machine's GLOBAL identity and makes human-peer resolution non-deterministic.
|
|
Neutralize git config discovery so user-side peers fall back to the configured value.
|
|
"""
|
|
|
|
import pytest
|
|
|
|
from openviking.ingest import peer
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _isolate_git_identity(tmp_path_factory, monkeypatch):
|
|
empty = tmp_path_factory.mktemp("gitcfg") / "empty.cfg"
|
|
empty.write_text("")
|
|
monkeypatch.setenv("GIT_CONFIG_GLOBAL", str(empty))
|
|
monkeypatch.setenv("GIT_CONFIG_SYSTEM", str(empty))
|
|
monkeypatch.setenv("GIT_CONFIG_NOSYSTEM", "1")
|
|
peer._GIT_PEER_CACHE.clear()
|
|
yield
|
|
peer._GIT_PEER_CACHE.clear()
|