* feat(memory): support event tag filtering
Add session-level default event tags, commit-time overrides, durable queue propagation, and first-write vector index tagging. Include config update APIs and coverage for serialization, concurrency, extraction, and HTTP behavior.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* feat(memory): expose event tags in SDKs and CLI
Add session default tag configuration, config updates, and commit-time event tag overrides across embedded Python, standalone Python, TypeScript, Go, and the Rust CLI. Preserve explicit empty-tag semantics and document each public interface.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(sdk): align legacy session tag APIs
Forward commit-time event tags through the legacy Python HTTP shims and align BaseClient session signatures without adding a new abstract-method requirement for existing subclasses.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* feat(session): allow updating auto-commit policy
Extend PATCH session config to atomically update event tags and auto-commit settings. Merge policy objects by field, use explicit null to disable automatic commits, preserve omitted fields, and expose the contract across SDKs and CLI.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(session): align session config interfaces
Replace the generic session create config JSON flag with explicit event-tag and auto-commit options. Preserve omitted, object, and null auto-commit semantics across HTTP, embedded clients, SDKs, and CLI, reject ambiguous null policy fields, and handle nullable event configuration consistently.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* test(session): trim redundant event tag tests
---------
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
* fix(server): stop exporting raw query strings and buffering zip responses in observability
Sweep findings: B-03, B-13. Prevent query secrets from reaching traces and keep ZIP responses streaming.
(cherry picked from commit d8ac3dc33b)
* fix(session): tolerate missing/corrupt archive in Phase-2 replay
(NotFoundError / _ArchiveMessagesCorruptError) on a missing or corrupt
archive messages.jsonl instead of returning []. That PR added skip-on-
missing tolerance to the read path (_get_uncovered_archive_messages) and to
resume_queued_commit, but not to the Phase-2 commit replay path
(_prepare_phase2_archive_messages), which calls _read_archive_messages
unguarded while rolling earlier failed archives into the current commit.
Consequence: a terminally-failed earlier archive whose messages.jsonl is
missing/corrupt (legacy "no messages" terminal data, or produced by #3417's
own archive_read terminal path) makes every subsequent commit's Phase-2
extraction raise -> caught by _run_memory_extraction's except -> the current
archive is terminal-failed too. Because the poisoned archive is only removed
from replay once "covered" (which requires a later archive to complete), and
no later archive can ever complete, the session's memory extraction is
permanently poisoned. Raw messages are safe, but extraction is stuck.
Fix: wrap the replay-loop _read_archive_messages call in the same tolerance
_get_uncovered_archive_messages already uses -- skip + warn on not-found
(_is_storage_not_found) and on _ArchiveMessagesCorruptError, re-raise real
storage failures. The skipped archive stays in covered_failed so the current
archive's .done marks it covered, clearing the poison permanently.
Adds a regression test asserting the replay skips a failed archive with a
missing messages.jsonl (and marks it covered) instead of raising, and that a
real storage failure still propagates.
Follow-up to #3417.
(cherry picked from commit 5b8ec9e68a)
* fix(client): align client surfaces without leaking memory metadata
Reconstructs the client-parity work from upstream PR #3439 on current main and strips reserved memory metadata before line slicing in both embedded and HTTP reads.
Based-on: 48b411d58c
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* fix(index): propagate semantic vectorization failures safely
Reconstructs upstream PR #3437 on current main, carries enqueue failures through SemanticDagExecutor, and drains the attempt's embedding tracker before retry-visible failure propagation.
Based-on: 02387deb09
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* fix(core): close privacy and embedding failure gaps
* fix(memory): strip repeated metadata trailers
* fix(core): close public memory visibility gaps
* ci: skip embedding-dependent resource test without secrets
---------
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* chore: clear unused files
* fix(tests): fix unit test
* refactor(auth): introduce plugin-based authentication architecture
Replace the monolithic `openviking/server/auth.py` with an extensible
plugin-based auth system. This refactor extracts the three built-in modes
(`dev`, `api_key`, `trusted`) into separate `AuthPlugin` implementations,
adds a registry for third-party plugins, and preserves all existing behavior
while enabling custom authentication backends (e.g. LDAP, OIDC, mTLS).
Key changes:
- **New public API**: `AuthPlugin` (ABC) and `register_auth_plugin` decorator.
- **New registry**: `AuthPluginRegistry` supports runtime registration.
- **Built-in plugins**: `DevAuthPlugin`, `ApiKeyAuthPlugin`, `TrustedAuthPlugin`.
- **Config change**: `auth_mode` widened from `Literal` to `str` for custom modes.
- **Validation delegated**: `validate_server_config()` now delegates to the active
plugin's `validate_config()`, preserving existing validation semantics.
- **Router compatibility**: All existing `require_*` decorators and `resolve_identity`
/ `get_request_context` dependencies remain unchanged. Routers import the same
symbols from `openviking.server.auth`.
- **Tests**: `conftest.py` manually wires the DevAuthPlugin in ASGI tests (lifespan
not triggered). `test_auth.py` expanded with plugin registration and validation tests.
- **Docs**: `04-authentication.md` (en/zh) updated with plugin registration examples.
Co-Authored-By: claude-sonnet-4-6 <noreply@anthropic.com>
* fix(tests): fix trusted mode test
* fix(tests): fix unit test
* fix(cli): remove unexisted transaction observer
* docs: update skills definition
* docs: update skills definition
* docs: update skills definition
* docs: update skills definition
* fix(skills): now we allow viking://agent/skills again, and optimize CLI for skills
* docs(skills): use -p instead of --parent in agent skills examples
Align the `ov skills add` examples in the context-types and viking-uri
docs with the short flag `-p` introduced for `ov skills list/find/show`,
so all four user-facing examples consistently demonstrate the short form
when targeting `viking://agent/skills`.
Co-Authored-By: claude-sonnet-4-6 <noreply@anthropic.com>
* fix(tests): error check for api key
* fix(tests): unit test wait until resource not busy
* fix(tests): unit test wait until resource not busy
* fix(sdk): args form in skills find
* fix(skills): pass target uri in request body
---------
Co-authored-by: claude-sonnet-4-6 <noreply@anthropic.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
Previously, adding messages required one HTTP request per message,
making bulk operations (e.g. memory extraction, history migration)
very slow due to network round-trip overhead.
Changes:
- Add POST /api/v1/sessions/{id}/messages/batch endpoint
- Add BatchAddMessageRequest model with max_length=500 limit
- Extract _resolve_message_parts() helper to deduplicate part resolution
- Add _defer_meta_save parameter to Session.add_message() for batch optimization
- Add batch_add_messages method to Python SDK clients (base/http/sync)
- Add batch_add_messages to Session wrapper class
- Update LangChain integration to use batch API
- Update Rust CLI add_memory to use batch API
* feat(fs): add count API for directory entry counting
Adds a dedicated `count` endpoint that returns the exact number of files
and sub-directories under a directory by traversing the filesystem,
distinct from `stat`'s vector-index-based estimate. Wired through
VikingFS, FSService, HTTP router and sync/async/local SDK clients.
* feat(cli): add `ov count` command for directory entry counting
Wires the new fs.count HTTP endpoint into the Rust CLI. Adds
`-r/--recursive` and `-a/--all` flags. Documentation updated with
CLI usage examples.
* fix
---------
Co-authored-by: dingben.db@bytedance.com <dingben.db@bytedance.com@bytedance.com>
* feat(ovpack): add v2 manifest and conflict policy
Add a portable OVPack manifest for scalar metadata and make imports validate scope, derived files, and conflicts before writing.
* fix(ovpack): remove import vectorize option
Make OVPack imports always rebuild vectors in the target environment, keep legacy packages compatible, and reject unsupported manifest versions before writing.
* fix(ovpack): remove force import alias
Use on_conflict as the single OVPack import conflict policy and reject removed force inputs.
* fix(ovpack): regenerate runtime vector metadata
Keep type portable but stop exporting or applying created_at, updated_at, and active_count from OVPack manifests.
* fix(ovpack): validate manifest contents
* fix(ovpack): require manifests for imports
* fix(ovpack): close manifest validation gaps
* fix(ovpack): defer parent creation until validation passes
* fix(ovpack): remove export size guard
* fix(ovpack): support session and scope-root restores
* docs(ovpack): document full backup migration
* feat(ovpack): add backup restore workflow
* fix(ovpack): validate import scope compatibility
* feat(server): add operation telemetry for session create/add_message/commit APIs
Wrap session.create, session.add_message and session.commit HTTP handlers with
run_operation so callers can opt in via TelemetryRequest and receive a
telemetry summary in the response. Propagate the telemetry parameter through
the async/sync HTTP clients, the local client and the public SDK so all
client modes expose a consistent surface.
* refactor(client/local): move part imports into _add_message_impl where they are used
* feat(memory): add agent trajectory and experience extraction
Add a two-phase agent memory pipeline with schema-driven trajectory and experience extraction, plus system-managed source trajectory tracking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): wire agent memory extraction into session flow
Enable the agent memory pipeline behind config and invoke trajectory/experience extraction during session memory processing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): agent memory pipeline — trajectory timestamps, experience merge, concurrent extraction
- Trajectory filenames now include a timestamp suffix (via _stamp_trajectory_names
in compressor_v2 before apply_operations), so trajectory_name in both the
filename and MEMORY_FIELDS carries the full timestamped name
- Experience extraction: add merge operation (write generalized + delete_uris),
fix delete lock conflict (pass lock_handle to viking_fs.rm), and inherit
source_trajectories from deleted experiences before merge
- Near-duplicate trajectory dedup removed from memory_updater; delete moved
before write to avoid AGFS sibling lock contention
- session.py: restore user memory extraction and run user + agent memory
concurrently via asyncio.gather (agent memory gated by agent_memory_enabled)
- directories.py: trajectories and experiences directories added to agent
memory preset with abstract/overview; cases and patterns removed
- Simplify trajectory/experience YAML descriptions and instructions
- extract_loop: skip refetch for add_only schemas; add logging for URI resolution
and operation dispatch to aid diagnosis of duplicate experience writes
- demo_agent_memory.py: replace three-round demo with two same-domain rounds
to specifically test the experience edit path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): add agent_only flag to prevent user memory from processing trajectory/experience
- Add `agent_only: true` to trajectory.yaml and experience.yaml schemas
- Add `agent_only` field to `MemoryTypeSchema` dataclass
- Parse `agent_only` from YAML in `MemoryTypeRegistry._parse_memory_type`
- Filter out agent_only schemas in both `prefetch` and `get_memory_schemas`
in `SessionExtractContextProvider`, so trajectory/experience are only
processed by the agent memory extraction pipeline
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(memory): add e2e integration test for agent memory two-phase pipeline
- test_trajectory_and_experience_extraction: runs two same-domain sessions,
asserts Round 1 creates the experience and Round 2 edits it (no duplicate),
and verifies all trajectory filenames carry a timestamp suffix
- test_no_agent_only_schemas_in_user_memory: unit-level check that
trajectory/experience schemas are filtered out of SessionExtractContextProvider
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: remove demo_agent_memory.py, replaced by integration test
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): agent memory two-phase pipeline — trajectory + experience extraction
Phase 1 (trajectory): extract execution summaries from conversation, one per business domain.
Phase 2 (experience): prefetch top-5 candidate experiences + source trajectories, single no-tool
LLM call to Update/Replace/Create/Skip.
Key changes:
- AgentExperienceContextProvider: rewrite as prefetch-all + single no-tool call; top-3 candidates
include source_trajectories for grounding; prefetched_uris tracked to skip refetch check
- AgentTrajectoryContextProvider: remove read tool (was causing hallucination); tighten instruction
- ExtractLoop: fix prefetch URI tracking (old format was broken); guard tool_choice on empty tools
- compressor_v2: deserialize trajectory content before passing to experience phase; restore
user/agent memory concurrent execution in session.py
- memory_updater: downgrade diff_match_patch ImportError from tracer.error to tracer.info
- volcengine_vlm: trace tool calls and response content separately
- experience/trajectory yaml: refine field descriptions and Reflect section wording
- e2e test: add skipif guard, tracer init, two-iteration loop, persistent demo dir
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore(memory): remove unused source trajectory tool and noisy prints
Drop the unused get_source_trajectories memory tool after phase-2 moved to
prefetch-only context, and replace source_trajectory debug prints with tracer logs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(client): support explicit embedded user for agent memory tests
Allow LocalClient to accept an explicit UserIdentifier and add an integration test covering user+agent agent-memory isolation in embedded mode.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat：test
* feat(agent-memory): prefetch experience files into read_file_contents + cap source_trajectories
- AgentExperienceContextProvider.prefetch now populates _read_file_contents for
each candidate experience, fixing two issues on the Replace path:
1. resolve_operations could never find delete_file_contents → old file was never deleted
2. inherited_traj_uris was always empty → source_trajectories not inherited
On the Update path this also eliminates the extra _check_unread_existing_files
LLM round-trip that was previously triggered for every edit.
- Move deserialize_content/deserialize_metadata imports from inline to module top.
- AgentTrajectoryContextProvider.prefetch signature simplified (no unused args).
- _append_trajectories_to_experiences: cap source_trajectories at 5 most recent URIs
to prevent unbounded growth over many sessions (MAX_SOURCE_TRAJECTORIES = 5).
- e2e test cleaned up: single focused test, remove redundant Replace-path tests,
filter .abstract.md in _list_non_overview_entries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(agent-memory): update experience and trajectory memory schema prompts
- experience.yaml: restructure content format from 4-section to 3-section
(Situation / Approach / Reflect), rewrite rules to emphasize machine
readability, mutual exclusivity between Approach and Reflect, and
abstraction mandate for generalization.
- trajectory.yaml: extend content format with explicit Trajectory steps
(intent + actions + progress) and Fail reason field; add exhaustive
tracking and tool-call formatting rules.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(agent-memory): harden memory pipeline robustness
- session.py: use gather(return_exceptions=True) so user and agent
memory tasks fail independently; each side logs its own error and
falls back to [] instead of losing the other side's results
- compressor_v2: remove redundant rm before write_file in
_append_trajectories_to_experiences — agfs PUT is atomic overwrite,
so the prior delete only added a data-loss window; also drop the
duplicate ExtractContext/MemoryIsolationHandler construction in
_run_extract_phase and fix its outdated docstring
- extract_loop: remove stray blank line after prefetch tracking block
- memory_updater: remove extra blank line inside class body
- experience.yaml / trajectory.yaml: add missing trailing newlines
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat：fix agent memory test
* chore(memory): rename experience.yaml→experiences.yaml, trajectory.yaml→trajectories.yaml
* chore(memory): rename memory_type experience→experiences, trajectory→trajectories
* chore(memory): remove dead _read_files tracking in extract_loop
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Extend the ``target_uri`` parameter from ``str`` to ``Union[str, List[str]]``
across the full find/search stack so callers can scope a single query to
multiple directories in one request:
- server: FindRequest / SearchRequest accept list[str] target_uri
- service: SearchService.find / .search forward list[str]
- storage: VikingFS.find normalizes list[str], canonicalizes each entry,
and forwards the full list as target_directories to the retriever
(matching the existing behaviour of VikingFS.search)
- clients: BaseClient, LocalClient, AsyncHTTPClient, SyncHTTPClient,
AsyncOpenViking and SyncOpenViking signatures updated; the HTTP client
gains a ``_normalize_target_uri`` helper that applies
``VikingURI.normalize`` to each non-empty entry
Single-string behaviour is fully preserved: a plain ``str`` is normalized
internally to a one-element list, and empty ``""`` keeps today's
no-target semantics.
* feat(session): add account namespace policy and shared sessions
Unify namespace resolution across filesystem, indexing, and session storage.
Add account-shared session paths, role_id auth semantics, and an HTTP demo
script for the four namespace-policy combinations.
* space
* fix(pack): skip derived semantic files in ovpack transfer
Keep ovpack imports resilient to stale sidecars and rebuild semantics through the normal queue instead of restoring derived files verbatim.
* Revert "fix(pack): skip derived semantic files in ovpack transfer"
This reverts commit f4e4db8401.
* fix(namespace): default legacy accounts to agent-shared policy
Clarify that memory.agent_scope_mode is deprecated and document the supported agent memory migration paths.
* fix: ensure session files exist when creating new session (local mode)
- Simplify logic: check if session exists and ensure it if not
- Remove must_exist parameter
- This prevents 'file not found' errors when adding the first message
* fix: ensure session files exist when creating new session
Qin Haojie