* feat(resources): support source headers for HTTP imports
Allow HTTP(S) resource imports to pass request headers for authenticated object storage sources. Fetch authenticated sources into a request-scoped snapshot and keep credentials out of parser inputs and queued jobs.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
* fix(resources): reserve source headers from args
Treat source_headers as a top-level add_resource field so it is rejected from args and filtered consistently from queued processor inputs.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
* feat(resources): accept TOS auth through args
Replace generic source header passthrough with explicit tos_signature and tos_access args for HTTP TOS object imports. Keep credentials request-scoped and out of parser and queue payloads.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
* refactor(resources): name transient TOS args
Centralize TOS credentials that are accepted from args but excluded from durable resource jobs.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
---------
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
* feat: add freshness-aware parent aggregation
Defer wide-directory abstract/overview regeneration until the configured freshness threshold is reached while continuing changed-file semantic and vector processing.
Persist freshness metadata atomically, make parent bubbling L0-aware, preserve separate semantic/vector statuses, and keep explicit waits synchronous.
Rebuild every sampled summary on threshold refresh and always retry directory vectorization so stale sidecars or transient vector failures cannot be silently accepted.
Add focused coverage for freshness policy, pending-state consumption, sampled-summary refresh, vector retries, and parent bubbling.
Co-authored-by: TRAE CLI <traecli@bytedance.com>
* feat: ov reindex support --recursive
* feat: ov reindex support --recursive
* feat: ov reindex support --recursive
* feat: ov reindex support --recursive, and applied to memory
* feat: ov reindex support --recursive, and applied to memory
---------
Co-authored-by: TRAE CLI <traecli@bytedance.com>
* perf(resource): make wait=false ingestion durable
Move source preparation and parser work behind the durable AddResource task
boundary, while keeping task-owned credentials private and short-lived.
* fix(feishu): preflight add-resource roots
* test(fs): align tree rel_path fakes with binding
* feat(pdf): refactor MinerU parsing to the official file_parse API
* feat(pdf): remove mineru_api_key from configuration and examples
* feat(pdf): preflight MinerU /health during service initialization
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(reindex): support tag updates
Add replace and append tag modes to reindex vector rebuilds, preserve omission-aware behavior, and propagate options through background tasks and namespace rebuilds. Align Python, TypeScript, Go, and CLI interfaces with tests and documentation.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(reindex): lock file targets exactly
Use an exact path lock for existing file targets while retaining tree locks for directories and prune-orphans scopes. Add a regression test for single-file reindex.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(reindex): handle prune file targets
Use exact locks for existing file targets in prune-orphans mode while retaining tree scope for missing targets. Document the existing Go ReindexOptions wait semantics and add lock regression coverage.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
---------
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(watch): avoid global lock during resource moves
Allow unrelated watch operations to continue while viking_fs.mv is running, while serializing overlapping resource paths with a move fence. Preserve transaction integrity across persistence failures, rollback, and caller cancellation.
* feat(resource): add URI mutation coordinator
* refactor(watch): separate target rewrites from resource moves
* refactor(fs): own resource move watch transaction
* feat(watch): coordinate refreshes with URI mutations
* refactor(core): share URI mutation coordinator
* test(watch): focus URI move coverage
* test(watch): reuse existing URI move coverage
---------
Co-authored-by: chenpengfei <chenpengfei@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
Keep expired terminal tasks cached when persistent deletion fails so cleanup can retry without resurrecting stale records after restart.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): support event tag filtering
Add session-level default event tags, commit-time overrides, durable queue propagation, and first-write vector index tagging. Include config update APIs and coverage for serialization, concurrency, extraction, and HTTP behavior.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* feat(memory): expose event tags in SDKs and CLI
Add session default tag configuration, config updates, and commit-time event tag overrides across embedded Python, standalone Python, TypeScript, Go, and the Rust CLI. Preserve explicit empty-tag semantics and document each public interface.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(sdk): align legacy session tag APIs
Forward commit-time event tags through the legacy Python HTTP shims and align BaseClient session signatures without adding a new abstract-method requirement for existing subclasses.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* feat(session): allow updating auto-commit policy
Extend PATCH session config to atomically update event tags and auto-commit settings. Merge policy objects by field, use explicit null to disable automatic commits, preserve omitted fields, and expose the contract across SDKs and CLI.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* fix(session): align session config interfaces
Replace the generic session create config JSON flag with explicit event-tag and auto-commit options. Preserve omitted, object, and null auto-commit semantics across HTTP, embedded clients, SDKs, and CLI, reject ambiguous null policy fields, and handle nullable event configuration consistently.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* test(session): trim redundant event tag tests
---------
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
* feat(config): add enable_watch_scheduler toggle for read-only replicas
Add a top-level `enable_watch_scheduler` config flag (default true) and gate
`WatchScheduler.start()` on it in service initialization.
Read-only replicas that share a writer's data must not run the periodic
watch/refresh background loop: it would execute writes against shared storage
(vectordb / backups) and race the writer when persisting watch task state,
which has no cross-process lock. The scheduler instance is still created and
wired for on-demand read paths; only its background loop is skipped.
Defaults to true, so existing single-instance deployments are unaffected.
* fix
* fix
* fix(server): stop exporting raw query strings and buffering zip responses in observability
Sweep findings: B-03, B-13. Prevent query secrets from reaching traces and keep ZIP responses streaming.
(cherry picked from commit d8ac3dc33b)
* fix(session): tolerate missing/corrupt archive in Phase-2 replay
(NotFoundError / _ArchiveMessagesCorruptError) on a missing or corrupt
archive messages.jsonl instead of returning []. That PR added skip-on-
missing tolerance to the read path (_get_uncovered_archive_messages) and to
resume_queued_commit, but not to the Phase-2 commit replay path
(_prepare_phase2_archive_messages), which calls _read_archive_messages
unguarded while rolling earlier failed archives into the current commit.
Consequence: a terminally-failed earlier archive whose messages.jsonl is
missing/corrupt (legacy "no messages" terminal data, or produced by #3417's
own archive_read terminal path) makes every subsequent commit's Phase-2
extraction raise -> caught by _run_memory_extraction's except -> the current
archive is terminal-failed too. Because the poisoned archive is only removed
from replay once "covered" (which requires a later archive to complete), and
no later archive can ever complete, the session's memory extraction is
permanently poisoned. Raw messages are safe, but extraction is stuck.
Fix: wrap the replay-loop _read_archive_messages call in the same tolerance
_get_uncovered_archive_messages already uses -- skip + warn on not-found
(_is_storage_not_found) and on _ArchiveMessagesCorruptError, re-raise real
storage failures. The skipped archive stays in covered_failed so the current
archive's .done marks it covered, clearing the poison permanently.
Adds a regression test asserting the replay skips a failed archive with a
missing messages.jsonl (and marks it covered) instead of raising, and that a
real storage failure still propagates.
Follow-up to #3417.
(cherry picked from commit 5b8ec9e68a)
* fix(client): align client surfaces without leaking memory metadata
Reconstructs the client-parity work from upstream PR #3439 on current main and strips reserved memory metadata before line slicing in both embedded and HTTP reads.
Based-on: 48b411d58c
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* fix(index): propagate semantic vectorization failures safely
Reconstructs upstream PR #3437 on current main, carries enqueue failures through SemanticDagExecutor, and drains the attempt's embedding tracker before retry-visible failure propagation.
Based-on: 02387deb09
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
* fix(core): close privacy and embedding failure gaps
* fix(memory): strip repeated metadata trailers
* fix(core): close public memory visibility gaps
* ci: skip embedding-dependent resource test without secrets
---------
Co-authored-by: zhiheng.liu <zhiheng.liu@bytedance.com>
