* fix(parse): handle parentheses in Markdown image paths
The image regex !\[([^\]]*)\]\(([^)]+)\) used [^)]+ for the path capture
group, which truncates at the first ) character. When document titles
or filenames contain balanced parentheses (e.g. "文档_17 (17号项目)"), the
generated image paths include ) and the regex captures a truncated,
non-existent path. This causes _resolve_image_path() to fail silently
(WARNING only), and the image is never copied to VikingFS or sent to
VLM for understanding.
Fix: replace the path capture group with (?:[^()]|\([^()]*\))+, which
allows one level of balanced parentheses inside the path while still
terminating at the correct closing ) of the Markdown image syntax.
Add focused tests covering balanced parens in directory and filename
components, URLs with parens, multiple images on one line, and
non-matching of plain links.
Fixes#3455
* fix(test): exercise MarkdownParser._image_pattern directly, remove unused import
Address review feedback on #3462:
1. Tests now import and instantiate MarkdownParser to access the
production _image_pattern regex, instead of compiling an independent
copy. Tests fail if the production regex regresses.
2. Remove unused `import pytest` (Ruff F401).
* fix(parse): rewrite parenthesized image paths
* test: remove extra image rewrite regression case
* fix(parse): share markdown image parsing for rewrite
* refactor(parse): keep markdown image fix minimal
---------
Co-authored-by: zhangyu.34 <zhangyu.34@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
* fix(ov): redact gateway secrets in config show and create root key with 0600
Sweep findings: E-02, E-03. Prevent credential disclosure in output and at key creation.
* test(ov): drop trivial init-key permission test
The 0600 fix is a one-line OpenOptions::mode; a dedicated tokio+tempfile
test module for a single mode assertion is not worth its weight. The
redaction test in store.rs (a real multi-case security behavior) stays.
* fix(bot): pass sender_name in all channel adapters
Sweep findings: C-01. Supply display-name fallbacks so inbound messages reach the bus.
* fix(bot): make sender_name optional and wire real WhatsApp pushName
Root-cause guard: _handle_message required sender_name positionally, but
InboundMessage.sender_name is str|None=None and context.py already falls back
to sender_id, so the required-ness was an accidental signature/contract
mismatch. Make it optional (reordered after the still-required chat_id/content;
all 11 call sites use keyword args) so no future adapter can crash on it.
WhatsApp: the previous call read data.get("senderName")/data.get("pushName"),
neither of which the bridge ever sends, so it silently always fell back to the
numeric id. Forward baileys' msg.pushName through the bridge payload and read it
in Python, so WhatsApp group chats show real display names like other channels.
* fix(rerank): support DashScope nested request/response envelope
OpenAIRerankClient sent a flat request body ({"model", "query",
"documents"}) and parsed "results" at the top level of the response.
DashScope (qwen3-rerank) requires a nested envelope:
Request: {"model", "input": {"query", "documents"}, "parameters": ...}
Response: {"output": {"results": [...]}, "request_id", "usage"}
This caused DashScope rerank to silently fail — the response had no
top-level "results" key, so the client returned None.
Changes:
- Add _is_dashscope() to detect DashScope endpoints by host marker.
- Add _build_request_body() that produces the nested envelope for
DashScope and the flat body for standard OpenAI/Cohere services.
- Add _extract_results() that reads output.results for DashScope and
top-level results for standard services.
- Accept both "relevance_score" (singular, DashScope) and
"relevance_scores" (plural, some providers) in result items.
- Add 13 tests covering host detection, body construction, response
parsing, end-to-end mocked flows for both providers, plural key
handling, empty documents, and sparse results.
Fixes#3459
* fix(rerank): detect DashScope protocol by URL path, not hostname
Reviewer noted the previous hostname-based switch broke the documented
qwen3-rerank compatible-api endpoint (/compatible-api/v1/reranks), which
must use the flat OpenAI-style body and top-level results.
Switch to path-based detection: only /api/v1/services/rerank uses the
native nested input/output envelope; everything else (including the
DashScope compatible-api and generic OpenAI/Cohere gateways) keeps the
flat protocol. Rename _is_dashscope -> _uses_nested_envelope for clarity.
Add regression tests covering the compatible-api flat path and reconcile
the existing native-path fixtures to the nested envelope.
* docs(rerank): use qwen3-rerank for compatible-api example
The compatible-api/v1/reranks endpoint uses the flat OpenAI-compatible
protocol; qwen3-vl-rerank is a native-envelope model served at
/api/v1/services/rerank. Align the example model with the endpoint the
implementation selects by URL path.
---------
Co-authored-by: zhangyu.34 <zhangyu.34@bytedance.com>
- Add directory_marker_mode: none to all S3 config examples
- Add S3-compatible storage notes with required fields table
- Add Docker networking guidance for Linux vs macOS/Windows
- Remove private IP addresses from examples (use localhost)
- Apply changes to both English and Chinese versions
* docs: revamp README for readability, route detail to docs.openviking.ai
The README had grown to ~850 lines, 60% of it provider-config JSON that
duplicates the deployed configuration guide. Rewritten to ~253 lines:
- Lead with what the product is, a Studio screenshot, and five feature
bullets, each outlinked to docs.openviking.ai
- Move benchmarks (LoCoMo, tau2-bench, HotpotQA) above the fold; drop two
derived tables in favor of one-sentence summaries + ./benchmark links
- Collapse install to the init/doctor golden path; all provider JSON,
ov.conf templates, env vars, and Windows setup now route to the
configuration guide (verified live)
- Add the previously missing "Use it with your agent" section linking all
10 integration docs
- README_CN (zh docs links) and README_JA (en docs links; ja docs not
deployed) rewritten to mirror section-for-section
- New hero screenshot docs/images/studio-playground.png from
openviking.ai/studio
All 76 external URLs and every repo-relative link verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4oDxmJojygyomsz9BBhhn
* docs: fix review findings — restore uncovered detail, drop false pointers
Adversarial review of the revamp found four claims pointing at coverage
that does not exist:
- Restore `cargo install --git ... ov_cli` build-from-source path (was
deleted with no docs destination; docset has no cargo install anywhere)
- Restore `ov reindex` mode documentation (vectors_only /
semantic_and_vectors / prune_orphans / --dry-run / no alias warning) —
covered by no linked doc
- Remove "per-agent breakdown is in ./benchmark" (benchmark/ holds
reproduction scripts, not result tables)
- Remove "Reproduce it from ./benchmark" on the 5-dataset RAG summary
(adapters exist for only 3 of 5 datasets)
Also: EN/JA quick-start grep example now targets docs/en instead of
docs/zh. Applied identically to README.md, README_CN.md, README_JA.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4oDxmJojygyomsz9BBhhn
* docs: apply review feedback — blog philosophy link, deployed doc links, drop dead widgets
- Link the design-philosophy essay (The Database Paradigm for Context
Engineering, blog.openviking.ai) from the Why section so the old
README's design narrative has a durable home; add Blog to community
- Switch remaining ./docs about-us links (header + community, incl. QR
anchors) to docs.openviking.ai; zh anchors verified against deployed
page ids (#飞书群 / #微信群)
- Remove the star-history chart (service currently renders nothing) and
the stale "May 2026 Update" banner line
- Caption now states the Studio link is a live demo, no install needed
Applied identically to README.md, README_CN.md, README_JA.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4oDxmJojygyomsz9BBhhn
* docs: benchmark charts, easier quick start, logo padding
- Replace the three benchmark tables with one theme-aware SVG chart
(light/dark via <picture>): LoCoMo and tau2-bench as grouped bars,
gray = without OpenViking, blue = with. HotpotQA leaves the README;
full results link to the benchmark report on blog.openviking.ai.
Hand-written SVG, exact numbers from the tables — no generated images.
- Rework Quick start reading flow: nohup folds into the install block,
note that pip install already ships the ov client CLI, close with a
two-link "Next steps" (CLI setup, Deployment). ov reindex modes and
the cargo source install move to the CLI setup doc (en+zh) so the
README stays an easy entry.
- Shrink logo artwork to 0.7 inside the same 842x842 canvas for
breathing room.
Applied to README.md, README_CN.md, README_JA.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4oDxmJojygyomsz9BBhhn
* docs: enlarge logo artwork 1.1x within same canvas
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4oDxmJojygyomsz9BBhhn
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(task): recover add-resource jobs after restart
Persist asynchronous add-resource work in QueueFS so interrupted jobs can resume instead of leaving tasks running forever.
* fix(queue): omit parser args from prepared jobs
* fix(queue): fail when semantic source is missing
* fix(session): return in-flight archive messages in get_session_context (#3129)
Seed latest_completed_index from 0 instead of commit_count so that
archives whose Phase 1 has completed (messages written, commit_count
advanced) but whose Phase 2 is still running (.done not yet written)
are treated as pending rather than already completed.
The release/0.3.x implementation seeded from 0 and did not have
this bug; the regression was introduced when commit_count was
adopted as the seed value.
* test(session): deterministic regression for pending archive context (#3129)
Replace the monkey-patched commit_async test with a direct
filesystem-state test that sets up the post-Phase-1 archive
(messages.jsonl present, commit_count advanced, no .done marker)
and asserts get_session_context still surfaces the archived
messages.
This is deterministic regardless of the queue-worker architecture
because it creates the archive state directly via the mock AGFS
and loads a fresh session from that state, never calling
commit_async or touching the session compressor.
* perf(vectordb): coalesce auto cuVS rebuilds during bulk ingest
Add an opt-in bulk-ingest maintenance scope that coalesces Auto cuVS background rebuilds across multiple write batches.
- defer derived GPU maintenance until the outermost bulk scope exits while keeping native writes and persistence visible per call
- harden the background worker against debounce, generation, shutdown, and stale-candidate races
- preserve suspension across index replacement and retire replaced workers
- wait for the final Auto GPU snapshot before vectordb_perf records search QPS
- document that the scope is non-transactional and only schedules readiness on exit
Auto cuVS and background rebuild remain disabled by default. Native CPU and remote backends use no-op hooks, so their existing behavior and dtype are unchanged.
* fix(vectordb): reject stale index replacements
* fix(vectordb): harden bulk rebuild lifecycle
---------
Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
* feat(parse): add large image processing for image parser
- Add large_image_processor.py: detect large images (>10MB or >4096px),
create low-res previews, split into grid tiles, and generate grid
overlay images with tile labels
- Refactor ImageParser.parse() to integrate large image processing pipeline
- Enable SVG-to-PNG conversion in utils.py (cairosvg/wand)
- Rename ImageConfig.max_dimension to preview_max_dimension and add new
config fields: max_file_size_mb, max_tile_size_mb, max_tile_dimension_px,
tile_overlap_px, large_image_threshold_dimension
- Update ov.conf.example with new image config options
* fix(parse): correct tile dimension comment from 1024px to 2048px
* fix(parse): fix tile label path in grid overlay to include tiles/ directory
* fix(parse): register missing image extensions for ImageParser
TIFF, ICO, DIB, ICNS, SGI, JP2 were not in IMAGE_EXTENSIONS, causing
them to fallback to TextParser. All are supported by PIL.
* fix(parse): preserve PNG format for tiles instead of always converting to JPEG
* fix(parse): address review feedback for large image processing
- Wire config.image to ImageParser in ParserRegistry (was missing)
- Remove unnecessary preview creation for small images (broke LA mode PNG)
- Enforce max_tile_size_mb on tiles with quality reduction and resize fallback
- Remove 64-tile hard cap that conflicted with max_tile_dimension_px
- Add comment explaining why original file is not saved for large images
* refactor(parse): remove max_tile_size_mb as it is a soft suggestion
max_tile_size_mb was a soft constraint that was not enforced
consistently. Remove it from config, constants, and all enforcement
logic. Tile dimension (max_tile_dimension_px) remains the sole constraint.
* fix(parse): use CJK-capable font for grid overlay labels
The old font loading only tried macOS-specific paths and fell back to
PIL's default bitmap font, which cannot render CJK characters in
filenames. Add a cross-platform CJK font lookup that covers Linux
(Noto/Droid/WQY/DejaVu), macOS (PingFang), and Windows (MSYH/SimSun).
* fix(parse): convert non-VLM-supported image formats to PNG on save
Image formats like TIFF, ICO, DIB, ICNS, SGI, JP2 are not recognized
by VLM backends (OpenAI/LiteLLM/VolcEngine only support PNG/JPEG/GIF/
WebP/BMP) or by embedding_utils for image vectorization. When a file
with one of these extensions is parsed, convert it to PNG and use a
.png extension so that downstream pipelines see consistent data.
SVG files (already PNG-converted via cairosvg) also get the .png
extension for the same reason.
* fix(parse): import io for SVG conversion
---------
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
Monday