The bytes_row STRING type uses a uint16 length prefix, capping a single
field at 65535 bytes. Add a new TEXT field type (enum value 9) that mirrors
STRING semantics (utf-8 str round-trip) but uses a uint32 length prefix,
lifting the per-field limit to ~4GB.
TEXT is added only at the physical bytes_row layer, across all serializers
that must stay byte-identical: the C++ engine (bytes_row.h/.cpp), the abi3
boundary (abi3_engine_backend.cpp, decoding to str not bytes), the pure
Python fallback (store/bytes_row.py), and the engine API (_python_api.py).
Existing types and the CandidateData.fields field are untouched, so old
on-disk data stays readable without reindex.
Fields opt into the new type via metadata={"field_type": FieldType.text}.
Add TestTextFieldType covering >65535-byte round-trips, py<->cpp cross
read/write, binary consistency, and metadata-based declaration.
Co-authored-by: TRAE CLI <noreply@bytedance.com>
* perf(vectordb): coalesce auto cuVS rebuilds during bulk ingest
Add an opt-in bulk-ingest maintenance scope that coalesces Auto cuVS background rebuilds across multiple write batches.
- defer derived GPU maintenance until the outermost bulk scope exits while keeping native writes and persistence visible per call
- harden the background worker against debounce, generation, shutdown, and stale-candidate races
- preserve suspension across index replacement and retire replaced workers
- wait for the final Auto GPU snapshot before vectordb_perf records search QPS
- document that the scope is non-transactional and only schedules readiness on exit
Auto cuVS and background rebuild remain disabled by default. Native CPU and remote backends use no-op hooks, so their existing behavior and dtype are unchanged.
* fix(vectordb): reject stale index replacements
* fix(vectordb): harden bulk rebuild lifecycle
---------
Co-authored-by: Yuanqing Zhao <2604121+yuanqingz@users.noreply.github.com>
读取路径容错：search_by_vector 对每条 candidate 的 fields 做防御性 json.loads，
跳过无法解析的损坏数据（例如被存储层 uint16 长度前缀截断的不完整 JSON），并同步
过滤 cands_list / pk_list / scores_list 保持对齐，避免单条坏数据让整批查询抛异常、
进而被上层 index backend 静默降级为空召回。
When a single candidate's `fields` JSON is corrupted, `json.loads` raised
JSONDecodeError and failed the whole batch query; the index backend then
swallowed it and returned [], silently dropping the entire recall. We now skip
the bad candidates while keeping cands_list / pk_list / scores_list aligned,
mirroring the existing None-skip logic right above. Also normalize the index
backend error log to `logger.error(..., exc_info=True)` instead of an inline
traceback dump.
- local_collection.search_by_vector: defensive json.loads + aligned filtering
- viking_vector_index_backend.query: logger.error(..., exc_info=True)
- tests: add regression test test_search_skips_candidate_with_corrupted_fields
Co-authored-by: chenpengfei <chenpengfei@bytedance.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* lisence: change the main lisence from Apache-2.0 to AGPL-v3
* lisence: change the main lisence from Apache-2.0 to AGPL-v3
* lisence: change the main lisence from Apache-2.0 to AGPL-v3
---------
Co-authored-by: openviking <openviking@example.com>
* refactor(sandbox): remove docker/aiosandbox backends, simplify SRT config
- Remove docker and aiosandbox from available backends
- Remove settings_path from SrtBackendConfig (now auto-generated in workspace)
- Update SRT settings path to workspace/sandboxes/{session}-srt-settings.json
- Update README examples to use srt backend and remove settingsPath
* refactor(sandbox): remove docker/aiosandbox backends, simplify SRT config
- Remove docker and aiosandbox from available backends
- Remove settings_path from SrtBackendConfig (now auto-generated in workspace)
- Update SRT settings path to workspace/sandboxes/{session}-srt-settings.json
- Update README examples to use srt backend and remove settingsPath
* fix: remove unused handle_chat_direct function and fix unused logs variable
* Fix UTF-8 issues in chat command
* Add tab indentation to Think, Calling, and Result lines in CLI output
* Add first release workflow
* Update release workflow with correct working directory
* 修改 SessionKey 构建逻辑：统一使用 type="cli"，channel_id 默认 "default"，chat_id 作为 session_id 使用
* Implement machine unique ID as default session ID for ov chat
* Remove unsupported --logs parameter from chat command
* 统一 Python 和 Rust CLI 的默认 session ID 生成逻辑
* 修改日志
* 去掉log依赖
* docs: add VikingBot quick start section to READMEs
* fix: use vikingbot chat instead of ov chat in READMEs
* Revert "fix: use vikingbot chat instead of ov chat in READMEs"
This reverts commit 59f4e87ba0.
* fix: use UUID v4 for machine ID generation in both Rust and Python
* refactor: move truncate_utf8 to utils, fix chat history path, and use BotProcess dataclass
* refactor: update machine ID generation and remove unused chat_v2
- Update Python to use py-machineid library
- Update Rust to use machine-uid crate
- Remove unused chat_v2.rs
- Move machine ID from file storage to system-provided IDs
- Add fallback to "default" if system ID is unavailable
* 优化格式
* ruff format .
* feat: 增加对私部vikingdb的支持
* feat: add search_with_sparse_logit_alpha (#71)
* refactor: Refactor S3 configuration structure and fix Python 3.9 compatibility issues (#73)
* refactor: Refactor S3 configuration structure and fix Python 3.9 compatibility issues
- Extract S3-related configurations from AGFSConfig into a new S3Config class
- Update agfs_manager.py to use the new S3Config structure
- Modify test_config_validation.py to adapt to the new configuration structure
- Update configuration example files to reflect the new configuration structure
- Fix Python 3.9 compatibility issue in embedding_queue.py by replacing `EmbeddingMsg | None` with `Optional[EmbeddingMsg]`
* fix: validate_config validate error
* fix: fix ci (#74)
* refactor: unify async execution utilities into run_async (#75)
- Create centralized run_async() in openviking/utils/async_utils.py
- Remove duplicate _run_async from session.py
- Remove duplicate run_coroutine_sync from observers/async_utils.py
- Replace asyncio.run() calls in sync_client.py with run_async
- Update observers to use the unified run_async utility
- Update tests and documentation for new observer API
* 原生部署的vikingdb由外部来管理
---------
Co-authored-by: kkkwjx <zhoujiahui.01@bytedance.com>
Co-authored-by: baojun-zhang <zhangbaojun.1@bytedance.com>
Co-authored-by: qin-ctx <qinhaojie.exe@bytedance.com>
