mirror of
https://github.com/volcengine/OpenViking.git
synced 2026-10-01 01:38:07 +08:00
Jiahui ZhouandTRAE CLI
f857349d6b
The bytes_row STRING type uses a uint16 length prefix, capping a single
field at 65535 bytes. Add a new TEXT field type (enum value 9) that mirrors
STRING semantics (utf-8 str round-trip) but uses a uint32 length prefix,
lifting the per-field limit to ~4GB.
TEXT is added only at the physical bytes_row layer, across all serializers
that must stay byte-identical: the C++ engine (bytes_row.h/.cpp), the abi3
boundary (abi3_engine_backend.cpp, decoding to str not bytes), the pure
Python fallback (store/bytes_row.py), and the engine API (_python_api.py).
Existing types and the CandidateData.fields field are untouched, so old
on-disk data stays readable without reindex.
Fields opt into the new type via metadata={"field_type": FieldType.text}.
Add TestTextFieldType covering >65535-byte round-trips, py<->cpp cross
read/write, binary consistency, and metadata-based declaration.
Co-authored-by: TRAE CLI <noreply@bytedance.com>