Router.predict_batch dispatched on_predict_start, on_error and on_predict_end
without a timeout, so Router(hooks_timeout=...) was silently ignored on the
batched path while it applied to predict. Thread the instance value through and
add a per-call hooks_timeout, validated like the other entry points.
Documents the lock interaction: a timed-out hook keeps running outside the
hooks_concurrent=False lock, so an overrunning hook no longer blocks the ones
behind it.
Follow-ups from a review of this PR:
- `run_coroutine_sync(coro, loop=...)` now raises `ValueError` when the loop
is not running, and when it is the calling thread's own loop. Both cases
previously blocked forever (`run_coroutine_threadsafe(...).result()`) with
no coroutine ever scheduled or no thread able to run it. The docstring said
the loop must not be the calling thread's; it did not say it must be running.
- `hooks_timeout` must now be positive. `0` and negatives are rejected with
`ValueError` at construction and per call, instead of racing on a
zero-length `thread.join`. Added `laya.hooks.validate_timeout`, used by the
constructors and the per-call overrides on `Agent`, `ONNXAgent` and `Router`.
- A hook that runs under a timeout now runs in a copy of the caller's
`contextvars` context, so a request id or tracing span set by the caller is
visible to the hook.
- `AsyncHook` rejects an object implementing none of the hook events, matching
`normalise_hooks`.
Tests pin each of these; docs note the loop and timeout requirements and the
thread-growth caveat. The thread-per-call model is unchanged: a hung hook
still keeps its thread, which the docs now state more directly.
- AsyncHook wraps an async hook so its coroutine events run to completion in the
sync core; dispatch also runs any awaitable a hook returns, so a plain async
callable works too. A caller that already runs a loop uses a background loop
instead of asyncio.run, to avoid deadlock.
- dispatch gains timeout, bounding each hook call in seconds; a hook that
overruns raises TimeoutError (or warns with hooks_raise=False). Agent, Router
and ONNXAgent take hooks_timeout at the instance level and override it per call
on the predict surfaces.
- Tests and docs updated (api, errors, examples, index).
* feat(hooks): opt-in prediction hooks to observe or shape every decision
Add on_predict_start/on_predict_end and a Hook protocol to Agent, Router and
ONNXAgent, plus on_route/on_load/on_evict on the Router. Hooks receive a mutable
PredictContext: a start hook can redact or rewrite the state/questions, or call
ctx.skip(...) to serve a cached result and skip the forward pass; an end hook can
rewrite the results. Unset hooks are a no-op, so the default path is unchanged.
Adds tests/test_hooks.py (50 checks, no weights), examples/hooks/ and
docs/hooks.md.
* fix(hooks): run start/error/end on start-hook failure; expose model id; per-call on_route
Review pass:
- A failing on_predict_start hook now still runs on_error and on_predict_end,
instead of aborting before the try/finally.
- Agent and ONNXAgent populate ctx.model from the checkpoint id, matching the
documented context.
- Router per-call hooks= now apply to on_route too (and route() takes hooks and
hooks_raise), so a per-call hook covers the whole call.
- normalise_hooks rejects a class instead of an instance with a clear error.
Adds 9 checks to tests/test_hooks.py (59 total).
* test(hooks): regression + API-stability tests; fix router cache-hit routing
- tests/test_hooks_api.py pins the hook surface (parameter names/defaults,
PredictContext fields, lifecycle events, exports, class defaults) so an
accidental API change fails CI.
- Regression checks for the review-pass fixes, plus hooks_concurrent storage.
- Router.predict now adds routing on a cache-hit skip, keeping its documented
return contract.
- PredictContext uses identity equality/hash so a hook can store it in a set.
- normalise_hooks rejects non-callable lifecycle attributes.
* feat(hooks): add run_id for tracing; document real-world coverage
- PredictContext.run_id is a unique id shared by every hook of one call, so a
tracer can correlate start/end/error spans without its own bookkeeping.
- docs/hooks.md: prior-art mapping (browser-use on_step_*, OpenAI Agents SDK
RunHooks/AgentHooks), a tracing example, and a note that hooks are synchronous
and must not block (laya.serve runs inference on a single worker).
- tests: run_id regression + API field guard; prod scenario correlating spans.
* docs(hooks): expand hooks.md into a docs/hooks/ folder
Replace the single hooks.md with a seven-page reference: overview, full API,
lifecycle flowcharts, error handling, patterns and anti-patterns, examples and
tracing. Every public hook API is covered, links and anchors are checked by
test_packaging.py, and the README/examples links point at docs/hooks/index.md.
* feat(hooks): runtime registration, scoped install, per-call token budget
- HookRegistry mixin: add_hook / remove_hook / hooks_installed on Agent, Router
and ONNXAgent. Thread-safe; a call reads a snapshot, so mutation never disturbs
a call in flight.
- PredictContext.max_len / head_max_len: a start hook, or a per-call max_len= /
head_max_len= argument, shapes the token budget for one call without touching
shared agent config. Honored by Agent and ONNXAgent, propagated from Router.
- Tests: 86 hook checks, 101 API-stability checks; docs updated (API, lifecycle,
patterns, examples).
---------
Co-authored-by: Nandakishor <48623612+NandhaKishorM@users.noreply.github.com>
Emiliano Gandini Outeda