docs(schema): address CR suggestions and sync schema cache documentation

- Add release fragment .changes/1400-schema-cache-with-plugins.md and update AGENTS.md and schema-runtime-cache.md to describe isolated child process builder.
- Add explanatory comments to queryDeliverySchemaPayload for empty args delegation.
- Unify schema cache builder timeout to cli.DefaultSchemaCacheBuilderTimeout.
- Simplify writeSchemaCacheBuilderError signature.
- Reuse open repair cache backend in openRepairBackend and close superseded backend in setRepairCache.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
john
2026-09-20 01:10:37 +08:00
co-authored by Claude Opus 4.7
parent f49b06bb9d
commit c3dda4a8ea
8 changed files with 68 additions and 11 deletions
@@ -0,0 +1,5 @@
---
category: Fixed
---
- **Persistent Schema cache with plugins** (#1400) — delegates Schema cache assembly and repair to an isolated child process when plugins are present, keeping persistent cache active without inheriting plugin runtime side effects.
+1 -1
View File
@@ -2,7 +2,7 @@
category: Changed
---
- Introduce Schema delivery inside the existing Schema command of the single `dws` executable. Production does not produce or embed Schema identity at compile or release time. On supported ends (darwin/linux/windows amd64/arm64) each machine generates identity from live declarations at install or first `dws schema`, writes authenticated protobuf shards under the shared or user cache directory, and later hits verify digests then read those shards. Miss or corruption repairs from live assembly; when the shared cache exists but cannot be locked for repair (typically root-owned read-only), the repair falls back to the per-user cache and later processes reuse it. Empty local identity generates then uses the cache; it is not a permanent live-only mode. When plugins or other runtime extensions change the command surface, cache publication/repair/prewarm are skipped and command help/schema queries keep serving read-only from the cache (plugin commands never enter the Schema surface, cached or live). The POSIX installer only advertises a cross-user shared cache when the warmed artifacts are root-owned, matching the runtime's shared-path ownership rule; a cache warmed by an ordinary user under a writable custom root is reported as installing-user-only instead.
- Introduce Schema delivery inside the existing Schema command of the single `dws` executable. Production does not produce or embed Schema identity at compile or release time. On supported ends (darwin/linux/windows amd64/arm64) each machine generates identity from live declarations at install or first `dws schema`, writes authenticated protobuf shards under the shared or user cache directory, and later hits verify digests then read those shards. Miss or corruption repairs from live assembly; when the shared cache exists but cannot be locked for repair (typically root-owned read-only), the repair falls back to the per-user cache and later processes reuse it. Empty local identity generates then uses the cache; it is not a permanent live-only mode. When plugins or other runtime extensions change the command surface, cache assembly and repair are delegated to an isolated child process using the pristine pre-registration environment, keeping persistent cache active without inheriting plugin runtime side effects. Plugin commands never enter the Schema surface, cached or live. The POSIX installer only advertises a cross-user shared cache when the warmed artifacts are root-owned, matching the runtime's shared-path ownership rule; a cache warmed by an ordinary user under a writable custom root is reported as installing-user-only instead.
- Keep one complete Cobra tree for every public invocation. Compact typed metadata and shared builders reduce complete-tree allocations; process argv does not select a product factory or a utility-only tree.
- Reduce temporary allocations during Schema validation and command initialization.
- Normative notes live in `docs/rfc-schema-runtime-cache.md` only; no sibling plan/design/performance pages are kept.
+5 -3
View File
@@ -34,9 +34,11 @@ identity from this binary's live declarations, writes authenticated disk
shards, and later processes load that local identity then verify digests
before reading protobuf. A missing sidecar generates then uses the cache;
it is not a permanent live-only mode. Plugins that change the command
surface still disable cache publication, repair, and prewarm; read-only
serving of the unchanged reviewed surface continues (plugin commands never
enter the Schema surface, cached or live). Tests may also inject identity via
surface delegate cache assembly and repair to an isolated child process
using the pristine environment captured before plugin registration; the parent
process does not assemble cache directly, while reading, repairing, and
publishing authenticated cache remains fully active. Plugin commands never
enter the Schema surface, cached or live. Tests may also inject identity via
`RegisterSchemaCacheOptions`.
See `docs/rfc-schema-runtime-cache.md` for the single RFC covering the
local-identity shipping model, complete-tree performance contract, cache
+5 -5
View File
@@ -21,7 +21,6 @@ import (
"os"
"os/exec"
"strings"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cli"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/pkg/edition"
@@ -29,12 +28,12 @@ import (
const (
schemaCacheBuilderArgument = "--_dws-schema-builder=1"
schemaCacheBuilderTimeout = 30 * time.Second
)
var maxSchemaCacheBuilderStderr = 64 << 10
var (
schemaCacheBuilderTimeout = cli.DefaultSchemaCacheBuilderTimeout
schemaCacheBuilderCommand = exec.CommandContext
schemaCacheBuilderExecutable = os.Executable
schemaCacheBuilderEnvironment = cli.SchemaAssemblyEnvironmentSnapshot
@@ -50,13 +49,12 @@ var (
// RunSchemaCacheBuilder handles only the private declaration-builder process.
// It must run before normal root construction so plugin discovery is impossible.
func RunSchemaCacheBuilder(args []string, output io.Writer) (bool, int) {
_ = output
if len(args) != 1 || args[0] != schemaCacheBuilderArgument {
return false, 0
}
result, err := schemaCacheBuilderAssemble(context.Background())
if err != nil {
return true, writeSchemaCacheBuilderError(output, err)
return true, writeSchemaCacheBuilderError(err)
}
if err := cli.WriteSchemaCacheBuildResult(output, result); err != nil {
return true, 1
@@ -88,7 +86,7 @@ func buildSchemaCacheResult(ctx context.Context) (cli.SchemaCacheBuildResult, er
return cli.SchemaCacheBuildResult{Artifacts: artifacts, Identity: identity}, nil
}
func writeSchemaCacheBuilderError(output io.Writer, err error) int {
func writeSchemaCacheBuilderError(err error) int {
_, _ = fmt.Fprintf(os.Stderr, "Schema cache builder: %v\n", err)
return 1
}
@@ -135,6 +133,8 @@ func buildSchemaCacheInChild(ctx context.Context) (cli.SchemaCacheBuildResult, e
if err != nil {
return cli.SchemaCacheBuildResult{}, fmt.Errorf("resolve CLI executable: %w", err)
}
// Ensure child execution is bounded by the builder timeout even if the
// caller passes a context without an explicit deadline.
ctx, cancel := context.WithTimeout(ctx, schemaCacheBuilderTimeout)
defer cancel()
cmd := schemaCacheBuilderCommand(ctx, executable, schemaCacheBuilderArgument)
@@ -271,7 +271,7 @@ func TestCrossPlatformCoverageSchemaCacheBuilderOutputLimits(t *testing.T) {
}
func TestCrossPlatformCoverageSchemaCacheBuilderError(t *testing.T) {
if code := writeSchemaCacheBuilderError(&bytes.Buffer{}, errors.New("builder failed")); code != 1 {
if code := writeSchemaCacheBuilderError(errors.New("builder failed")); code != 1 {
t.Fatalf("error exit code = %d, want 1", code)
}
}
+12 -1
View File
@@ -30,10 +30,14 @@ import (
const (
defaultSchemaCacheLockTimeout = 250 * time.Millisecond
// DefaultSchemaCacheBuilderTimeout defines the maximum duration allowed for
// isolated schema cache generation before timing out.
DefaultSchemaCacheBuilderTimeout = 30 * time.Second
)
var (
defaultSchemaCacheBuilderTimeout = 30 * time.Second
defaultSchemaCacheBuilderTimeout = DefaultSchemaCacheBuilderTimeout
canonicalJSONMarshal = json.Marshal
compactLeafMarshal = jsonutil.MarshalIndent
// schemaCachePayloadLoadBeforeInnerLock is the test seam between the
@@ -826,8 +830,12 @@ func (r *schemaCacheRuntime) repairCacheBackend() *schemacache.Cache {
func (r *schemaCacheRuntime) setRepairCache(cache *schemacache.Cache) {
r.repairCacheMu.Lock()
old := r.repairCache
r.repairCache = cache
r.repairCacheMu.Unlock()
if old != nil && old != cache {
_ = old.Close()
}
}
func (r *schemaCacheRuntime) userCacheBackend() *schemacache.Cache {
@@ -906,6 +914,9 @@ func (r *schemaCacheRuntime) repairWithIsolatedBuilder(cache *schemacache.Cache,
}
func (r *schemaCacheRuntime) openRepairBackend() (*schemacache.Cache, error) {
if existing := r.repairCacheBackend(); existing != nil {
return existing, nil
}
opts := r.optionsSnapshot()
options := []schemacache.Option{}
if opts.Counters != nil {
@@ -752,3 +752,36 @@ func TestCrossPlatformCoverageRepairUncertainLockTimeoutFailsToIsolatedRequired(
t.Fatalf("expected requires isolated builder error, got: %v", err)
}
}
func TestCrossPlatformCoverageRepairBackendReuseAndClose(t *testing.T) {
t.Cleanup(restorePackageCLISchemaDeliveryForTest)
restorePackageCLISchemaDeliveryForTest()
home := realHomeCacheDir(t, ".dws-repair-backend-reuse-")
schemacache.UseUserCacheDirForTest(t, home)
goos, goarch := coverageCacheGOOSARCH()
r := newSchemaCacheRuntime(SchemaCacheOptions{
Enabled: true, AllowGenerate: true, Edition: "open", GOOS: goos, GOARCH: goarch,
})
cache1, err := r.openRepairBackend()
if err != nil {
t.Fatal(err)
}
cache2, err := r.openRepairBackend()
if err != nil {
t.Fatal(err)
}
if cache1 != cache2 {
t.Fatalf("expected reused repair cache, got %v vs %v", cache1, cache2)
}
cache3, err := schemacache.Open("open", schemacache.WithUserOnly())
if err != nil {
t.Fatal(err)
}
defer cache3.Close()
r.setRepairCache(cache3)
if got := r.repairCacheBackend(); got != cache3 {
t.Fatalf("expected updated repair cache, got %v", got)
}
}
+6
View File
@@ -322,6 +322,12 @@ func queryDeliverySchemaPayload(args []string) (map[string]any, error) {
if loaded := runtimeDeliveryLiveCatalog.Load(); loaded != nil {
return schemaPayloadFromLoadedCatalog(*loaded, args)
}
// Production commands route empty args to deliverySchemaOverviewPayload
// (or deliverySchemaAllPayload for --all). When queryDeliverySchemaPayload
// is invoked with empty args (e.g. in test assertions exercising the query
// loader without targeting a specific path), delegate to deliverySchemaAllPayload
// so cache-backed/isolated repair is used instead of in-process live assembly
// which is prohibited in plugin-uncertain runtimes.
if len(args) == 0 {
return deliverySchemaAllPayload()
}