SRD-007 — Process data model (data plane + execution frames)¶
| Field | Value |
|---|---|
| Status | Accepted |
| Version | v.1 |
| Date | 2026-06-12 |
| Owner | Ruslan Gabitov |
| Implements | ADR-010 v.1 Process Data Model |
| Refines | ADR-001 v.5 Execution Model |
This SRD lands ADR-010: persistent data moves into per-instance container scopes owned by a dedicated data-plane component with whole-operation atomicity; each node execution works on an execution frame keyed by (track, node); nodes keep only immutable data definitions plus their ADR-009 lifetime state. It retires the architecture audit's §1.2 critical data race structurally and sheds the Instance's scope role (audit §2.3, first step).
1. Background & motivation¶
1.1 Current state (verified against the code)¶
Storage and locking are broken in four distinct ways:
Instance.addData(instance.go:505-532) splits a read-modify-write across lock acquisitions:RLocklookup → unlock → unlocked mutation of the shared inner map →Lockwrite-back. Two tracks adding data concurrently lose updates or fault the runtime (audit §1.2).Instance.getData(instance.go:537-571) readsinst.scopeswith no lock at all, including its parent-walk loop.Instance.AddData(instance.go:716-738) takesinst.m.Lock()and then callsaddData, which takesinst.m.RLock()— Go'sRWMutexis not reentrant, so this self-deadlocks by construction. Its only production caller isUserTask.Exec(user_task.go:202re.AddData(ut, dd...)); it survives only because every test mocks the scope (user_task_test.go:93).Instance.ExtendScope(instance.go:812-842) is a TOCTOU:RLockexistence check, drop lock,Lockinsert, thenRegisterDataoutside any lock.
The execution data model contradicts itself:
- A node's working data is registered under a name-keyed path
(
/<process>/<nodeName>) bytrack.prepareNodeExecution→ExtendScope→RegisterData(track.go:506-516,instance.go:835), and torn down by a deferredLeaveScopethat fires beforeExeceven runs (the defer is insideprepareNodeExecution,track.go:514) — node reads succeed only because the unlockedgetDatawalks up to the root after the node path is gone. - Execution data lives in mutable node fields:
activity.dataPath(activity.go:29, writtentask.go:129),Event.dataPath(event.go:125, writtenstart.go:135/end.go:158, readevent.go:430),ExclusiveGateway.scope(exclusive.go:18, written inExecexclusive.go:77, read inFindexclusive.go:163-171),UserTask.resChan(user_task.go:42, written/read only insideExec,user_task.go:194/198). ADR-009 made these per-instance; they are still per-node where they must be per-execution — loops and the Parallel fork's two-tracks-one-node crossing clobber them by construction. - Shared parameter instances are mutated in place:
IoSpec.Parameters()returns live*Parameterpointers (io_spec.go:109-123);task.LoadDatawrites into them (task.go:106,114),task.updateOutputswrites values and flips state (task.go:222-231),throwEvent.LoadDatadoes the same to shareddataInputs(event.go:381-392), andServiceTaskmutates its operation's message items (service_task.go:183-184,:209-213). One set of instances per node, shared across executions. catchEvent.UploadData(ctx)(event.go:263) has the wrong signature forscope.NodeDataProducer(UploadData(ctx, s Scope)), so the track's producer assertion (track.go:633) never matches a catch event — catch-side data upload is dead code today.
Everything is the Instance: renv.RuntimeEnvironment embeds
scope.Scope (renv.go:19), the track passes t.instance to every node
Exec (track.go:529) and to UploadData (track.go:638) — the
instance-as-scope coupling the audit's §2.3 god-object finding describes.
Runtime variables: GetData intercepts a reserved path
/<process>/RUNTIME (instance.go:751-753) and synthesizes STARTED_AT,
STATE, TRACKS_CNT parameters on demand (getRuntimeVar,
instance.go:574-618). Read today only by TestMonitoring
(instance_test.go:47-75). Owner decision (review, 2026-06-12): the feature
stays, served by the data plane as a reserved read-only subtree.
1.2 Why¶
ADR-010 v.1 decides the model; the Parallel gateway (ADR-005) made concurrent tracks routine, so every defect above sits on the execution hot path. Per the audit triage, finding §1.2 is remediated here — structurally, not by patching lock placement.
2. Goals & scope¶
2.1 Goals (in scope)¶
- G1. A dedicated data-plane component owns the container-scope tree; every operation is atomic under one lock (ADR-010 §2.2).
- G2. Every node execution works on an execution frame keyed by (track, node), with per-frame parameter/property instances, atomic batch commit on success, discard on failure (ADR-010 §2.3).
- G3. Nodes hold only immutable data definitions + ADR-009 lifetime
state: the four execution-data fields and
NodeDataLoader/RegisterDataare deleted (ADR-010 §2.4). - G4. The track hands each node a per-execution environment;
renvstops embeddingscope.Scope; theExec(ctx, renv.RuntimeEnvironment)shape is unchanged (ADR-010 §2.4). - G5.
-raceclean for concurrent tracks committing to one scope and for two tracks crossing one node; audit §1.2's code is physically deleted. - G6. RUNTIME vars survive as a reserved read-only subtree of the data plane (owner decision §1.1).
- G7. A worked example exercises the persistent-data model (none does today); ADR-010 flips to Accepted with RU twins; roadmap/SAD/cross-doc pins synced.
2.2 Non-goals (deferred)¶
- Model-layer data-flow semantics — InputSet selection order, availability
("unavailable") gating,
DataStatesemantics,IoSpec/InputSetstructural review: the planned data-flow ADR. Current evaluation behavior is preserved as-is, only re-targeted at frames. DataStore, durable persistence, rehydration — the future Persistence & State ADR.- Sub-process child scopes — the tree and
OpenScope/CloseScopeexist for them, but no sub-process lands here (roadmap WS-C4). - Scope-change subscriptions / listeners — the observability ADR.
- The events-subsystem defects (audit 1.3/1.4/1.5) — the FIX run.
3. Requirements¶
3.1 Functional¶
| # | Requirement |
|---|---|
| FR-1 | internal/scope becomes the data plane. The Scope interface (scope.go:31-64) and NodeDataLoader (scope.go:69-80) are deleted; Scope is reborn as a concrete struct owning the container-scope tree: map[DataPath]map[string]data.Data + one sync.Mutex. Operations — GetData(from DataPath, name string), GetDataByID(from DataPath, id string) (both walk parent-ward to root), Commit(at DataPath, dd ...data.Data) (atomic batch, the only writer), OpenScope(DataPath) / CloseScope(DataPath) (child containers; used for the root now, sub-processes later), Root() — each fully under the mutex; no compound operation spans acquisitions. All public methods validate every parameter (nil data, empty names, invalid paths). |
| FR-2 | scope.Frame — the per-execution working set, created by the track per node execution, identity (trackID, nodeID): per-frame input/output *data.Parameter instances built from the node's IoSpec definitions via NewParameter(name, iae.Clone()) (io_spec_obj.go:52, item.go:321), per-frame property instances (activity/event properties — visible to this execution only, per the standard), and lookup that resolves frame-first then container walk-up. Frame.Commit() flushes outputs to the container scope as one Scope.Commit batch; Frame.Discard() drops everything (failure path — container scope observes nothing). |
| FR-3 | Per-execution environment. renv.RuntimeEnvironment (renv.go:16-28) drops the embedded scope.Scope and instead exposes the frame-backed data access (GetData/GetDataByID resolving frame → container; it implements data.Source for expression evaluation). The track constructs it per execution (instance services + frame); Exec(ctx, renv.RuntimeEnvironment) signatures across nodes are unchanged — the value becomes per-execution (today t.instance, track.go:529). |
| FR-4 | Track hooks rework (track.go:494-546): prepareNodeExecution creates the frame and runs the consumer role (input loading into the frame); the frame lives through Exec; finalizeNodeExecution runs the producer role and Frame.Commit(); failure/termination paths call Frame.Discard(). The Leave-before-Exec defect disappears with ExtendScope/LeaveScope themselves. |
| FR-5 | Node migration — activities. task: delete dataPath + RegisterData (task.go:128-151); LoadData/UploadData re-target the frame's parameter instances (today's in-place mutation of shared IoSpec pointers at task.go:106-114, :222-231 ends). ServiceTask: per-execution operation message items (frame-held instances; service_task.go:183, :209-213 re-targeted). UserTask: resChan becomes an Exec-local (written/read only inside Exec today, user_task.go:194/198); the result data goes to frame outputs committed by the track — the re.AddData self-deadlock path (user_task.go:202) is deleted with AddData itself. |
| FR-6 | Node migration — events. Event.dataPath + RegisterData deleted (event.go:124-125, start.go:134-138, end.go:157-161); throwEvent.LoadData fills frame input instances (today event.go:381-392 mutates shared maps); emitEvent's scope read (event.go:430) resolves via the per-execution environment; catchEvent.UploadData gets the producer-role signature and goes live (today dead: wrong signature, event.go:263), committing catch outputs through the frame. |
| FR-7 | Node migration — gateways. ExclusiveGateway.scope field deleted (exclusive.go:17-20); Find (exclusive.go:159-171) is served by the per-execution environment's data.Source instead of a captured node field. ParallelGateway unchanged — arrived+mu is ADR-009 lifetime state (parallel.go:18-22). |
| FR-8 | Instance sheds the scope role. Delete: scopes/rootScope/runtimeScope fields (instance.go:101-107), all eight scope.Scope methods (instance.go:699-874), addData/getData/getRuntimeVar (instance.go:505-618), the scope.Scope interface assertion (instance.go:898). The Instance owns a *scope.Scope created at New; loadProperties (instance.go:196-227) becomes a Scope.Commit of process properties into the root scope. |
| FR-9 | RUNTIME reserved subtree. The data plane serves /<process>/RUNTIME read-only: STARTED_AT, STATE, TRACKS_CNT (names per instance.go:38-46), fed by an instance-provided supplier so values stay live; writes to the reserved path are rejected. TestMonitoring (instance_test.go:47-75) migrates to the data plane. |
| FR-10 | Mocks & contracts. .mockery.yaml (:8-16): drop Scope/NodeDataLoader mocks, keep/adjust NodeDataConsumer/NodeDataProducer to their new signatures, regenerate mockrenv for the slimmed RuntimeEnvironment. internal/exec/doc.go §"Data management" (doc.go:70-100) rewritten for the frame protocol. |
| FR-11 | Worked example. New examples/process-data/: a process with properties and data objects flowing through service-task I/O across a Parallel fork — both branches read container data, write distinct results, the end event's throw data shows the committed values. Exits 0 and prints the expected data; CI builds it like the existing examples. |
3.2 Non-functional¶
| # | Requirement |
|---|---|
| NFR-1 | -race clean: concurrent Commit/GetData from many goroutines, and the two-tracks-one-node crossing via the Parallel fork. |
| NFR-2 | Whole-operation atomicity is testable: no interleaving can observe a partially-applied batch commit. |
| NFR-3 | No behavior change for data-less processes: existing tests and all four examples pass unmodified (except where they assert deleted API). |
| NFR-4 | diff-coverage ≥95 % on touched files (target 100 % per the project standard); make ci green. |
| NFR-5 | Every new public API validates all parameters with self-identifying errors (project rule); exported symbols carry doc comments. |
4. Design & implementation plan¶
4.1 The data plane (internal/scope)¶
// Scope is the per-instance data plane: the container-scope tree and the
// single authority for persistent process data (ADR-010 §2.2).
type Scope struct {
m sync.Mutex
root DataPath
scopes map[DataPath]map[string]data.Data
rt RuntimeVarsSupplier // reserved /RUNTIME subtree (FR-9)
}
func New(root DataPath, rt RuntimeVarsSupplier) (*Scope, error)
func (s *Scope) Root() DataPath
func (s *Scope) GetData(from DataPath, name string) (data.Data, error)
func (s *Scope) GetDataByID(from DataPath, id string) (data.Data, error)
func (s *Scope) Commit(at DataPath, dd ...data.Data) error
func (s *Scope) OpenScope(p DataPath) error
func (s *Scope) CloseScope(p DataPath) error
DataPath and its methods (datapath.go) survive unchanged — confined to
container addressing. The interface→struct rebirth keeps the package and the
name (conception leads, naming follows); nodes never see the struct — they
see the per-execution environment.
4.2 The frame (internal/scope)¶
// Frame is the working set of one node execution by one track
// (ADR-010 §2.3): per-execution parameter and property instances,
// frame-first lookup, all-or-nothing commit.
type Frame struct {
trackID string
nodeID string
at DataPath // the containing container scope
inputs []*data.Parameter // per-frame instances (NewParameter + IAE.Clone)
outputs []*data.Parameter
props []data.Data // node property instances
scope *Scope
}
func NewFrame(trackID, nodeID string, at DataPath, s *Scope) (*Frame, error)
func (f *Frame) GetData(name string) (data.Data, error) // frame → container walk
func (f *Frame) GetDataByID(id string) (data.Data, error)
func (f *Frame) Put(dd ...data.Data) error // node-produced values
func (f *Frame) Commit() error // one Scope.Commit batch
func (f *Frame) Discard()
Carriers that never cross the prepare/execute/finalize boundary
(UserTask's response channel, the gateway's evaluation source) become
Exec-locals — per-execution by construction, nothing on the node. The frame
carries what crosses stage boundaries: I/O parameter instances, properties,
the ServiceTask operation's per-execution message items.
4.3 The per-execution environment¶
internal/renv.RuntimeEnvironment keeps engrenv.EngineRuntime,
InstanceID(), EventProducer(), RenderRegistrator() and replaces the
embedded scope.Scope with the frame-backed surface (GetData,
GetDataByID; it implements data.Source so expression evaluation reads
through it — replacing ExclusiveGateway.Find's captured-scope pattern).
The track builds one per execution: instance services + frame. Mocks
regenerate; every node Exec body that called re.GetDataByID(st.dataPath,…)
now calls re.GetDataByID(…) — the path argument disappears with the
node-held DataPath.
4.4 Consumer/producer roles¶
scope.NodeDataConsumer / scope.NodeDataProducer survive with
frame-oriented signatures (exact shape pinned in M3 code review; the contract
is: consumer fills the frame's inputs from its associations, producer yields
outputs for the track to commit). catchEvent.UploadData aligns to the
producer signature and becomes reachable (FR-6).
4.5 Milestones (each = one commit, independently CI-green)¶
Ordering note. Replacing the embedded
scope.Scopeinrenvripples through every node at once — so new types land first (unwired, M2…M3), and the switch-over is one atomic milestone (M4), mirroring SRD-006's M1 all-at-once interface change.
- M2 — data plane. The concrete
Scope+RuntimeVarsSupplier(FR-1, FR-9) with full unit tests: atomicity, walk-up resolution, batch commit, open/close, reserved-subtree read-only, parameter validation, and the concurrent-racetest (NFR-1/2). Old code untouched; both coexist. - M3 — frame.
scope.Frame(FR-2) + tests: per-frame instances (mutating a frame's parameter leaves the node'sIoSpecdefinitions and other frames untouched), frame-first lookup, commit batch, discard. Unwired. - M3a — id-generator concurrency fix (discovered at M2, owner-approved
addition):
foundation's id generator drove an unsynchronizedmath/rand.Randand lazily initialized its package variable with a racy check-then-assign — concurrent model-element construction was a data race. Fixed (goroutine-safe top-levelrand, eager init,RWMutexover the generator swap/fetch) with a-raceregression test. Hard precondition for M4: frames instantiate parameter instances from concurrent tracks, so every node crossing generates ids concurrently. - M4 — switch-over.
renvslims (FR-3); track hooks rework (FR-4); node migration (FR-5/6/7); Instance sheds scope (FR-8); deletions (NodeDataLoader, oldScopeinterface + Instance methods, node fields, clone-resets that zeroed them —activity.go:96,event.go:166,exclusive.go:52,user_task.go:151); mocks +.mockery.yaml+exec/doc.go(FR-10); all existing tests migrated;TestMonitoringmoves to the data plane. - M5 — acceptance.
examples/process-data(FR-11); all examples smoke;make ci; ADR-010 → Accepted + RU twin; SRD-007 §7 filled; roadmap/SAD sync; cross-doc pins audit (/check-srd).
5. Verification (Definition of Done)¶
| # | Check | Expectation |
|---|---|---|
| V1 | Unit: every Scope operation atomic; walk-up resolution; batch commit all-or-nothing; open/close; invalid-parameter rejection (FR-1). |
green |
| V2 | -race: N goroutines hammering Commit+GetData on one Scope; no race, no lost batch (NFR-1/2). |
-race clean |
| V3 | Frame isolation: two tracks crossing one node via a Parallel fork get distinct frames; both commit; container state consistent; node IoSpec definitions unmutated (FR-2, G5). |
no clobber, -race clean |
| V4 | Frame failure: a failing node execution leaves zero trace in the container scope (FR-2). | nothing committed |
| V5 | Per-kind migration: task/ServiceTask/UserTask/Start/End/Exclusive each load-execute-commit through frames; catch-event upload live; UserTask result reaches the container scope through the frame (FR-5/6/7). | green |
| V6 | RUNTIME subtree serves live STARTED_AT/STATE/TRACKS_CNT, rejects writes; TestMonitoring migrated (FR-9). |
green |
| V7 | Regression: existing suites + all four examples pass; data-less execution behavior unchanged (NFR-3). | green |
| V8 | examples/process-data exits 0 printing the expected committed values (FR-11). |
green |
| V9 | make ci green — race tests, diff-coverage ≥95 % touched, govulncheck (NFR-4). |
pass |
| V10 | Docs: ADR-010 Accepted + RU twin; roadmap/SAD synced; pins audited (G7). | done |
6. Risks & regressions¶
- M4 blast radius. The
renvslim-down touches every node kind, the track, the Instance, mocks, and tests in one commit. Mitigation: M2/M3 land the new world fully tested first; M4 is mechanical re-targeting against stable contracts; the per-kind V5 tests are written with M4. - Concurrent construction beyond the data plane. Frame instantiation
builds model elements (parameters, item-aware elements) from concurrent
track goroutines — every shared global in the construction path is a race
candidate. The id generator was the confirmed instance (fixed in M3a);
V3's
-racecrossing test is the standing guard for the rest of the path. - Clone fidelity of parameter instances.
ItemAwareElement.Clone()(item.go:321) must give value-independent copies for everyvalues.Valuekind a frame can carry; a shallow spot would silently re-share state. V3 asserts definition-vs-frame independence explicitly. - Expression-engine contract.
ExclusiveGatewayconditions read variables viadata.Source.Find(exclusive.go:142,159-171); the environment must preserveFind's resolution semantics (name lookup from the gateway's scope position) or conditions silently stop seeing data. Covered by the existing gateway tests re-targeted in M4. - ServiceTask operation messages. The per-execution message instances
must keep
Operation.Run's contract (operation.go:202); thegooper-functor path (used by both examples) is the regression canary (V7). - Behavioral note (intended): values committed by a node become visible to other tracks only at frame commit — today's mid-execution shared-map visibility narrows to the standard's copy/commit semantics. No current test or example depends on mid-execution visibility (verified: examples carry no data; tests mock the scope).
7. Implementation summary¶
Landed on branch feat/process-data-model (off master).
Milestone commits
d30dd2c— M2: the data plane (internal/scopeScopestruct — landed under the transitional exported namePlane, renamed at M4 — with whole-operation atomicity, batchCommit, reserved RUNTIME subtree).62dfc76— M3:scope.Frame(per-frame instances viaIAE.Clone, frame-first lookup, all-or-nothing commit, discard-on-failure).389f075— M3a: race-free id generation (foundation— top-levelmath/rand, eager init,RWMutexover the generator swap).fbfc018— M4: the atomic switch-over (legacyScopeinterface +NodeDataLoaderdeleted;renvslimmed to the per-execution surface;execEnv; track frame lifecycle; all node kinds migrated; Instance owns the plane and supplies RUNTIME vars; mocks regenerated).6c86620— M5:examples/process-data(FR-11).
Contract details pinned during implementation (per §4.4 the exact role signatures were an M3/M4 concern):
NodeDataConsumer.LoadData(ctx, *scope.Frame)/NodeDataProducer.UploadData(ctx, *scope.Frame).- Frame resolution order: inputs → properties → puts → container walk.
Output instances never resolve — they are write targets; resolving
them would let a not-yet-filled output shadow the data meant to fill it
at the producer stage (
updateOutputspulls the fill value by the output's own ItemDefinition id). - A
DataInputfilled by its association flips its frame instance to Ready (BPMN §10.4.2) — before, nothing ever performed the flip, so the real input path could not complete (it existed only behind scope mocks). ServiceTaskruns its operation on a per-execution clone; the result reaches the frame as a put keyed by the outgoing message item id. The pre-existing message flow was INVERTED (the output was copied into the operation message; the run result never reached the output) — fixed, with a missing output declaration no longer an exec-time error (model validation belongs to the data-flow ADR).catchEvent.UploadDatawent live (its old signature never satisfied the producer role); output associations still have no binding API (message-correlation work, WS-C3).
Discovered and fixed beyond the plan
- M3a (owner-approved insert): the id generator was a triple data race —
unsynchronized
rand.Rand, racy lazy init, unsynchronized setter. - The legacy
Instance.AddDataself-deadlock and the Leave-scope-before-Exec defect died with their code (§1.1).
Verification results
| # | Result |
|---|---|
| V1 | 🟢 scope-package suite: atomicity, walk-up, batch all-or-nothing, open/close, reserved subtree, validation. |
| V2 | 🟢 -race hammer (8 writers + 8 readers, no lost commit). |
| V3 | 🟢 TestDataCrossingParallelFork — two forked tracks commit per-branch results; -race clean. |
| V4 | 🟢 TestFrameDiscardOnFailure — a failing execution leaves zero container trace. |
| V5 | 🟢 per-kind suites: frame-based task data flow incl. error paths; throw/catch association branches (white-box); TestCatchEventUploadData (live producer role); TestExecuteNodeFailureStages. |
| V6 | 🟢 RUNTIME subtree (reserved, read-only, live values; TestMonitoring via RuntimeVar). |
| V7 | 🟢 full suite green (450+); all pre-existing examples run. |
| V8 | 🟢 examples/process-data exits 0 printing both committed results. |
| V9 | 🟢 make ci green; diff-coverage 95.1 % of 628 changed lines (gate ≥95). |
| V10 | 🟢 ADR-010 + SRD-007 → Accepted; roadmap synced; cross-doc pins audited (no downward refs; all pinned versions exist). |
8. References¶
- ADR-010 v.1 Process Data Model — the decision this lands.
- ADR-001 v.5 Execution Model — the two-layer runtime; the event loop stays out of the data plane.
- ADR-009 v.1 Per-instance node graph — lifetime state stays on nodes; this SRD removes only execution data.
- ADR-005 v.1 Gateways & Joins — the Parallel fork used by V3's crossing test.
- Architecture audit 2026-06-11
(
docs/audit/architecture-audit-2026-06-11.md) — §1.2 (retired here), §2.3 (first step here). - BPMN 2.0 §10.4 / §13.3.2 via
docs/bpmn-spec/semantics/data.md— copy semantics and lifecycle-synchronous binding the frame implements.
9. Open questions¶
- None. The RUNTIME-vars disposition was decided by the owner during review (keep, reserved subtree — FR-9); frame identity and failure semantics are decided in ADR-010; the exact consumer/producer signatures are pinned at M3 code review within the contract stated in §4.4.
Document History¶
| Version | Date | Author | Change |
|---|---|---|---|
| v.1 | 2026-06-12 | Ruslan Gabitov | Draft. Lands ADR-010: internal/scope reborn as the concrete data plane (container tree, one mutex, whole-operation atomicity, batch Commit, reserved read-only RUNTIME subtree); scope.Frame keyed (track, node) with per-frame parameter/property instances (NewParameter + IAE.Clone), frame-first lookup, all-or-nothing commit, discard-on-failure; renv drops the embedded scope.Scope for a frame-backed per-execution environment (also the data.Source for expressions); track hooks create/commit/discard frames; nodes lose dataPath/captured scope/RegisterData/NodeDataLoader (the AddData self-deadlock and the dead catchEvent.UploadData go with them); Instance sheds its eight scope methods and owns the data plane. Five milestones: data plane → frame → atomic switch-over → acceptance with a new examples/process-data. |
| v.1 | 2026-06-12 | Ruslan Gabitov | Accepted. Landed across M2–M5 (d30dd2c / 62dfc76 / 389f075 / fbfc018 / 6c86620). Contract details pinned during landing (folded into §7): frame role signatures take *scope.Frame; frame resolution excludes output instances (write targets); association-filled inputs flip Ready (BPMN §10.4.2); ServiceTask runs a per-execution operation clone with the result as a frame put — fixing the pre-existing inverted message flow; catchEvent.UploadData went live. M3a inserted (id-generator data race, discovered at M2). make ci green; diff-coverage 95.1 % (gate ≥95). |