Skip to content

discovery: converge live IED workflow with opened SCL static reporting - #339

Closed
masarray wants to merge 52 commits into
mainfrom
feature/discovery-static-workflow-parity-p0
Closed

masarray wants to merge 52 commits into
mainfrom
feature/discovery-static-workflow-parity-p0

Conversation

@masarray

@masarray masarray commented Sep 19, 2026 •

Copy link
Copy Markdown
Owner

Goal

Make live IED discovery converge on the same post-model workflow already used by Open SCL:

Discover IED -> canonical model -> IED Actions -> Static DataSet -> report-backed live signals.

Baseline protection

This branch starts from current main and preserves the physically proven bounded Smart Discovery / R7 engine lineage. P0.6 changes are consumer-side projection, presentation and workflow changes only; the engine semantic candidate remains isolated in ARIEC61850 PR #139 until separately validated.

P0 / P0.5 already established

  • live Discovery enters the same IED Actions workflow as Open SCL;
  • Static DataSet selection is capability-driven by the canonical model;
  • ordered DataSet membership is report projection authority;
  • configured RCB activation remains report-only with zero cyclic process polling;
  • IEC value text uses semantic presentation such as True [1], False [0], Open [01], Close [10], Intermediate [00], Bad state [11];
  • process values have compact visual differentiation without changing protocol truth.

P0.6 additions

  • value type badge is derived from declared IEC/MMS metadata only: F, I, U, B, E, DP; no numeric/name guessing is used to create the badge;
  • selected floating measurements receive bounded authoritative engineering-unit enrichment from CF units.SIUnit and units.multiplier, with inferred fallback disabled for this path;
  • unit enrichment is source-neutral and runs before RCB monitoring is armed for both Discovery and Open SCL workflows;
  • Save SCL while monitoring does not stop/disconnect/re-arm reporting; it exports the current canonical snapshot and skips optional save-time value enrichment while the report session is active;
  • unresolved structured Static DataSet memberships are resolved only through exact named schema leaves while preserving the original FCDA/FCD membership as display identity;
  • the runtime safety gate CanPublishToRuntime remains intact;
  • dedicated regression coverage locks all 12 field-observed structured Analog memberships (A, PPV, ThdA, ThdPPV phase/subphase rows) as 12 distinct publishable scalar rows.

Field target

For the physical relay used during acceptance:

  • Static members: 58
  • Selected live rows: 58
  • Analog: 22 / 22
  • Digital: 36 / 36
  • Report-backed: 58
  • Cyclic MMS process polling: 0

The 12 previously missing Analog rows are:

  • A.phsA / phsB / phsC
  • PPV.phsAB / phsBC / phsCA
  • ThdA.phsA / phsB / phsC
  • ThdPPV.phsAB / phsBC / phsCA

Required physical acceptance

CI is necessary but not sufficient. Before promotion this candidate must be physically checked for:

  1. Smart Discovery retains the accepted bounded request shape and latency.
  2. Discovery Static DataSet produces exactly 58/58 runtime rows with 22/22 Analog and 36/36 Digital.
  3. Configured RCB activation produces actual InformationReport traffic for all report-backed rows.
  4. Cyclic MMS process polling remains zero.
  5. Native type badges and authoritative units match the live model without guessed units.
  6. Save SCL succeeds while monitoring remains active.
  7. Saved SCL -> Open SCL reproduces the same DataSet member order, RCB binding and 58/58 runtime behavior.

This PR remains a field-test candidate until physical evidence is attached.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8c95fd7f29

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Services/NativeIec61850Client.cs Outdated
Comment on lines +134 to +135
if (SmartDiscoveryCaptureModeEnabled)
return await DiscoverSignalsSmartForCaptureAsync(cancellationToken, progress).ConfigureAwait(false);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Honor the fail-closed smart-discovery switch

In every ordinary build, SmartDiscoveryCaptureModeEnabled is already hard-coded to true, so this early return permanently activates the field-capture route even though evidence/SmartDiscoveryPromotion.props still sets SmartDiscoveryProductionPromoted=false and Directory.Build.targets says normal builds must remain on the legacy route until promotion authority exists. This bypasses the production gate and ships a path that explicitly defers the supplemental, sibling, equipment, reference, and unit enrichment used by the existing workflow; route activation must remain conditional on the reviewed promotion setting.

AGENTS.md reference: AGENTS.md:L172-L178

Useful? React with 👍 / 👎.

Assert.Contains("discovery.DataSetDirectories.SingleOrDefault", source, StringComparison.Ordinal);
Assert.Contains("TryVerifyStaticDataSetMemberOrder", source, StringComparison.Ordinal);
Assert.Contains("Members = modelDirectory.Members", source, StringComparison.Ordinal);
Assert.DoesNotContain("GetDataSetDirectoriesAsync", source, StringComparison.Ordinal);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reconcile the mutually exclusive static-path regression tests

This new assertion necessarily conflicts with DeterministicStaticReportPathRegressionTests.StaticPath_BypassesAdaptiveHybridPlannerAndPolling, which still asserts that the same source file contains GetDataSetDirectoriesAsync (line 14). Since this commit removes that call, a normal ARSAS.Tests run must fail regardless of runtime behavior; update or replace the existing contract in the same change so the suite expresses one directory policy.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

P0 implementation is now field-test candidate ready at afd9f21b08d6ea98759e985013216240019d8605.

Validated candidate paths:

  • Smart Discovery Field Capture run P1C: add native COMTRADE phasor and harmonics workspace views #294: success
    • artifact ARSAS-smart-discovery-pr134-win-x64 ID 10594833936
    • downloaded artifact ZIP SHA-256: 1177891b6640261a1493ddcf29b810af3269e395db06bdcaafb2f61a93837fd5
    • contained portable EXE SHA-256: e9d4f0d412107d6973e601fa03b2f5629a0e86e9d9b1146c491f0ec1fa4c6919
  • SCL Interoperability R7 run Fix IED FAT discovery and import Rev4 scope metadata #136: success
    • artifact ARSAS-r7-scl-interoperability-win-x64 ID 10594679294
    • downloaded artifact ZIP SHA-256: 47b005437710dac579aabf229bedefba50ecb770d7096a9d5fb0886492e502a2
    • contained portable EXE SHA-256: d46fa7f30d39b3641a9e697f31b0d66d9ec412fcb173187bcc7bf4ca2501256b
  • Both manifests pin ARSAS afd9f21b08d6ea98759e985013216240019d8605 and engine 648124097621046f5f127ceb1cf853fea54db730.
  • Merge Execution Guard, Mainline Readiness, Golden Budget, Golden Provenance, Repeat-Run Stability, convergence guard, and Production Promotion Guard are green.
  • Generic Build ARSAS has passed source/invariant checks, compile and application regression tests; packaging/smoke was still completing when this note was written.

P0 behavior now converges live discovery on the same post-model task flow used by an opened model:

Discover -> canonical live model -> IED Actions -> Static DataSet -> canonical ordered members -> configured RCB -> report-backed live values.

Key locked invariants:

  • completed discovery authority is reused;
  • no second DataSet-directory read after model construction;
  • canonical model member order is positional report authority;
  • live discovery snapshot is verification evidence only;
  • static mode has no cyclic MMS process polling or dynamic DataSet writes;
  • ordinary tracked source remains promotion-gated; field-capture/R7 builds install the verified smart route during the build transform.

Do not merge this PR on CI evidence alone. Physical acceptance is still required on the same IED:

  1. bounded/efficient Smart Discovery remains unchanged;
  2. Static DataSet action is available when the discovered model contains a DataSet;
  3. exact DataSet member rows appear;
  4. configured RCB activation succeeds;
  5. real InformationReport traffic updates those rows;
  6. no positional/type rejects;
  7. cyclic MMS process polling remains 0;
  8. Save SCL -> reopen produces the same DataSet/member/RCB behavior.

P1 capability-index/fingerprint work is intentionally isolated in stacked draft PR #340 so it cannot move this P0 field-test baseline.

Copy link
Copy Markdown
Owner Author

Text-value presentation regression is fixed on P0 head 4999b5a78f29070990e24933ce766e0239dae61c.

Root cause:

  • Static reporting now delivers valid semantic strings such as true, false, off, on, and intermediate-state.
  • The long-standing generic formatter handled typed bool / Dbpos values, but passed these report strings through unchanged.
  • Protocol/report evidence was correct; only operator-facing presentation lost the established IEC state context.

Fix boundary:

  • report publication now calls a report-specific presentation formatter;
  • raw report evidence is not rewritten;
  • discovery, DataSet authority, RCB activation, member ordering, and report projection are unchanged;
  • Boolean report strings normalize to True [1] / False [0];
  • DPC/Position report strings normalize contextually to Open [01], Close [10], Intermediate [00], Bad state [11];
  • generic non-position on/off enum text is deliberately not interpreted as DPC;
  • numeric/analog formatting is unchanged.

Regression tests added and confirmed present in the passing R7 test evidence:

  • FormatReportProcessValue_RestoresBooleanOperatorText
  • FormatReportProcessValue_RestoresDpcOperatorContext
  • FormatReportProcessValue_DoesNotInterpretGenericOnOffAsDpcOutsidePositionContext
  • FormatReportProcessValue_LeavesNumericMeasurementUnchanged

All P0 workflows are green on this head, including Build ARSAS, Field Capture, R7 interoperability, Merge Execution Guard, Mainline Readiness, Production Promotion Guard, Golden Budget/Provenance, Repeat-Run Stability, and convergence guard.

New R7 field-test artifact:

  • artifact ID: 10595522392
  • artifact ZIP SHA-256: 675cfbf1251425e1bbc35df71604c2e4e81835513a7183153a4c14b425c5f829
  • portable EXE SHA-256: b22a88a5ddd5cb4d1fdcf55ad00a5cad8dd07359756b0f0b1e28b487d42e5d6f
  • ARSAS commit: 4999b5a78f29070990e24933ce766e0239dae61c
  • engine commit: 648124097621046f5f127ceb1cf853fea54db730

Physical retest should verify only that the already-accepted discovery/static-report behavior remains unchanged while Live Signal Values return to the expected operator text. PR remains Draft until that check is complete.

Copy link
Copy Markdown
Owner Author

P0.6 parity work is now staged on this draft PR.

Field diagnostic root cause reproduced from the latest candidate:

  • canonical discovery already had 2 static DataSets / 58 members;
  • selection inventory represented all 58 members;
  • 14 descriptors had no primary scalar binding;
  • report-only selection therefore started with 46 runtime rows;
  • Analog coverage was 10/22 while Digital was already 36/36.

Changes now on the branch:

  • process-value badge is derived from declared IEC/MMS type only: F / I / U / B / E / DP; no numeric-shape/name guessing creates a type badge;
  • Static DataSet discovery performs bounded engineering-unit reads from exact CF units.SIUnit + units.multiplier before any RCB is armed; inferred unit fallback is disabled on this authority path;
  • Save SCL while monitoring no longer stops or reconfigures acquisition: it exports the current canonical model and skips optional save-time value enrichment;
  • structured static DataSet members are upgraded to exact named scalar runtime leaves from the authoritative schema (cVal.mag / mag / safe instantaneous fallback) while retaining the literal FCDA/FCD identity and keeping the existing CanPublishToRuntime safety gate;
  • the 12 missing Analog rows are the targeted structured phase/subphase members, so the intended field result is 22/22 Analog + 36/36 Digital = 58/58 report-backed rows.

A separate engine candidate is intentionally isolated in ARIEC61850 draft PR #139 for generic status CDC typing from exact stVal TypeSpecification. It adds no reads or wire traffic and is not merged/pinned here until its own CI and physical validation are complete.

No promotion/merge is requested by this comment. Physical field evidence remains the acceptance authority.

Copy link
Copy Markdown
Owner Author

P0.6 checkpoint — 2026-09-20

Current consumer head: 47ee31f7c5e321b71ab289db53698053334e5ef6.

CI evidence:

R7 field-test artifact:

  • Artifact ID: 10598776589
  • Artifact: ARSAS-r7-scl-interoperability-win-x64
  • Artifact ZIP SHA256: F74755CD024E77C2C7C8F08DEA0E7AA860F2ED7A629C0A906B383469333F403D
  • Extracted portable EXE SHA256: 30BE5EAB871FDF797642109A40FBBBEB2CAC3C6F50003483D72A860920AD5E5D

Physical acceptance focus for this candidate:

  1. Discovery -> Static DataSet = exactly 58 selected/live rows.
  2. Analog 22/22; Digital 36/36.
  3. All 58 remain report-backed with actual InformationReport and zero cyclic MMS process polling.
  4. F/I/U/B/E/DP badges come from declared metadata only.
  5. Engineering units match authoritative CF units.SIUnit/multiplier.
  6. Save SCL succeeds while monitoring stays active.
  7. Saved SCL -> Open SCL preserves 58 members/order/RCB behavior.

Important boundary: ARIEC61850 PR #139 is CI-green but is intentionally not integrated into this R7 candidate. Therefore the CBClsCounter SPS->INS semantic correction is not part of this physical binary yet.

Copy link
Copy Markdown
Owner Author

P0.7 finishing semantic/presentation patch — 2026-09-20

Physical P0.6 retest on AA1E1F06R4 established the consumer/runtime baseline before this patch:

  • 58 / 58 Static DataSet live rows;
  • Analog 22 / 22 and Digital 36 / 36;
  • all runtime rows report-backed by configured static RCBs;
  • cyclic MMS process polling = 0;
  • phase/subphase context is visible for the previously missing structured analog rows;
  • Save SCL succeeds while monitoring remains active;
  • saved SCL reopens and reproduces 58 live rows.

Finishing commit: 0d0b9204d6637e3d62e2eee94000386ae43cd0e9.

This patch is deliberately presentation/semantic-only:

  1. Static control-feedback projection now lets declared CDC semantics override generic primitive labels for operator type identity: DPC -> Dbpos/DP and SPC -> Boolean/B. Existing exact feedback rows are upgraded before monitoring; no reference/name/value heuristic is used.
  2. Type badges now also recognize declared SPS/SPC as B and DPS as DP.
  3. FAT Preview/PDF renders stored evidence through the same IEC 61850 operator vocabulary as live/event surfaces (True [1], False [0], Open [01], Close [10], etc.) while the persisted/raw evidence remains byte-for-text unchanged.
  4. Typed Boolean numeric 0/1 is normalized only when declared Boolean/SPS/SPC metadata proves the semantic family; analog/counter 0/1 remains numeric.
  5. Regression coverage explicitly locks raw-evidence immutability versus presentation normalization.

Acquisition boundaries remain untouched: Smart Discovery request shape, RCB planning/activation, report projection, DataSet selection authority, and cyclic-polling prohibition are unchanged.

ARIEC61850 PR #139 remains a separate engine semantic candidate for the CBClsCounter SPS->INS correction; it is not silently mixed into this consumer finishing commit.

masarray commented Sep 20, 2026 •

Copy link
Copy Markdown
Owner Author

P0.7 finishing candidate ready for physical visual/report retest

Head: 0d0b9204d6637e3d62e2eee94000386ae43cd0e9.

Green evidence on this exact head:

Artifacts:

  • R7 candidate artifact ID 10599133416, archive SHA256 a54aaf4e97764f2e796f095dec732428ac9117ab824e905ce5b26b8ec410da7c.
  • Field candidate artifact ID 10599297927, archive SHA256 1fa7886bd442c1da3fe0bb8ff875a0c3b5566ae06f674aa29a3f2e57f093b751.
  • Field test evidence artifact ID 10599263264, archive SHA256 087a6772f0b5443eb035b9b4364918d0aff6547b924ce0851ea73f5da72901c8.

Focused physical retest:

  1. Discovery IED CSWI/XCBR position Event Log and Live Value must show the same DP badge/state presentation as Open SCL.
  2. Boolean statuses must show B plus True [1] / False [0] with the same visual family on Discovery and Open SCL.
  3. FAT Report Preview/PDF must render historical raw true/false as True [1]/False [0] and position evidence with the live operator vocabulary; raw evidence persistence must remain unchanged.
  4. Confirm the already-passed 58/58, 22/22 Analog, 36/36 Digital, report-backed=58, polling=0, Save-SCL-while-live and SCL-reuse behavior remains unchanged.

Engine boundary: this artifact intentionally remains on ARIEC61850 648124097621046f5f127ceb1cf853fea54db730. PR #139 (9123c8aa...) is still the separate CBClsCounter integer semantic candidate and is not mixed into this visual/report finishing binary.

Copy link
Copy Markdown
Owner Author

P0.7 release-candidate baseline is now locked against regression

Lock commit: 3005ad5f12cf626e1a2f375b09d5c1e2feb15f25.

New repository authorities:

  • evidence/p0.7-release-candidate-lock.json — machine-readable accepted physical/runtime/presentation baseline plus remaining SCL semantic gap.
  • docs/P0-7_RELEASE_CANDIDATE_LOCK.md — human-readable non-regression and release rules.
  • tests/ARSAS.Tests/P07ReleaseCandidateLockRegressionTests.cs — executable CI guard.

The lock preserves the physically accepted point reached at consumer head 0d0b9204...:
58/58 live rows, Analog 22/22, Digital 36/36, 58 report-backed, cyclic MMS process polling=0, structured phase context, metadata-only badges, Discovery/Open-SCL state presentation parity, Save-SCL-while-live, reusable 58-row SCL, and FAT Preview semantic display without mutating raw evidence.

The remaining work is explicitly scoped as SCL semantic parity, not acquisition redesign. CBClsCounter is recorded as the known concrete example: current baseline can export/reload it as SPS/Boolean; target is INS/integer from exact live type evidence.

Any future ARIEC61850 semantic integration must preserve every locked P0.7 invariant. A candidate that fixes SCL metadata but regresses reporting, polling, 58-row coverage, Save SCL, reuse, or presentation is rejected by contract.

Copy link
Copy Markdown
Owner Author

Release tracking is now explicit in ARSAS issue #341: SCL semantic parity remains the release blocker, while the P0.7 runtime/presentation baseline is frozen by commit 3005ad5f12cf626e1a2f375b09d5c1e2feb15f25.

Copy link
Copy Markdown
Owner Author

Closing this historical P0/P0.7 candidate rather than leaving it as an apparently mergeable active workstream. Against current main, the remaining delta is only the old P0.7 release-candidate lock/evidence/test, while the accepted v1.6.40 release and post-release SCL issue #341 supersede that candidate authority. This branch is 68 commits behind main. Do not use it as a new base; historical evidence remains available in PR history.

@masarray masarray closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant