Add unified FAT/SAT test and evidence workspace by masarray · Pull Request #110 · masarray/arsas

masarray · 2026-07-28T06:22:27Z

Purpose

Implement ARSAS P2 as a unified, schema-versioned FAT/SAT Test & Evidence Workspace rather than another disconnected protocol viewer.

Workspace

opens from the ARSAS main header without coupling to monitoring runtime state
captures project, site, bay/system, IED, operator, witness, scope, app provenance, and engine provenance
supports editable test sequence, area, title, procedure, expected result, actual result, operator note, deviation, execution identity/time, and outcome
outcomes: NotRun, Pass, Fail, Review, Blocked, NotApplicable
summary always exposes PASS/FAIL/REVIEW/BLOCKED/NOT RUN and evidence counts

Default IEC 61850 plan

The initial plan covers:

IED and project identity
MMS discovery
report acquisition and reconnect recovery
GOOSE state/timing
bounded Sampled Values evidence
authorized guarded control
IEC 61850 file transfer
configured/generated/live SCL comparison
deviation and closeout review

Evidence integrity

files are hashed with SHA-256 at attachment time
source size and SHA-256 are re-verified before package export
changed or missing evidence stops export
duplicate evidence content is not attached twice to the same test case
limits: 256 MB per evidence file, 100 evidence files per test, 1,000 test cases per workspace

Persistence and audit package

schema v1 *.arsas-fat.json
stable workspace/test/evidence GUIDs
unsupported schema versions are rejected instead of guessed
save uses .partial plus atomic move
audit ZIP contains:
- workspace.json with package-relative evidence paths
- report.md with scope, disposition, outcomes, deviations, and evidence references
- evidence/... immutable source files
- SHA256SUMS.txt
final ZIP receives a package-level SHA-256
local absolute source paths are not leaked into the portable package

Acceptance boundary

A package is COMPLETE only when no test remains NotRun and no test is Fail, Review, or Blocked. Operator-entered outcomes remain operator-owned. The workspace does not claim formal conformance, universal interoperability, calibrated measurement, or authorization for live control.

Final automated validation at head `47b4005c8484c594e03ba4e49712d15c32bf94e4`

Build ARSAS run 707 — success
- source, license, premium UX, GOOSE, SMV, SAS, and engine-lock invariants
- immutable ARIEC61850 checkout and API checks
- complete solution restore and Release build
- 17/17 application regression tests passed
- TRX evidence artifact uploaded
- portable Windows package published
generated artifacts:
- ARSAS-win-x64
- ARSAS-test-evidence
- ARSAS-source-snapshot

The installer workflow did not run for this stacked delta because no installer, release, version, project, or engine-lock path changed. The full WPF application and portable package were compiled from the P2 head.

Regression coverage

bounded default IEC 61850 plan
JSON save/open round-trip
unsupported-schema rejection
real ZIP audit-package contents
source-path redaction
report disposition and engine provenance
evidence checksum listing
export rejection when evidence changes after attachment

Stack boundary

This PR is intentionally based on P1 branch agent/p1-sv-evidence-bundle / PR #109 and contains only the P2 delta. Public stable release remains 1.6.18.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47b4005c84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-28T06:30:53Z

+        var duplicate = document.TestCases.GroupBy(item => item.TestCaseId).FirstOrDefault(group => group.Key == Guid.Empty || group.Count() > 1);
+        if (duplicate is not null)
+            throw new InvalidDataException("Test-case identities must be non-empty and unique.");
+        foreach (var testCase in document.TestCases)


Reject undefined result values before computing disposition

Because JsonStringEnumConverter accepts integer enum tokens by default, a workspace containing "result": 99 passes this validation. If every other test passes, the undefined result is counted as neither NotRun nor blocking, so IsComplete becomes true and the audit report falsely declares the package complete; validate every result with Enum.IsDefined before accepting the document.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-28T06:30:53Z

+                var source = Path.GetFullPath(evidence.SourcePath);
+                if (!File.Exists(source))
+                    throw new FileNotFoundException($"Evidence file '{evidence.DisplayName}' is missing.", source);
+                var bytes = await File.ReadAllBytesAsync(source, cancellationToken);


Stream evidence files while building the archive

When a workspace contains several large evidence files, this reads each entire file into a new byte array and retains every array in entries until ZIP creation begins. The documented limits permit 256 MB per file and 100 files per test, so even a handful of valid captures can exhaust process memory and make export fail; hash and copy each file directly into its archive entry instead of buffering the complete evidence set.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-28T06:30:53Z

+    private void New_Click(object sender, RoutedEventArgs e)
+        => ApplyDocument(_workspaceService.CreateDefault(), null);


Confirm before replacing an unsaved workspace

After an operator edits outcomes, notes, or evidence references without saving, clicking New immediately replaces the current document and rows with a default workspace. There is no dirty-state check or confirmation, so potentially lengthy FAT/SAT execution records are irrecoverably discarded; prompt to save or cancel before applying the replacement document.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-28T06:30:53Z

+            throw new InvalidDataException($"Unsupported FAT/SAT schema version {document.SchemaVersion}. Expected {FatSatWorkspaceDocument.CurrentSchemaVersion}.");
+        if (document.WorkspaceId == Guid.Empty)
+            throw new InvalidDataException("Workspace identity is missing.");
+        if (document.TestCases.Count > 1000)


Reject null collections as malformed workspace data

Opening otherwise valid JSON containing "testCases": null makes deserialization assign null despite the property initializer, and this dereference throws NullReferenceException rather than the intended InvalidDataException; similarly, null test-case elements or evidence collections fail later. These exceptions bypass Open_Click's normal malformed-file error handling, so validate collection and element nullability before accessing them.

Useful? React with 👍 / 👎.

masarray · 2026-07-28T06:54:54Z

Closing as superseded by the dedicated IO List Testing direction agreed for ARSAS. This PR is also stacked on PR #109, which was closed unmerged after evidence-correctness findings, so retargeting it to main would reintroduce that rejected ancestry. The reusable ideas—schema-versioned persistence, immutable evidence hashes, atomic save/export, and portable audit packages—should be reintroduced later on top of PR #111's IED-scoped IO test domain rather than as a second generic FAT/SAT workspace.

masarray added 8 commits July 28, 2026 13:14

feat: add versioned FAT SAT workspace model

c8deb6d

feat: add FAT SAT persistence and audit package service

435cde3

feat: add FAT SAT workspace window

d3ef263

feat: implement FAT SAT workspace execution UI

811e2c4

feat: expose FAT SAT workspace from ARSAS header

0a8405e

test: validate FAT SAT persistence and audit evidence integrity

a984332

docs: define FAT SAT workspace evidence contract

6ae511b

docs: record P2 FAT SAT workspace

47b4005

masarray marked this pull request as ready for review July 28, 2026 06:26

chatgpt-codex-connector Bot reviewed Jul 28, 2026

View reviewed changes

masarray closed this Jul 28, 2026

masarray mentioned this pull request Sep 3, 2026

G2.6/G2.7 Smart Dynamic RCB: field-stable per-IED bootstrap + full eligible residual coverage #230

Closed

Navigation Menu

Add unified FAT/SAT test and evidence workspace - #110

Add unified FAT/SAT test and evidence workspace#110
masarray wants to merge 8 commits into
agent/p1-sv-evidence-bundlefrom
agent/p2-fat-sat-workspace

masarray commented Jul 28, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Uh oh!

masarray commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		private void New_Click(object sender, RoutedEventArgs e)
		=> ApplyDocument(_workspaceService.CreateDefault(), null);

Conversation

masarray commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Workspace

Default IEC 61850 plan

Evidence integrity

Persistence and audit package

Acceptance boundary

Final automated validation at head 47b4005c8484c594e03ba4e49712d15c32bf94e4

Regression coverage

Stack boundary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

masarray commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

masarray commented Jul 28, 2026 •

edited

Loading

Final automated validation at head `47b4005c8484c594e03ba4e49712d15c32bf94e4`