diff --git a/.github/ISSUE_TEMPLATE/device-compatibility.yml b/.github/ISSUE_TEMPLATE/device-compatibility.yml index d9587aace..473d475ad 100644 --- a/.github/ISSUE_TEMPLATE/device-compatibility.yml +++ b/.github/ISSUE_TEMPLATE/device-compatibility.yml @@ -6,7 +6,7 @@ body: - type: markdown attributes: value: | - Report service-level evidence, not a universal vendor conclusion. Remove credentials, private endpoints, customer names, confidential SCL, captures and disturbance records. + One issue = one bounded service result. A submission is not verified evidence and does not update the published matrix. Read the [evidence intake and maintainer review policy](https://github.com/masarray/arsas/blob/main/docs/evidence-intake-review.md) before posting. Remove credentials, private endpoints, customer/project names, confidential SCL, raw captures and disturbance records. Never upload a private file expecting maintainers to sanitize it in public. - type: input id: arsas-version attributes: @@ -65,6 +65,22 @@ body: options: [Verified repeatedly, Conditional success, Observed once, Failed with known evidence, Not tested] validations: required: true + - type: textarea + id: expected-observed + attributes: + label: Expected versus observed behavior + description: Give the exact positive or negative result for this service. For a planned test, state "Not tested - no result yet"; do not infer a pass. + placeholder: "Expected: ... / Observed: ... / Failure or negative result: ..." + validations: + required: true + - type: input + id: public-evidence-link + attributes: + label: Public sanitized evidence link (optional) + description: Link only to material you are authorized to publish. An implementation PR alone is not a field-test capture. + placeholder: https://github.com/masarray/arsas/issues/... + validations: + required: false - type: textarea id: conditions attributes: @@ -76,7 +92,7 @@ body: id: diagnostics attributes: label: Sanitized diagnostics - description: Paste exact ARSAS text or attach a sanitized screenshot. Do not include secrets or confidential project data. + description: Paste sanitized ARSAS text or attach a reviewed screenshot. For a planned test, write No capture yet. Never attach raw private captures, confidential SCL, credentials or disturbance records. validations: required: true - type: checkboxes @@ -88,3 +104,5 @@ body: required: true - label: I understand this report is service-specific and is not IEC 61850 conformance certification. required: true + - label: I understand an issue is a submission, not verified evidence; publication requires maintainer review and a separate registry PR. + required: true diff --git a/docs/evidence-intake-review.md b/docs/evidence-intake-review.md new file mode 100644 index 000000000..663b01f4e --- /dev/null +++ b/docs/evidence-intake-review.md @@ -0,0 +1,35 @@ +# ARSAS service-evidence intake and review + +This is the public review procedure for [compatibility evidence](https://masarray.github.io/arsas/compatibility.html) and the [device compatibility issue form](https://github.com/masarray/arsas/issues/new?template=device-compatibility.yml). It is a documentation and maintainer-review workflow, not an automatic test, endorsement, vendor certification, or IEC 61850 conformance process. + +## 1. Submit one bounded service result + +Open one issue per device context and IEC 61850 service. Record the **actual test date**, exact ARSAS version/tag, Windows version, anonymized device/firmware or disclosure boundary, service, observed outcome, acquisition conditions, expected versus observed behavior, and sanitized diagnostics or a public evidence link. Include negative results and reproducible steps. State whether the submission is a field test, sanitized diagnostic, engineering history, or a planned test with no result. If retesting a published profile, supply its ID; do not overwrite historical dates or assume that the latest release was the version used. + +**Privacy first:** never publish credentials, private IPs/endpoints, customer/project names, confidential SCL, raw network captures, or disturbance records. Review screenshots, text and attachments for sensitive identifiers before posting. If safe publication is impossible, report only a bounded sanitized description and mark the evidence unavailable for public verification. Do not upload private files and ask the maintainer to sanitize them publicly. + +A newly opened issue is **submitted, not verified**. An issue link alone does not establish a physical-relay test or a new compatibility status. Planned tests and implementation-only history cannot serve as a current-stable field retest. + +## 2. Maintainer review gate + +Review the report before considering any registry change: + +- **Privacy and authority:** safe-to-publish identifiers; approved network/test context; control tests only in an authorized non-energized environment. +- **Provenance:** actual date, exact tested ARSAS version, evidence type, device disclosure boundary and traceable public record; identify whether raw field capture is publicly linked. +- **Scope and reproducibility:** one service, explicit positive/negative outcome, conditions, permissions, relevant MMS/RCB/GOOSE/control paths, expected versus observed behavior. +- **Claim precision:** separate *observed*, *conditional*, *verified*, *known issue*, *not tested* and *not declared*; a missing service is not a tested failure. Reporting configuration is not proof of live report delivery. +- **Retest:** confirm actual repeat execution on the declared release; engineering PRs or a registry edit are not a retest. Keep earlier evidence dates and unknowns intact. + +If details are missing, request sanitized clarification in the issue and leave the matrix unchanged. If a submission cannot be publicly inspected or responsibly bounded, keep it as a report only, not a published compatibility result. Do not represent maintainer review as independent certification. + +## 3. Promote only with a reviewed PR + +A maintainer may open a separate PR to update `landing/device-evidence.json` and the English/Indonesian compatibility pages. In that PR: + +1. Cite the reviewed public issue or sanitized record *for the exact service*, with test date, version, conditions and record type. A private attachment is not a public evidence link. +2. Set or retain each service status based on its own observation. Do not turn `not-tested` or `not-declared` into a claim by copying another service's result. +3. Change `testedArsasVersion`, `lastRetest`, `rawFieldCaptureLinked` or current-stable language only when the specific supporting public test and date exist. Distinguish engineering history from raw field captures. +4. Reconcile registry, service-to-record links, coverage gaps, English/Indonesian pages and claim boundaries; run source, rendered-site, adoption/field-proof and exact-head PR CI. +5. Merge only after review and required checks pass; verify the final production Pages deployment. The published matrix is authoritative only after this gate. + +An issue may remain open or be closed without a registry change. A green CI validates consistency of the published claims and links, **not the physical truth of a device test**. diff --git a/landing/templates/bukti-kompatibilitas.html b/landing/templates/bukti-kompatibilitas.html index 6bd016425..51fbddcd8 100644 --- a/landing/templates/bukti-kompatibilitas.html +++ b/landing/templates/bukti-kompatibilitas.html @@ -80,7 +80,7 @@

Lihat service IEC 61850 mana yang memiliki evidence publik—dan mana yang b
Evidence field historis · 2026-07

Profile field anonim B — evidence RCB dan export SCL

Vendor, model, instalasi dan firmware tidak dipublikasikan. Evidence tetap read-only sampai action export yang disetujui dilakukan.

Evidence field: 2026-07. Versi ARSAS saat capture: Belum tercatat secara publik. Retest pada v{{STABLE_VERSION}}: Belum terdokumentasi. Update halaman atau registry bukan retest field.

ServiceStatus
MMS association dan live modelObserved
Evidence reportingConditional
Selected-RCB exportVerified
File, GOOSE, controlNot tested

Service → jejak engineering publik: MMS association / live model / opsi RCB read-only · PR #36; export selected-RCB · PR #79. Evidence opsi RCB bukan bukti delivery report live; tautan ini bukan raw field capture ataupun pengujian baru pada stable terbaru.

Kondisi

Evidence option RCBIntegrasi aplikasiRelease exact exportPahami workflow RCB & SCL →
-
Machine-readable source

Profile evidence di-govern melalui JSON.

Registry publik device-evidence.json memuat profile ID, status service, tanggal historis, versi/retest yang belum diketahui, engineering record per service, dan kondisi yang sama dengan halaman ini. CI menolak status tidak dikenal, link hilang atau named-device claim tanpa metadata evidence.

Buka evidence JSON
Kontribusi profile baru

Laporkan behavior service, bukan hanya merek.

Sertakan versi ARSAS, operating system, service yang diuji, diagnostic sanitized, firmware bila boleh dibuka dan hasil yang reproducible. Credential serta identifier rahasia harus dihapus.

Buka form kompatibilitasClaim governance
+
Machine-readable source

Profile evidence di-govern melalui JSON.

Registry publik device-evidence.json memuat profile ID, status service, tanggal historis, versi/retest yang belum diketahui, engineering record per service, dan kondisi yang sama dengan halaman ini. CI menolak status tidak dikenal, link hilang atau named-device claim tanpa metadata evidence.

Buka evidence JSON
Kontribusi profile baru

Laporkan behavior service, bukan hanya merek.

Sertakan versi ARSAS, operating system, service yang diuji, diagnostic sanitized, firmware bila boleh dibuka dan hasil yang reproducible. Credential serta identifier rahasia harus dihapus.

Kirim → review maintainer → PR registry: GitHub Issue adalah laporan, bukan evidence field yang sudah diverifikasi. Reviewer memeriksa privasi, versi/tanggal tepat, hasil expected versus observed dan kondisi service; hanya PR registry terpisah yang direview dapat mengubah matrix ini. Planned test dan riwayat engineering saja bukan bukti retest fisik baru.

Buka form kompatibilitasCara evidence direviewClaim governance
{{> download-cta-id}} diff --git a/landing/templates/compatibility.html b/landing/templates/compatibility.html index 048ad68e1..8bc5e8494 100644 --- a/landing/templates/compatibility.html +++ b/landing/templates/compatibility.html @@ -80,7 +80,7 @@

See exactly which IEC 61850 services have public evidence—and which do not
Historical field evidence · 2026-07

Anonymized field profile B — RCB and SCL export evidence

Vendor, model, installation and firmware are not published. Evidence remains read-only until an approved export action is requested.

Field evidence: 2026-07. ARSAS version at capture: Not publicly recorded. Retest on v{{STABLE_VERSION}}: Not documented. Updating this page or registry is not a field retest.

ServiceStatus
MMS association and live modelObserved
Reporting evidenceConditional
Selected-RCB exportVerified
Files, GOOSE, controlNot tested

Service → public engineering trail: MMS association / live model / read-only RCB options · PR #36; selected-RCB export · PR #79. RCB option evidence does not establish live report delivery; these links are not raw field captures or new stable-release tests.

Conditions

RCB option evidenceApplication integrationExact export releaseUnderstand the RCB & SCL workflow →
-
Machine-readable source

Evidence profiles are governed in JSON.

The public device-evidence.json registry contains the same profile IDs, service statuses, historical dates, explicit version/retest unknowns, service-level engineering records and conditions shown here. CI rejects unsupported status values, missing links or a named-device claim without explicit evidence metadata.

Open evidence JSON
Contribute a new profile

Report the service behavior—not only the brand.

Provide the ARSAS version, operating system, service under test, sanitized diagnostics, firmware when disclosure is allowed and a reproducible result. Confidential identifiers and credentials must be removed.

Open compatibility formClaim governance
+
Machine-readable source

Evidence profiles are governed in JSON.

The public device-evidence.json registry contains the same profile IDs, service statuses, historical dates, explicit version/retest unknowns, service-level engineering records and conditions shown here. CI rejects unsupported status values, missing links or a named-device claim without explicit evidence metadata.

Open evidence JSON
Contribute a new profile

Report the service behavior—not only the brand.

Provide the ARSAS version, operating system, service under test, sanitized diagnostics, firmware when disclosure is allowed and a reproducible result. Confidential identifiers and credentials must be removed.

Submit → maintainer review → registry PR: a GitHub issue is a report, not verified field evidence. A reviewer checks privacy, exact version/date, expected versus observed results and service conditions; only a separate reviewed registry PR can change this matrix. Planned tests and engineering history alone do not establish a new physical retest.

Open compatibility formHow evidence is reviewedClaim governance
{{> download-cta}} diff --git a/scripts/validate-adoption-proof.py b/scripts/validate-adoption-proof.py index e30f1099a..29642cfcb 100644 --- a/scripts/validate-adoption-proof.py +++ b/scripts/validate-adoption-proof.py @@ -291,6 +291,18 @@ def main() -> int: issue_form = read(ROOT / ".github" / "ISSUE_TEMPLATE" / "device-compatibility.yml", errors) require_values(issue_form, "device-compatibility.yml", ("id: evidence-date", "id: evidence-kind", "id: prior-profile", "actual test date", "engineering or implementation history only"), errors, "R6.3 evidence intake") + review_url = "https://github.com/masarray/arsas/blob/main/docs/evidence-intake-review.md" + require_values(issue_form, "device-compatibility.yml", ( + review_url, "one issue = one bounded service result", "id: expected-observed", + "id: public-evidence-link", "a submission, not verified evidence", + "separate registry PR", "raw private captures", + ), errors, "R6.4 evidence intake/review gate") + review_policy = read(ROOT / "docs" / "evidence-intake-review.md", errors) + require_values(review_policy, "docs/evidence-intake-review.md", ( + "submitted, not verified", "Privacy first", "Maintainer review gate", + "Promote only with a reviewed PR", "actual test date", "exact tested ARSAS version", + "not a retest", "registry", "not the physical truth", + ), errors, "R6.4 review policy") for name in ("compatibility.html", "bukti-kompatibilitas.html"): text = read(TEMPLATES / name, errors) @@ -302,6 +314,18 @@ def main() -> int: ("Review matrix service", "Review batas klaim", "Pahami workflow file service →", "Pahami workflow RCB & SCL →") ) require_values(text, name, proof_route, errors, "R6 evaluator proof route") + if 'data-evidence-intake="submitted-not-verified"' not in text or review_url not in text: + errors.append(f"{name}: missing R6.4 submission-to-review policy route") + if name == "compatibility.html": + require_values(text, name, ( + "Submit → maintainer review → registry PR", "a GitHub issue is a report", + "Planned tests and engineering history alone", + ), errors, "R6.4 EN publication gate") + else: + require_values(text, name, ( + "Kirim → review maintainer → PR registry", "GitHub Issue adalah laporan", + "Planned test dan riwayat engineering saja", + ), errors, "R6.4 ID publication gate") for profile in profiles: if not isinstance(profile, dict): continue diff --git a/scripts/validate-product-build.py b/scripts/validate-product-build.py index 26162681d..0de15a3e0 100644 --- a/scripts/validate-product-build.py +++ b/scripts/validate-product-build.py @@ -321,6 +321,8 @@ def main() -> int: ) for value in contract: if value not in matrix_text: errors.append(f"{page}: missing rendered interoperability proof value {value}") + for value in ('data-evidence-intake="submitted-not-verified"', 'docs/evidence-intake-review.md'): + if value not in matrix_text: errors.append(f"{page}: missing rendered R6.4 review gate {value}") for value in ('data-evidence-freshness="true"', f"v{latest.get('version')}", 'data-tested-version="not-recorded"', 'data-current-stable-retest="not-documented"'): if value not in matrix_text: errors.append(f"{page}: missing rendered historical/current stable distinction {value}") if "{{STABLE_VERSION}}" in matrix_text: diff --git a/scripts/validate-product-source.py b/scripts/validate-product-source.py index ff1594200..20f4f91f3 100644 --- a/scripts/validate-product-source.py +++ b/scripts/validate-product-source.py @@ -305,6 +305,9 @@ def main() -> int: for value in proof_route: if value not in rendered: errors.append(f"{label}: missing interoperability proof route value {value}") + for value in ('data-evidence-intake="submitted-not-verified"', 'docs/evidence-intake-review.md'): + if value not in raw: + errors.append(f"{label}: missing R6.4 intake/review route {value}") for value in ('data-evidence-freshness="true"', '{{STABLE_VERSION}}', 'data-tested-version="not-recorded"', 'data-current-stable-retest="not-documented"'): if value not in raw: errors.append(f"{label}: missing historical field/retest disclosure {value}")