Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
fetch-depth: 0
- uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6
with:
global-json-file: global.json
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,7 @@ The repository check enforces the [project licence boundary](docs/licence-bounda
for the application, core library, tests and C# tooling. Every build also checks
the effective MSBuild declarations and local project references.

The same tool enforces [source provenance](docs/provenance.md): complete file
classification, immutable reuse records, full notices and portable-archive checks.

Read [CONTRIBUTING](CONTRIBUTING.md), [security reporting](SECURITY.md) and the [code of conduct](CODE_OF_CONDUCT.md). ArcNotes remains **AGPL-3.0-only**; see [LICENSE](LICENSE) and [third-party notices](THIRD_PARTY_NOTICES.md).
58 changes: 58 additions & 0 deletions docs/provenance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Source provenance (WP00.03)

The [accepted Design profile](https://github.com/ArcForges/ArcForges-Design/blob/5322d698a1b650a52a5a139d986dd85b00b48581/docs/assurance/reference-coverage-and-provenance.md)
governs the current ArcNotes repository. The retired initialization repository is
historical lineage, not a source/build prerequisite. Desktop UI remains native
Avalonia/Skia. This change adds no browser UI or tooling runtime dependency.

Before reusing material, complete `eng/provenance/template.json` as a record in
`eng/provenance/records/<id>.json`. The ten fields identify the exact source,
commit and paths, file-level licence, attribution, target, disposition, oracle,
NOTICE and lifetime. Generated material also identifies each generator and input;
temporary material names its removal trigger and owner. The Licensing and
Provenance Owner reviews actual evidence within the closed decision table in
`eng/policy/reuse-policy.json`. A maintainer-authorized review may exercise that
role; an `approved` string alone cannot replace review. Record unresolved conflicts
under `eng/provenance/conflicts` and resolve them before accepting the material.

`eng/provenance/files.json` accounts for every tracked file, including tracked
ignored files, and every non-ignored new file during local checks. Reused targets
have exact record bindings and SHA-256 values; all other files are explicitly
classified as current first-party material. Review must also detect copied material
inserted into an existing file: an inventory check cannot establish authorship.
The original records reconcile existing legal documents honestly and do not claim
that approval preceded their historical introduction.

Used records remain immutable, even after retirement. A changed source, target,
intent or obligation requires a new revision with `supersedes`; retain its earlier
records and update only the active binding. CI reads its trusted event base and
requires the complete comparison history. A retained external file cannot silently
become first-party material. The explanatory template is never an approved record.

Run the existing C# checks and tests from `CONTRIBUTING.md`. After completing and
reviewing new records/inventory, regenerate the deterministic summary with:

```sh
dotnet run --project eng/ArcForges.Repository -- provenance-notice
dotnet run --project eng/ArcForges.Repository -- check
```

The checker rejects missing/blank fields, unclassified files, unknown/prohibited
licences, missing or changed targets, escaping/linked paths, altered history,
broken supersession and missing notices. Its C# adaptation records the Apache
Contracts source at `18a970c67c463f1971ca05773b80f31a1b1ba1a7`; full original terms
and changed-file attribution remain in the repository. This source/build tool is
absent from the Native AOT application. No sibling source is consumed during builds.

Portable staging requires committed, audited source. It retains the root licence,
six exact upstream legal documents, the generated package source summary and the
actual source-audit receipt. Packing and independent release verification read the
real ZIP/tar entries, compare full legal bytes and require a passing receipt for
the candidate's clean source commit. Existing dependency notices remain intact;
the summary does not replace them. The same gates run on all five native hosts.

Locked dependencies, Native AOT, native UI actions, real Cloud requests, failure
cases and exact public release assets retain their existing gates. Source or
licence checks do not establish product completeness, OS trust signing or later
commercial readiness. Each contribution records its own observed runtime and
publication evidence after full review and successful CI.
78 changes: 74 additions & 4 deletions eng/ArcForges.Repository/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ public static async Task<int> Main(string[] args)
await Run("git", ["config", "--worktree", "core.hooksPath", ".githooks"]);
break;
case ["check"]: await Check(); break;
case ["provenance-notice"]: await ProvenancePolicy.Check(Directory.GetCurrentDirectory(), writeNotice: true); break;
case ["version"]:
var generatedVersion = Version(Environment.GetEnvironmentVariable("GITHUB_RUN_NUMBER") ?? "0", Environment.GetEnvironmentVariable("GITHUB_RUN_ATTEMPT") ?? "1");
Console.WriteLine(generatedVersion);
Expand All @@ -38,7 +39,7 @@ public static async Task<int> Main(string[] args)
Directory.CreateDirectory("artifacts/evidence");
await Run(Executable(rid), ["--smoke-live", "--evidence", Path.GetFullPath($"artifacts/evidence/{rid}.json")]);
break;
case ["pack", var rid, var version, var commit]: Pack(rid, version, commit); break;
case ["pack", var rid, var version, var commit]: await Pack(rid, version, commit); break;
case ["verify", var path, var version, var commit]:
var manifests = Directory.GetFiles(path, "manifest.json", SearchOption.AllDirectories);
if (manifests.Length != Rids.Length) throw new InvalidOperationException("Expected all five native candidates.");
Expand All @@ -50,7 +51,7 @@ public static async Task<int> Main(string[] args)
}
Console.WriteLine("Verified all five immutable native candidates.");
break;
default: throw new ArgumentException("Use hooks, check, version, prepare RID VERSION, smoke RID, pack RID VERSION COMMIT, or verify DIRECTORY VERSION COMMIT.");
default: throw new ArgumentException("Use hooks, check, provenance-notice, version, prepare RID VERSION, smoke RID, pack RID VERSION COMMIT, or verify DIRECTORY VERSION COMMIT.");
}
return 0;
}
Expand Down Expand Up @@ -86,6 +87,7 @@ private static async Task Prepare(string rid, string version)
{
ValidateRid(rid);
ValidateVersion(version);
await CheckSourceForDistribution();
var source = Path.GetFullPath($"artifacts/publish/{rid}");
var stage = Stage(rid);
if (Directory.Exists(stage)) throw new InvalidOperationException("Staging already exists; use a fresh worktree/output directory.");
Expand All @@ -100,6 +102,10 @@ private static async Task Prepare(string rid, string version)
}
foreach (var name in new[] { "LICENSE", "THIRD_PARTY_NOTICES.md", "README.md" }) File.Copy(name, Path.Combine(stage, name));
WriteDependencyNotices(stage);
var requiredNotices = ProvenancePolicy.PackageNotices(Directory.GetCurrentDirectory());
File.WriteAllBytes(Path.Combine(stage, "notices/source-provenance.txt"), requiredNotices["notices/source-provenance.txt"]);
ProvenancePolicy.VerifyPackageNotices(requiredNotices, path => File.Exists(Path.Combine(stage, path)) ? File.ReadAllBytes(Path.Combine(stage, path)) : null);
File.Copy("artifacts/evidence/provenance.json", Path.Combine(stage, "notices/provenance-source.json"));
if (!File.Exists(Executable(rid))) throw new InvalidOperationException("Published executable is missing.");
if (rid.StartsWith("osx-", StringComparison.Ordinal))
{
Expand Down Expand Up @@ -154,11 +160,21 @@ private static void WriteDependencyNotices(string stage)
File.Copy(file, Path.Combine(stage, "notices/upstream", Path.GetFileName(file)));
}

private static void Pack(string rid, string version, string commit)
private static async Task CheckSourceForDistribution()
{
await ProvenancePolicy.Check(Directory.GetCurrentDirectory());
if ((await Capture("git", ["status", "--porcelain"])).Length != 0)
throw new InvalidOperationException("Commit reviewed source before producing a source-bound portable candidate.");
}

private static async Task Pack(string rid, string version, string commit)
{
ValidateRid(rid);
ValidateVersion(version);
if (commit.Length != 40 || !commit.All(char.IsAsciiHexDigit)) throw new ArgumentException("Expected a full Git commit.");
await CheckSourceForDistribution();
if ((await Capture("git", ["rev-parse", "HEAD"])).Trim() != commit)
throw new InvalidOperationException("Pack source differs from the candidate revision.");
var evidence = $"artifacts/evidence/{rid}.json";
using var smoke = JsonDocument.Parse(File.ReadAllText(evidence));
ValidateSmoke(smoke.RootElement, rid, version, commit);
Expand All @@ -175,6 +191,7 @@ private static void Pack(string rid, string version, string commit)
using var gzip = new GZipStream(file, CompressionLevel.Optimal);
TarFile.CreateFromDirectory(Stage(rid), gzip, false);
}
VerifyArchiveNotices(archive, commit, Directory.GetCurrentDirectory());
File.Copy(evidence, Path.Combine(folder, "smoke.json"));
File.Copy(Path.ChangeExtension(evidence, ".png"), Path.Combine(folder, "screen.png"));
var manifest = new Candidate(rid, version, commit, name, Hash(archive), Hash(evidence));
Expand All @@ -183,7 +200,7 @@ private static void Pack(string rid, string version, string commit)
Console.WriteLine($"Packed verified {rid}: {name}");
}

public static string VerifyCandidate(string manifestPath, string version, string commit)
public static string VerifyCandidate(string manifestPath, string version, string commit, string? sourceRoot = null)
{
var manifest = JsonSerializer.Deserialize<Candidate>(File.ReadAllText(manifestPath), Json) ?? throw new InvalidOperationException("Missing manifest.");
ValidateRid(manifest.Rid);
Expand All @@ -200,9 +217,61 @@ public static string VerifyCandidate(string manifestPath, string version, string
throw new InvalidOperationException("Download checksum mismatch.");
using var evidence = JsonDocument.Parse(File.ReadAllText(smoke));
ValidateSmoke(evidence.RootElement, manifest.Rid, version, commit);
VerifyArchiveNotices(archive, commit, sourceRoot ?? Directory.GetCurrentDirectory());
return manifest.Rid;
}

private static void VerifyArchiveNotices(string archive, string commit, string sourceRoot)
{
var expected = ProvenancePolicy.PackageNotices(sourceRoot);
const string receiptName = "notices/provenance-source.json";
var selected = new Dictionary<string, byte[]>(StringComparer.Ordinal);
var members = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
void Retain(string path, Stream? stream)
{
while (path.StartsWith("./", StringComparison.Ordinal)) path = path[2..];
path = path.TrimEnd('/');
if (path.Length == 0 || path == ".") return;
if (path.StartsWith('/') || path.IndexOfAny(['\\', ':', '\0']) >= 0 || path.Split('/').Any(p => p is "" or "." or ".."))
throw new InvalidOperationException("Escaping archive member: " + path);
if (!members.Add(path)) throw new InvalidOperationException("Duplicate or case-colliding archive member: " + path);
if (!expected.ContainsKey(path) && path != receiptName) return;
if (stream is null) throw new InvalidOperationException("Required notice is not a regular archive file: " + path);
using var memory = new MemoryStream();
stream.CopyTo(memory);
if (!selected.TryAdd(path, memory.ToArray())) throw new InvalidOperationException("Duplicate archive notice: " + path);
}
if (archive.EndsWith(".zip", StringComparison.Ordinal))
{
using var zip = ZipFile.OpenRead(archive);
foreach (var entry in zip.Entries)
{
if (((entry.ExternalAttributes >> 16) & 0xF000) == 0xA000)
throw new InvalidOperationException("Linked ZIP member: " + entry.FullName);
using var stream = entry.Open();
Retain(entry.FullName, stream);
}
}
else
{
using var file = File.OpenRead(archive);
using var gzip = new GZipStream(file, CompressionMode.Decompress);
using var tar = new TarReader(gzip);
while (tar.GetNextEntry() is { } entry)
{
if (entry.EntryType is TarEntryType.SymbolicLink or TarEntryType.HardLink)
throw new InvalidOperationException("Linked tar member: " + entry.Name);
Retain(entry.Name, entry.DataStream);
}
}
ProvenancePolicy.VerifyPackageNotices(expected, path => selected.GetValueOrDefault(path));
if (!selected.TryGetValue(receiptName, out var receipt)) throw new InvalidOperationException("Missing candidate source provenance.");
using var report = JsonDocument.Parse(receipt);
if (report.RootElement.GetProperty("sourceCommit").GetString() != commit ||
report.RootElement.GetProperty("dirty").GetBoolean() || report.RootElement.GetProperty("result").GetString() != "passed")
throw new InvalidOperationException("Candidate provenance does not match clean reviewed source.");
}

private static void ValidateSmoke(JsonElement smoke, string rid, string version, string commit)
{
if (!smoke.GetProperty("success").GetBoolean() || !smoke.GetProperty("nativeAot").GetBoolean() ||
Expand All @@ -223,6 +292,7 @@ private static string Hash(string path)
private static async Task Check()
{
await Run("git", ["diff", "--check"]);
await ProvenancePolicy.Check(Directory.GetCurrentDirectory());
using var upstream = JsonDocument.Parse(File.ReadAllText("third-party/sources.json"));
foreach (var entry in upstream.RootElement.EnumerateArray())
{
Expand Down
Loading
Loading