Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 66 additions & 2 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ on:
- "scripts/inject-site-measurement.py"
- "scripts/validate-site-measurement.py"
- "scripts/check-site-health.py"
- "scripts/generate-privacy-pages.py"
- "scripts/stamp-site-build.py"
- "scripts/verify-pages-deployment.py"
- ".github/workflows/pages.yml"
pull_request:
branches: [ main ]
Expand All @@ -24,6 +27,9 @@ on:
- "scripts/inject-site-measurement.py"
- "scripts/validate-site-measurement.py"
- "scripts/check-site-health.py"
- "scripts/generate-privacy-pages.py"
- "scripts/stamp-site-build.py"
- "scripts/verify-pages-deployment.py"
- ".github/workflows/pages.yml"
workflow_dispatch:

Expand All @@ -34,6 +40,9 @@ concurrency:
group: pages
cancel-in-progress: true

env:
CANONICAL_ROOT: https://masarray.github.io/arsas/

jobs:
validate:
name: Validate ARSAS product website
Expand Down Expand Up @@ -109,12 +118,27 @@ jobs:
- name: Build deterministic product website
run: python scripts/build-product-site.py --output _site --release-evidence /tmp/arsas-published.json

- name: Generate bilingual privacy pages
run: python scripts/generate-privacy-pages.py --output _site --release-evidence /tmp/arsas-published.json

- name: Configure optional client measurement
env:
GA4_MEASUREMENT_ID: ${{ vars.GA4_MEASUREMENT_ID }}
run: python scripts/inject-site-measurement.py _site --measurement-id "$GA4_MEASUREMENT_ID"

- name: Validate measurement contract
- name: Stamp source and workflow attestation
shell: bash
run: |
set -euo pipefail
commit_timestamp="$(git show -s --format=%cI "$GITHUB_SHA")"
python scripts/stamp-site-build.py _site \
--source-commit "$GITHUB_SHA" \
--source-ref "$GITHUB_REF" \
--commit-timestamp "$commit_timestamp" \
--workflow-run-id "$GITHUB_RUN_ID" \
--workflow-run-attempt "$GITHUB_RUN_ATTEMPT"

- name: Validate consent, privacy and measurement contract
env:
GA4_MEASUREMENT_ID: ${{ vars.GA4_MEASUREMENT_ID }}
run: python scripts/validate-site-measurement.py _site --measurement-id "$GA4_MEASUREMENT_ID"
Expand Down Expand Up @@ -180,10 +204,50 @@ jobs:
id: deployment
uses: actions/deploy-pages@v4

verify-production:
name: Verify public Pages attestation
if: github.event_name != 'pull_request'
needs: deploy
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
show-progress: false

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Verify deployed commit, privacy and measurement state
env:
GA4_MEASUREMENT_ID: ${{ vars.GA4_MEASUREMENT_ID }}
shell: bash
run: |
set -euo pipefail
stable_version="$(python -c 'import json; print(json.load(open("landing/latest.json"))["version"])')"
if [ -n "${GA4_MEASUREMENT_ID:-}" ]; then measurement_enabled=true; else measurement_enabled=false; fi
python scripts/verify-pages-deployment.py \
--base-url "$CANONICAL_ROOT" \
--source-commit "$GITHUB_SHA" \
--stable-version "$stable_version" \
--measurement-enabled "$measurement_enabled" \
--output _validation/production-attestation.md

- name: Upload production attestation
if: always()
uses: actions/upload-artifact@v4
with:
name: production-pages-attestation
path: _validation/production-attestation.*
if-no-files-found: error
retention-days: 90

notify-indexing:
name: Notify search engines
if: github.event_name != 'pull_request'
needs: deploy
needs: verify-production
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down
42 changes: 39 additions & 3 deletions .github/workflows/site-measurement.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ on:
paths:
- "landing/**"
- "scripts/build-product-site.py"
- "scripts/generate-privacy-pages.py"
- "scripts/stamp-site-build.py"
- "scripts/verify-pages-deployment.py"
- "scripts/inject-site-measurement.py"
- "scripts/validate-site-measurement.py"
- "scripts/check-site-health.py"
Expand All @@ -27,6 +30,9 @@ on:
paths:
- "landing/**"
- "scripts/build-product-site.py"
- "scripts/generate-privacy-pages.py"
- "scripts/stamp-site-build.py"
- "scripts/verify-pages-deployment.py"
- "scripts/inject-site-measurement.py"
- "scripts/validate-site-measurement.py"
- "scripts/check-site-health.py"
Expand All @@ -46,7 +52,7 @@ env:

jobs:
quality:
name: Validate measurement and internal links
name: Validate measurement, privacy and internal links
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down Expand Up @@ -77,12 +83,27 @@ jobs:
- name: Build deterministic website
run: python scripts/build-product-site.py --output _site --release-evidence /tmp/arsas-published.json

- name: Generate bilingual privacy pages
run: python scripts/generate-privacy-pages.py --output _site --release-evidence /tmp/arsas-published.json

- name: Configure optional client measurement
env:
GA4_MEASUREMENT_ID: ${{ vars.GA4_MEASUREMENT_ID }}
run: python scripts/inject-site-measurement.py _site --measurement-id "$GA4_MEASUREMENT_ID"

- name: Validate page, download, language, 404 and Web Vitals measurement
- name: Stamp source and workflow attestation
shell: bash
run: |
set -euo pipefail
commit_timestamp="$(git show -s --format=%cI "$GITHUB_SHA")"
python scripts/stamp-site-build.py _site \
--source-commit "$GITHUB_SHA" \
--source-ref "$GITHUB_REF" \
--commit-timestamp "$commit_timestamp" \
--workflow-run-id "$GITHUB_RUN_ID" \
--workflow-run-attempt "$GITHUB_RUN_ATTEMPT"

- name: Validate page, consent, privacy, download, language, 404 and Web Vitals measurement
env:
GA4_MEASUREMENT_ID: ${{ vars.GA4_MEASUREMENT_ID }}
run: python scripts/validate-site-measurement.py _site --measurement-id "$GA4_MEASUREMENT_ID"
Expand Down Expand Up @@ -150,12 +171,27 @@ jobs:
- name: Build current website
run: python scripts/build-product-site.py --output _site --release-evidence /tmp/arsas-published.json

- name: Generate bilingual privacy pages
run: python scripts/generate-privacy-pages.py --output _site --release-evidence /tmp/arsas-published.json

- name: Configure current client measurement
env:
GA4_MEASUREMENT_ID: ${{ vars.GA4_MEASUREMENT_ID }}
run: python scripts/inject-site-measurement.py _site --measurement-id "$GA4_MEASUREMENT_ID"

- name: Check deployed pages, release assets and 404 response
- name: Stamp reporting build attestation
shell: bash
run: |
set -euo pipefail
commit_timestamp="$(git show -s --format=%cI "$GITHUB_SHA")"
python scripts/stamp-site-build.py _site \
--source-commit "$GITHUB_SHA" \
--source-ref "$GITHUB_REF" \
--commit-timestamp "$commit_timestamp" \
--workflow-run-id "$GITHUB_RUN_ID" \
--workflow-run-attempt "$GITHUB_RUN_ATTEMPT"

- name: Check deployed pages, privacy routes, release assets and 404 response
id: deployed_health
continue-on-error: true
run: python scripts/check-site-health.py --site _site --output _measurement --remote --base-url "$CANONICAL_ROOT"
Expand Down
63 changes: 48 additions & 15 deletions docs/website-measurement.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ARSAS website measurement

ARSAS uses a lightweight, evidence-oriented measurement pipeline. Runtime analytics are optional and remain disabled when no valid measurement ID is configured. Search and performance reports are private GitHub Actions artifacts; they are not deployed to the public website.
ARSAS uses a lightweight, evidence-oriented measurement pipeline. Runtime analytics are optional and remain disabled when no valid measurement ID is configured. Even when configured, Google Analytics is denied by default and its client is not loaded until the visitor explicitly allows optional analytics. Search and performance reports are private GitHub Actions artifacts; they are not deployed to the public website.

## What is measured

Expand All @@ -14,13 +14,25 @@ ARSAS uses a lightweight, evidence-oriented measurement pipeline. Runtime analyt
| Are links broken or 404s occurring? | Build-time crawler + deployed probe + GA4 | Missing files/fragments, HTTP failures, 404 paths and referrers |
| Are Core Web Vitals healthy? | Browser PerformanceObserver + PageSpeed/CrUX | LCP, CLS and INP field/lab evidence |

The browser client disables Google advertising signals, does not request ad-personalization signals, respects `Do Not Track`, loads asynchronously and performs no network request when the measurement ID is absent.
The browser client disables Google advertising signals, does not request ad-personalization signals, respects `Do Not Track`, loads asynchronously and performs no network request when the measurement ID is absent or consent has not been granted.

## Consent and privacy contract

- Default Google consent state: `analytics_storage=denied`.
- Advertising storage, ad-user-data and ad-personalization remain denied.
- The public GA4 client is dynamically loaded only after **Allow analytics**.
- Declining analytics does not change downloads, content access or application behavior.
- The local key `arsas_analytics_consent_v1` stores only `granted` or `denied`.
- Do Not Track overrides a stored grant and keeps analytics disabled.
- Revoking consent reloads the current page to stop the already-loaded client before further interaction.
- `privacy.html` and `privasi.html` are bilingual `noindex,follow` policy pages and never load the analytics client.
- Project policy requires a two-month GA4 event-data retention window before analytics is enabled.

## Repository configuration

Configure these **Actions variables**:

- `GA4_MEASUREMENT_ID`: public web stream ID such as `G-XXXXXXXXXX`. Leaving it empty keeps client measurement disabled.
- `GA4_MEASUREMENT_ID`: public web stream ID such as `G-XXXXXXXXXX`. Leaving it empty keeps client measurement disabled and suppresses the first-visit consent prompt.
- `GA4_PROPERTY_ID`: numeric GA4 property ID used by the private reporting workflow.
- `GSC_SITE_URL`: the exact verified Search Console property, normally `https://masarray.github.io/arsas/` for a URL-prefix property.
- `PAGESPEED_URLS`: optional comma-separated URLs. When omitted, the workflow checks the English and Indonesian home/download pages plus Smart Reporting and Guides.
Expand All @@ -32,17 +44,37 @@ Configure these **Actions secrets**:

Grant the service account Viewer/read access only. It does not need permission to modify analytics, Search Console, releases or the website.

Before setting `GA4_MEASUREMENT_ID`, verify in GA4 that event-level retention is two months, Google Signals is disabled, Google Ads is not linked for this project, and no User-ID collection is configured.

## Production deployment attestation

Every Pages artifact is stamped after build with:

- full source commit SHA;
- source ref;
- source commit timestamp;
- GitHub Actions workflow run ID and attempt;
- stable release version;
- privacy and measurement activation state.

After `actions/deploy-pages`, `scripts/verify-pages-deployment.py` fetches the public `build-info.json` with cache-busting parameters and retries until the public source commit matches the just-deployed commit. It also verifies both privacy routes, denied-by-default consent metadata, the configured measurement state and the shared consent controls on the homepage.

A stale public build, missing privacy route or measurement-state mismatch fails the workflow. IndexNow notification depends on this attestation and is skipped when production is stale.

The `production-pages-attestation` artifact retains Markdown and JSON evidence for 90 days.

## Workflow behavior

`.github/workflows/site-measurement.yml` runs:

- on relevant pull requests and pushes: deterministic build, measurement contract validation, internal links and fragment validation;
- on relevant pull requests and pushes: deterministic build, privacy generation, deployment stamping, consent/measurement validation, internal links and fragment validation;
- every Monday at 03:17 UTC, or manually: deployed page checks, official release-asset checks, an intentional 404 probe, GA4 aggregate reports, Search Console reports and PageSpeed/CrUX collection.

Artifacts:

- `site-measurement-quality`: local link and instrumentation evidence, retained for 30 days;
- `site-measurement-<run>`: private Markdown/JSON traffic, search, 404 and Core Web Vitals evidence, retained for 90 days.
- `site-measurement-quality`: local link, privacy and instrumentation evidence, retained for 30 days;
- `site-measurement-<run>`: private Markdown/JSON traffic, search, 404 and Core Web Vitals evidence, retained for 90 days;
- `production-pages-attestation`: deployed-commit and privacy evidence, retained for 90 days.

The same Markdown report is written to the GitHub Actions job summary.

Expand All @@ -57,7 +89,7 @@ These thresholds are implemented in `scripts/build-site-measurement-report.py` a

## Event contract

The local `landing/analytics.js` client emits:
After consent, the local `landing/analytics.js` client emits:

- `page_view`;
- `page_not_found`;
Expand All @@ -74,14 +106,15 @@ Every event carries page path, page title, site language, content group and stab

## Interpreting Core Web Vitals

Browser RUM events provide continuous observations from measured visits. The scheduled PageSpeed report remains the decision source for field CWV because it uses CrUX data when enough real-user samples exist. When CrUX has insufficient traffic, the report retains Lighthouse lab values and marks field data unavailable instead of inventing a pass/fail result.
Browser RUM events provide continuous observations from consented visits. The scheduled PageSpeed report remains the decision source for field CWV because it uses CrUX data when enough real-user samples exist. When CrUX has insufficient traffic, the report retains Lighthouse lab values and marks field data unavailable instead of inventing a pass/fail result.

## Continuous-improvement loop

1. Review the weekly job summary.
2. Repair any broken internal link or failed 404 behavior immediately.
3. Prioritize high-impression pages with low CTR for title, description and intent alignment.
4. Compare English and Indonesian traffic before deciding which translations to expand.
5. Trace download clicks back to the page that generated them.
6. Investigate repeated 404 paths and add a valid route or redirect where appropriate.
7. Treat poor LCP, CLS or INP as a release-quality issue, then confirm the improvement in the next field-data cycle.
1. Confirm the latest `production-pages-attestation` matches `main` before interpreting any website metric.
2. Review the weekly job summary.
3. Repair any broken internal link or failed 404 behavior immediately.
4. Prioritize high-impression pages with low CTR for title, description and intent alignment.
5. Compare English and Indonesian traffic before deciding which translations to expand.
6. Trace download clicks back to the page that generated them.
7. Investigate repeated 404 paths and add a valid route or redirect where appropriate.
8. Treat poor LCP, CLS or INP as a release-quality issue, then confirm the improvement in the next field-data cycle.
Loading
Loading