Skip to content

Send the networks the TSPU cuts off to openipc.ru with a 301 - #381

Merged
widgetii merged 2 commits into
masterfrom
blocked-nets-redirect
Oct 3, 2026
Merged

widgetii merged 2 commits into
masterfrom
blocked-nets-redirect

Conversation

@widgetii

@widgetii widgetii commented Oct 3, 2026

Copy link
Copy Markdown
Member

In Russia the TSPU freezes a TLS connection to the origin's network after the first 20-odd kilobytes. Camera uploads from Russian providers die mid-body and end as nginx 408s: 561 on 2026-10-02, 300-570 a day since at least 2026-09-19.

Evidence (production, 2026-10-02/03)

  • Packet capture on the origin, 11:45 UTC: four uploads from Rostelecom, KES and SOVAM addresses completed TCP and TLS, delivered 22.8-24.9 KB with every segment ACKed, then went silent in both directions. nginx's FIN at the 60 s body timeout was retransmitted seven times and never acknowledged.
  • In the same period every upload that came through openipc.ru (proxied from a host inside Russia) was stored: 552 × 201, 8 × 429, no 408.

Change

  • conf.d/openipc-blocked-nets.conf: geo $realip_remote_addr $openipc_blocked_net, loading openipc-blocked-nets.list — every network announced by an AS registered in Russia (4,999 ASes, 12,334 IPv4 + 1,905 IPv6 networks), generated by deploy/blocked-nets.py from iptoasn.com and committed.
  • sites-available/org.openipc, HTTPS server: if ($openipc_blocked_net) { return 301 https://openipc.ru$request_uri; } — pages, APIs and POST /snapshots alike. Port 80 is untouched (stock firmware fetches ipctool and posts reports over plain HTTP).
  • Keyed on the peer, not the realip-rewritten address: behind a mirror the reader is Russian, and openipc.ru's own host is in a Russian AS (197695), so every set_real_ip_from address is listed as not blocked. Otherwise the mirror's requests would be redirected back to it.

Known consequence: cameras on current firmware run curl without -L, so they do not follow the 301. Five Russian cameras that upload directly today (355 uploads stored since 2026-10-02) stop until their firmware follows redirects (curl -L --post301, no mirror named). The six that freeze today are already failing.

Tests

  • deploy/nginx/check-config.sh --seam (nginx 1.26 in a container): from a stand-in blocked address, GET /ru/get-started?a=1, POST /snapshots and GET /api/v1/wizard/… each answer 301 to the same path and query on openipc.ru; the rest of the seam unchanged.
  • service/deploytest TestBlockedNetsGoToTheMirror: peer key, every trusted mirror exempt, list well-formed and covering the frozen cameras but not the origin or GitHub, redirect on HTTPS only and not on dev. Keying on $remote_addr and dropping a mirror's exemption were each checked to fail it.
  • deploy/push-nginx.sh dry run: the host differs from this branch only by the vhost lines and the two new files.

Not yet applied to the host.

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Redirect Russian-network HTTPS traffic to the openipc.ru mirror

🐞 Bug fix ⚙️ Configuration changes 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Redirect affected HTTPS requests to openipc.ru before origin connections stall mid-upload.
• Generate and commit Russian-AS network ranges, exempting trusted mirrors to prevent redirect
 loops.
• Test redirect coverage and preserve plain-HTTP routes used by stock firmware.
Diagram

graph TD
  Table["iptoasn table"] --> Generator["Network generator"] --> List["Committed CIDRs"] --> Geo["Peer-address geo"] --> Decision{"Blocked peer?"} -->|yes| Mirror["openipc.ru mirror"]
  Peer["Incoming peer"] --> Geo
  Decision -->|no| Origin["Origin routes"]
  Peer -->|plain HTTP| Origin
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Enable firmware redirect following before redirecting uploads
  • ➕ Avoids interrupting uploads from cameras that currently reach the origin.
  • ➖ Leaves already-stalling uploads unresolved until firmware is updated and deployed.
2. Redirect page and API requests but exclude snapshots
  • ➕ Preserves uploads from current firmware that does not follow redirects.
  • ➖ Does not fix the mid-body snapshot failures motivating the change.

Recommendation: The origin-side, peer-keyed redirect is a reasonable immediate response to the measured connection stalls, and mirror exemptions are essential. Treat deployment as a deliberate compatibility trade-off: current firmware without redirect following will stop uploading even from Russian cameras whose direct uploads still succeed.

Files changed (7) +14494 / -2

Bug fix (1) +6 / -0
org.openipcRedirect matching HTTPS requests to openipc.ru +6/-0

Redirect matching HTTPS requests to openipc.ru

• Adds a server-level 301 to the same path and query on openipc.ru for matched peers, including snapshot uploads. The plain-HTTP server remains unchanged.

deploy/nginx/sites-available/org.openipc

Tests (2) +113 / -1
check-config.shExercise blocked-peer redirects in the nginx seam +21/-1

Exercise blocked-peer redirects in the nginx seam

• Copies the generated list into the container, adds a stand-in blocked address, and checks that HTTPS GET and POST requests receive a 301 preserving their path and query.

deploy/nginx/check-config.sh

blocked_nets_test.goGuard peer matching, exemptions, and redirect scope +92/-0

Guard peer matching, exemptions, and redirect scope

• Checks the geo key, trusted-mirror exemptions, CIDR formatting and representative coverage. Also verifies that only the production HTTPS vhost redirects and preserves the request URI.

service/deploytest/blocked_nets_test.go

Documentation (1) +4 / -1
CLAUDE.mdDocument the blocked-network mirror redirect +4/-1

Document the blocked-network mirror redirect

• Adds the Russian-AS redirect to the site's routing overview and points maintainers to the generator and nginx configuration.

CLAUDE.md

Other (3) +14371 / -0
blocked-nets.pyGenerate collapsed Russian-AS CIDRs +88/-0

Generate collapsed Russian-AS CIDRs

• Fetches or reads an iptoasn table, filters ranges by AS registration country, and writes collapsed IPv4 and IPv6 nginx rules. Rejects implausibly small input before replacing the output.

deploy/blocked-nets.py

openipc-blocked-nets.confClassify connecting peers with mirror exemptions +41/-0

Classify connecting peers with mirror exemptions

• Defines an nginx geo rule keyed on $realip_remote_addr and includes the generated CIDRs. Explicitly exempts trusted mirror peers so proxied requests cannot redirect back to a mirror.

deploy/nginx/conf.d/openipc-blocked-nets.conf

openipc-blocked-nets.listCommit generated Russian-AS network rules +14242/-0

Commit generated Russian-AS network rules

• Contains the generated CIDR rules loaded by nginx: 12,334 IPv4 and 1,905 IPv6 networks from 4,999 ASes. This file is present in the checkout and described as committed, although its contents were omitted from the supplied diff.

deploy/nginx/conf.d/openipc-blocked-nets.list

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Russian readers cannot use club actions ✓ Resolved
Description
org.openipc redirects blocked readers to openipc.ru, but the club API's post middleware accepts
a browser Origin only when it matches the configured openipc.org site URL. Browser POSTs made from
the mirror therefore receive 403 before Telegram or email sign-in, reports, and other club actions
reach their handlers.
Code

deploy/nginx/sites-available/org.openipc[R109-110]

+    if ($openipc_blocked_net) {
+        return 301 https://openipc.ru$request_uri;
Evidence
The redirect makes openipc.ru the browser's origin. Club calls use relative URLs, while the API
compares the browser's Origin to its configured site URL and rejects a mismatch before invoking the
handler.

deploy/nginx/sites-available/org.openipc[107-111]
frontend/apps/site/src/lib/club.ts[69-74]
service/internal/club/api.go[65-110]
service/internal/config/config.go[35-38]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new redirect moves blocked readers onto openipc.ru, where the club API rejects their browser POSTs because its origin check expects openipc.org.
## Fix Focus Areas
- deploy/nginx/sites-available/org.openipc[109-110]
- service/internal/club/api.go[88-110]
- frontend/apps/site/src/lib/club.ts[69-74]
## Recommended Fix
Allow the explicitly trusted Russian mirror as a valid club POST origin while retaining rejection of unrelated origins. Add tests for both the mirror and an untrusted origin.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Russian users cannot upload large reports ✓ Resolved
Description
org.openipc redirects blocked HTTPS report submissions to openipc.ru, whose proxy caps request
bodies at 1 MiB instead of the origin report location's 300 MiB. A redirected submission containing
a larger photo, document, or backup receives 413 at the mirror before the report handler can process
it.
Code

deploy/nginx/sites-available/org.openipc[R109-110]

+    if ($openipc_blocked_net) {
+        return 301 https://openipc.ru$request_uri;
Evidence
The new rule redirects the report endpoint; the mirror catch-all limits bodies to 1 MiB, whereas the
origin permits 300 MiB and the service accepts individual photos and documents up to 20 MiB.

deploy/nginx/sites-available/org.openipc[107-111]
deploy/nginx/mirrors/ru.openipc.snippet[104-137]
deploy/nginx/sites-available/org.openipc[373-390]
service/internal/reports/handler.go[24-31]
service/internal/reports/handler.go[38-47]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The redirect sends blocked users' HTTPS report uploads through a mirror limited to 1 MiB, while the origin accepts reports up to 300 MiB.
## Fix Focus Areas
- deploy/nginx/sites-available/org.openipc[109-110]
- deploy/nginx/mirrors/ru.openipc.snippet[104-137]
- deploy/nginx/sites-available/org.openipc[373-390]
## Recommended Fix
Configure the Russian mirror's report upload path with the origin's body-size and upload timeout settings before enabling the redirect for that path. Verify a report larger than 1 MiB reaches the origin through the mirror.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can copy the agent prompt from any finding and feed it to your IDE agent

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread deploy/nginx/sites-available/org.openipc
Comment thread deploy/nginx/sites-available/org.openipc
@widgetii

widgetii commented Oct 3, 2026

Copy link
Copy Markdown
Member Author

Both Qodo findings addressed in e270945:

  1. Club POSTs from the mirror — club.post now also accepts the origins of the names that proxy the site (CLUB_MIRROR_ORIGINS, default openipc.ru, опенипц.рф, openipc.kz, openipc.cloud). Other origins are still refused; TestAMirrorsPageCanPost covers the mirror, plain-HTTP and look-alike origins.
  2. Report uploads through openipc.ru — the mirror now has location ~ ^/api/v1/(?:reports$|club/) with the origin's limits (300m, 300 s, unbuffered); deploytest fails if they drift from the origin's. Applied on the mirror host; a 2 MB body through https://openipc.ru/api/v1/reports now reaches the service (which answered it) instead of the mirror's 413.

In Russia the TSPU freezes a TLS connection to the origin's network after the
first 20-odd kilobytes. Captured on the origin 2026-10-03: four camera uploads
from Rostelecom, KES and SOVAM addresses completed the handshake, delivered
22.8-24.9 KB and then went silent both ways; nginx's FIN at its 60 s body
timeout was retransmitted seven times unanswered. 561 uploads ended as 408s
that way on 2026-10-02, while all 552 that came through openipc.ru, proxied
from a host inside Russia, were stored.

The origin now answers every network announced by an AS registered in
Russia with a 301 to the same address on https://openipc.ru, uploads
included, before reading anything else -- the answer is small enough to
arrive before the freeze. The decision is the site's, so no client names a
mirror.

- conf.d/openipc-blocked-nets.list: 4,999 ASes, 12,334 IPv4 and 1,905 IPv6
  networks, generated by deploy/blocked-nets.py from iptoasn.com's table and
  committed; rerun the script to refresh it.
- The geo is keyed on the peer ($realip_remote_addr): behind a mirror the
  reader is Russian, and openipc.ru's own host is in a Russian AS, so every
  set_real_ip_from address is listed as not blocked. deploytest fails if one
  is missing or the key changes, and both mutations were checked to fail it.
- HTTPS server only: stock firmware fetches ipctool and posts reports over
  plain HTTP on port 80, and has no TLS to follow a redirect into.
- check-config.sh --seam sends GET, POST /snapshots and an API GET from a
  stand-in blocked address and requires a 301 to the same path and query on
  openipc.ru.
…gh openipc.ru

Two consequences of sending Russian networks to openipc.ru (Qodo, #381):

- The club refused every POST whose Origin was not CLUB_SITE_URL, so a
  reader on openipc.ru could not sign in or send. The origins of the names
  that proxy the site (CLUB_MIRROR_ORIGINS, defaulting to openipc.ru,
  опенипц.рф, openipc.kz and openipc.cloud) are accepted as well; any other
  origin is still refused, and a test holds both sides.
- openipc.ru capped every request at 1m, so a report with a photo in it got
  a 413 from the mirror. It now has a location for /api/v1/reports and
  /api/v1/club/ with the origin's limits (300m, five minutes, unbuffered),
  and deploytest fails if the two drift apart. Applied on the host and
  checked: a 2 MB body through https://openipc.ru/api/v1/reports reached the
  service, which answered it itself.
@widgetii
widgetii force-pushed the blocked-nets-redirect branch from 20f429f to fcf1d16 Compare October 3, 2026 13:18
@widgetii
widgetii merged commit d6f41c4 into master Oct 3, 2026
4 checks passed
@widgetii
widgetii deleted the blocked-nets-redirect branch October 3, 2026 13:26
widgetii added a commit to OpenIPC/majestic-webui that referenced this pull request Oct 3, 2026
openipc.org now answers networks that cannot reach it intact with a 301 to
a mirror (OpenIPC/website#381). In Russia the TSPU freezes an upload to it
after the first 20-odd kilobytes: 561 uploads ended as 408s on 2026-10-02.
Which networks, and which mirror, is the site's decision, so nothing is
named here; the camera only has to follow.

curl follows nothing without --location, and on a 301 or 302 it turns a
POST into a GET unless told --post301/--post302, which would lose the
picture. --max-redirs 3 bounds a loop.

Checked by running the script in a container with the camera's commands
stubbed: a server answering the POST with 301 to a second one, and the
second received a POST of 251,290 bytes carrying the file and mac_address
parts, and answered 201.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant