wunderground-service

• Why it's built this way
• Project layout
• Setup
  • 1. Generate secrets
  • 2. Run locally
  • 3. Run via Docker
  • 4. Expose it via Cloudflare
• Admin UI walkthrough
• Station lifecycle
• Client API
  • Signing a request
  • Endpoints
  • Self-service device registration
• Operational notes
• Testing
• Changelog
  • 2026-11-01 review

Development chat: https://claude.ai/code/session_01RVqY2nF7sB9Zc9qhtZSREu

A microservice that periodically scrapes temperature readings from Wunderground personal-weather-station dashboards (via a headless-browser Playwright scraper) and serves them to registered clients over an HMAC-authenticated API. Includes a password-protected admin web UI for managing stations, refresh schedules, and registered API clients, plus an email-OTP self-registration flow for devices.

Why it's built this way

• Go, stdlib only. No SQLite driver, no bcrypt/x-crypto, no router library, no cron library -- everything here (JSON-file storage, PBKDF2 password hashing, AES-GCM secret encryption, HTTP routing via Go 1.22's net/http pattern matching, goroutine-based scheduling) is hand-built on the standard library. That was originally a workaround for the dev environment's lack of Go-module-proxy access, but it turned out to be the right call anyway: a single static binary (~8.5MB stripped, as the Dockerfile builds it), no dependency chain to audit, easy to cross-compile for orange.
• JSON file instead of SQLite. At this project's scale (a handful of stations, a handful of API clients, low write frequency) a mutex-guarded in-memory store persisted to disk on mutation is simpler and has zero dependencies. The one place this was tuned for load: nonce replay-checks and API-client last_used_at timestamps do not hit disk per-request (see internal/store/store.go), so the temperature-read hot path stays fast even under heavy client traffic.
• HMAC + nonce, not just an API key. Every client request is signed with a per-client secret over method + path + timestamp + nonce, and the server rejects both stale timestamps and reused nonces. A captured request (e.g. from a compromised network) can't be replayed. See internal/clientapi/hmac.go.
• Client secrets are encrypted at rest (AES-256-GCM), not hashed -- the server needs the plaintext to recompute each request's HMAC, so unlike passwords/sessions this can't be a one-way hash. The encryption key is derived from WU_SESSION_SECRET.

Project layout

cmd/server/           main entrypoint
cmd/setpassword/      generates WU_ADMIN_PASSWORD_HASH
internal/config/      env-driven configuration + literal .env loader
internal/store/       JSON-backed persistence (stations, watches, clients, sessions)
internal/cryptoutil/  PBKDF2, tokens, AES-GCM
internal/auth/        admin sessions (password only, no 2FA)
internal/clientapi/   HMAC request verification for registered clients
internal/netutil/     shared real-client-IP helper (Cloudflare / proxy aware)
internal/scraper/     subprocess wrapper around wunderground_scraper.py
internal/scheduler/   per-station staggered refresh timers
internal/email/       SMTP sender (device-registration OTP emails)
internal/api/         client API (HMAC-protected) + device self-registration
internal/adminapi/    admin JSON API (session-protected)
internal/web/         admin HTML UI (login/dashboard/settings)
wunderground_scraper.py   the Playwright scraper

Setup

1. Generate secrets

openssl rand -hex 32                 # -> WU_SESSION_SECRET
go run ./cmd/setpassword             # -> WU_ADMIN_PASSWORD_HASH

Copy .env.example to .env and fill in both, plus WU_ADMIN_EMAIL and the WU_SMTP_* settings -- SMTP is required for self-service device registration, which emails its one-time codes to WU_ADMIN_EMAIL. (Admin login itself has no email step.)

The server refuses to start if WU_SESSION_SECRET is still the CHANGE-ME placeholder or shorter than 32 characters, or if WU_ADMIN_PASSWORD_HASH is malformed.

Keep the hash single-quoted, exactly as setpassword prints it (WU_ADMIN_PASSWORD_HASH='pbkdf2-sha256$210000$...'). PBKDF2 hashes contain $ characters, and both bash (source .env) and Docker Compose's env_file: reader treat $... in unquoted or double-quoted values as variable expansion, silently corrupting the hash into garbage that will never verify (ask me how I know). Single-quoted values are literal for bash, Compose, systemd EnvironmentFile=, and the app's own .env loader. If you ever see login fail with a correct password, check the server's startup log first -- a malformed-hash error there means this happened; the fix is the quoting, not resetting the password.

2. Run locally

go build -o wu-server ./cmd/server && ./wu-server

The binary loads .env from the current directory itself (treating $ as a literal character), so there's no need to source or export it. The scraper runs as uv run python wunderground_scraper.py --json -- <station> from the working directory, so run uv sync and uv run playwright install chromium there once first.

Visit http://localhost:8080/admin (or whatever WU_LISTEN_ADDR you set). For plain-HTTP local testing, set WU_COOKIE_SECURE=false or the browser won't send the session cookie back.

3. Run via Docker

cp .env.example .env   # fill in secrets
docker build -t wunderground-service .
docker run -d --name wunderground-service --env-file .env -p 127.0.0.1:8080:8080 -v wunderground-data:/app/data --shm-size=1gb wunderground-service

Or merge docker-compose.snippet.yml into your existing monitoring stack's docker-compose.yml. Inside the container localhost is the container itself, so point WU_SMTP_HOST at your relay's container name, host.docker.internal (see the commented extra_hosts in the snippet), or the host's LAN IP. The image has a built-in healthcheck against /health.

4. Expose it via Cloudflare

Since this is fronted by Cloudflare (matching the other hosts): point a Cloudflare Tunnel (cloudflared) at 127.0.0.1:8080 rather than opening a port on the router. Cloudflare terminates TLS at the edge, which matters here since the HMAC scheme protects request integrity and replay, not confidentiality in transit -- you still want TLS so the signature and response body aren't sniffable.

Per-IP rate limits use Cloudflare's CF-Connecting-IP header when present, then the last (nearest-proxy) X-Forwarded-For entry, then the TCP peer address. Don't expose the port directly to untrusted networks: a client that can reach it without going through Cloudflare could set CF-Connecting-IP itself.

Admin UI walkthrough

• /login -- username + password. Check "stay signed in" for a 30-day session instead of the normal 12-hour one. Login attempts are rate-limited per IP (10 per 15 minutes).
• Dashboard (/admin) -- add/remove stations, set a per-station refresh interval (blank = use the global default; otherwise at least 30s), bulk-select stations and apply an interval to all of them at once, trigger an immediate background refresh. The Source column shows whether a station was added here (admin) or by a client watching it (client); see Station lifecycle.
• Settings (/admin/settings) -- global refresh interval (minimum 30s), stagger amount, and API client registration.

Invalid input (bad station ID, interval under 30s, ...) is reported in a banner at the top of the page rather than silently ignored.

There's no email/2FA step for admin login -- password alone gets you in. If you want to add that later, internal/email.Send() (already used for device registration) is the plumbing: SMTP with explicit, non-automatic STARTTLS.

Station lifecycle

Each station records a source:

• admin -- added in the admin panel (or via POST /admin/api/stations). Stays monitored until an admin removes it, whether or not any client watches it. Adding a station that a client already caused to be monitored upgrades it to admin.
• client -- added automatically because a client started watching it. Removed automatically, and its refresh timer stopped, once no client watches it anymore -- including when the last watching client is revoked or deleted.

Removing a station in the admin panel also drops every client's watch on it. Data files from before the source field existed are migrated on startup: stations nobody watches are marked admin (client-added stations were always removed once unwatched, so those can only have come from the admin panel); watched ones are marked client.

Station IDs are normalized to upper case everywhere (admin UI, admin API, client API) and must be 3-32 letters/digits.

Client API

From Settings -> API clients, register a new client and give it a name. The client secret is shown exactly once -- copy both the client ID and secret immediately; if lost, revoke that client and register a new one. Devices can also register themselves; see Self-service device registration.

Signing a request

Every request to /api/* (except the two registration endpoints) must include:

Header	Value
X-Client-Id	the client ID
X-Timestamp	current Unix time (seconds)
X-Nonce	a random string, unique per request (8-128 chars)
X-Signature	hex(HMAC-SHA256(secret, METHOD + "\n" + PATH + "\n" + TIMESTAMP + "\n" + NONCE))

PATH is the request path only (e.g. /api/stations), no query string, no host. Requests outside a configurable clock-skew window (WU_CLIENT_AUTH_SKEW_S, default 300s) are rejected, and any (client_id, nonce) pair can only be used once within that window. Note the signature does not cover the request body, so TLS is what protects a POST body's integrity in transit.

Kotlin sketch for the Android app:

fun signRequest(clientId: String, secret: String, method: String, path: String): Headers {
    val ts = (System.currentTimeMillis() / 1000).toString()
    val nonce = UUID.randomUUID().toString()
    val message = "$method\n$path\n$ts\n$nonce"
    val mac = Mac.getInstance("HmacSHA256")
    mac.init(SecretKeySpec(secret.toByteArray(), "HmacSHA256"))
    val signature = mac.doFinal(message.toByteArray()).joinToString("") { "%02x".format(it) }
    return Headers.Builder()
        .add("X-Client-Id", clientId)
        .add("X-Timestamp", ts)
        .add("X-Nonce", nonce)
        .add("X-Signature", signature)
        .build()
}

Endpoints

Method	Path	Description
GET	/api/stations	all stations, current readings
GET	/api/stations/{id}	one station
GET	/api/stations/{id}/temperature	just the temperature (404 if unknown, 503 if no reading yet)
GET	/api/watch	the single station this client is watching (404 if none) -- Wear OS app
POST	/api/watch	body {"station_id": "..."} -- replace this client's watch with exactly this one station (switches away from whatever it was watching before) -- Wear OS app
DELETE	/api/watch	stop watching entirely
GET	/api/watches	every station this client watches -- companion app
POST	/api/watches	body {"station_id": "..."} -- add a station to this client's watch set, without disturbing others it already watches -- companion app
DELETE	/api/watches/{id}	remove one station from this client's watch set, without touching the rest

All require the signed headers above; there is no unauthenticated read path. Every station response includes refresh_interval_s, the effective interval -- clients should poll at roughly that cadence. A newly watched station gets an immediate first scrape; watching or unwatching a station that's already monitored never triggers an extra scrape.

The singular and plural APIs share the same underlying watch data, so a client-sourced station added by one client via /api/watch and by a different client via /api/watches survives until both release it (covered by internal/store/store_test.go). /api/watch (singular, "replace" semantics) is what a single-station client like the Wear OS app uses; /api/watches (plural, "add one at a time") is for a client that tracks several stations at once, like the companion app's home-screen widgets.

Self-service device registration

Method	Path	Auth
POST	/api/register/start	none -- body {"device_name": "..."}
POST	/api/register/verify	none -- body {"registration_id": "...", "code": "..."}

For a client with no credentials yet (e.g. the mobile companion app's first run): start emails a one-time code to WU_ADMIN_EMAIL and returns a registration_id; verify with the correct code auto-issues a new API client (client_id/client_secret, returned exactly once, same as creating one from the admin panel) and the device is ready to use /api/watch(es) immediately. These are the only two endpoints in the client API with no HMAC signature requirement, since the device has nothing to sign with yet -- the OTP-to-email step is what stands in for authentication here.

Abuse limits: start is rate-limited per IP (WU_REGISTRATION_RATE_LIMIT_*, default 5 per 15 minutes), since each call sends an email. Each registration allows WU_REGISTRATION_OTP_MAX_ATTEMPTS (default 5) guesses, enforced atomically so parallel requests can't exceed it, and expires after WU_REGISTRATION_OTP_MAX_AGE_S (default 10 minutes). Device names have control characters stripped and are truncated to 64 characters. Self-registered clients show up in the admin panel's client list labeled (self-registered) and can be revoked the same way.

Operational notes

• Resource cost is dominated by the scraper, not the API. Reading cached temperatures (even at high request volume) is a fast in-memory operation. Refreshing a station spawns a headless Chromium process via Playwright, which is the actual CPU/memory cost -- WU_MAX_CONCURRENT_SCRAPES (default 2) caps how many run simultaneously regardless of how many stations are due at once.
• Scrape timeouts kill the whole process tree. The scraper runs in its own process group, and on WU_SCRAPER_TIMEOUT_S the entire group (uv, Python, Playwright's driver, Chromium) is killed, so timeouts don't leave orphaned browsers behind.
• Stagger spreads first-refresh start times of never-checked or overdue stations (e.g. after a restart) across WU_DEFAULT_STAGGER_S (or the value saved in Settings) so N stations aren't all launching a browser in the same instant. Stations that have been checked keep their natural schedule (last check + interval) when anything is rescheduled, so admin edits and client watch changes don't cause extra scrapes or postpone due ones.
• WU_DEFAULT_REFRESH_INTERVAL_S / WU_DEFAULT_STAGGER_S only seed a new data file. After that, the values saved in the admin panel's Settings page win.
• Housekeeping: expired admin sessions and never-completed device registrations are pruned at startup and every ~10 minutes.
• Graceful shutdown: SIGINT/SIGTERM stop the scheduler, flush any pending API-client last_used_at writes, and drain in-flight HTTP requests (10s timeout) before exiting.
• Rotating WU_SESSION_SECRET invalidates all admin sessions and pending device registrations, and makes existing API client secrets undecryptable (their ciphertext was encrypted with the old key) -- you'd need to re-register clients afterward. Don't rotate it casually.
• Data file (WU_DATA_PATH) is written atomically (temp file + rename) with mode 0600. It holds session-token hashes, CSRF tokens, and encrypted client secrets, so keep it out of version control and backups you'd share (data/ is gitignored).

Testing

go vet ./... && go test -race ./...

Unit tests cover the store (shared singular/plural watch semantics, admin-pinned stations, client revoke/delete cleanup, legacy-data migration, interval validation, atomic OTP attempt counting under concurrency, nonce replay), the scheduler's delay/stagger logic, PBKDF2 against the RFC 7914 test vector, password hashing, AES-GCM round-trips, the .env loader's literal $ handling, and client-IP extraction.

Changelog

2026-11-01 review

Security:

• Device-registration OTP attempts are now counted atomically; previously parallel verify requests could all pass the "attempts < max" check before any incremented it, making the 6-digit code brute-forceable.
• Per-IP rate limiting no longer trusts the client-supplied first X-Forwarded-For entry or includes the source port; shared internal/netutil.ClientIP replaces three divergent copies.
• WU_SESSION_SECRET placeholder/short values are now fatal at startup (the old check compared against a different placeholder than .env.example used, so it never fired).
• setpassword prints the hash single-quoted (Docker Compose interpolates $ in unquoted and double-quoted env_file values), and the hash is unquoted on load so docker run --env-file works too. The .env.example claim that Compose env_file is literal was corrected.
• OTP digits use rejection sampling (no modulo bias); OTP and CSRF comparisons are constant-time; JSON request bodies are size-capped; device names are sanitized; email headers reject CR/LF and now include Date/Message-ID; SMTP error details are logged instead of returned to unauthenticated callers.
• Logout now requires the CSRF token.

Correctness:

• Admin-added stations are no longer deleted when the last client stops watching them (new source field, with migration of existing data).
• Watch + station add/remove happen under one lock, fixing races that could leave a client watching a deleted, never-refreshed station.
• Revoking or deleting a client releases its watches, so its stations stop being scraped.
• Rescheduling no longer scrapes a random station immediately (unsorted map order + offset 0) on every watch change or admin edit; stations keep their natural next-due time, and new stations get exactly one immediate scrape.
• Server-side validation: intervals must be blank or >= 30s and stagger >= 0 (a global interval of 0 previously caused a back-to-back Chromium launch loop). A saved stagger of 0 is no longer reset to the default on restart.
• Station IDs are normalized/validated everywhere (no more duplicate kabc/KABC stations, no IDs parsed as scraper flags); the scraper is invoked with --json -- <id>.
• Scrape timeouts kill the whole process group instead of leaving Python/Chromium orphans.
• Admin "Refresh now" in the HTML UI runs in the background; the server write timeout now scales with WU_SCRAPER_TIMEOUT_S so the JSON API's synchronous refresh isn't cut off.
• Admin form errors are shown instead of silently ignored; expired sessions/registrations are pruned.
• The scraper reports Playwright failures (e.g. missing browser) as one line instead of a full traceback in the dashboard tooltip.

Packaging and docs:

• Docker healthcheck fixed (the slim image has no wget, so the container was always "unhealthy"); now built into the Dockerfile using Python. Binaries are stripped; uv never re-syncs or downloads Python at runtime.
• Duplicate SMTP block removed from .env.example; README corrected (SMTP is required, internal/email is in use, no "trusted devices", tests now actually exist).
• Prebuilt wu-server binary and the data/ directory (which contained a live session record) are no longer shipped in the archive; both were already gitignored.