Normalize usernames for safe indexed lookup

.gitignore

@@ -43,3 +43,4 @@ client/config/.env.development
 # Local build/worktree scratch
 .turbo
 .worktrees
+.privacy-email-cutover

README.md

@@ -50,6 +50,19 @@ yarn start:client
 
 For more on the internals and contributing, check out the [wiki](https://github.com/coder13/LetsCube/wiki)
 
+## Identity privacy
+
+Let's Cube requests only the WCA OAuth `public` scope. WCA profile data is
+allowlisted at login, and the application does not request, retain, mirror,
+return, log, analyze, or search WCA email addresses or dates of birth. Product
+identity uses the WCA numeric user ID internally and exposes a WCA ID only when
+the user has explicitly enabled that existing profile preference.
+
+Any user-discovery feature must accept only its documented username and visible
+WCA ID formats. An email-like input must not be treated as an identifier or
+produce an existence signal. This is a required invariant for the Friend System,
+not a future discovery mode.
+
 ## Metrics
 
 The server stores pseudonymous room and authentication events in both the
@@ -71,9 +84,10 @@ access codes, OAuth credentials, chat content, scramble text, or solve times.
 ## PostgreSQL dual writes
 
 New MongoDB writes are mirrored into PostgreSQL without changing application
-reads. PostgreSQL receives users and preferences, rooms and participant state,
-attempts, durable solve results, and sanitized analytics events. OAuth access
-tokens are deliberately not copied. Writes use deterministic UUIDs and upserts,
+reads. PostgreSQL receives public identity and preferences, rooms and
+participant state, attempts, durable solve results, and sanitized analytics
+events. OAuth access tokens are deliberately not copied. Writes use
+deterministic UUIDs and upserts,
 so retries and future backfills are idempotent. Live room saves mirror only the
 attempts and results changed by that save; complete room snapshots are reserved
 for explicit backfills. Changing a room event explicitly replaces that room's
@@ -91,3 +105,7 @@ TLS connections can set `PGSSL=true` and provide a CA with `PGSSL_CA`;
 certificate verification is enabled by default. PostgreSQL failures are logged
 but do not fail the corresponding MongoDB-backed application operation during
 this migration phase.
+
+Username lookup uses a separately normalized, uniquely indexed key while
+preserving display casing. See [the normalized username migration](docs/username-migration.md)
+for the required collision audit, production order, and rollback procedure.

README_DEPLOYMENT.md

@@ -187,6 +187,53 @@ Keep this setting disabled until the legacy scheduler is redesigned. Setting
 it to `true` explicitly restores the old mode for development or controlled
 testing.
 
+## WCA Email Privacy Cutover
+
+The privacy cutover is deliberately separate from ordinary database migrations.
+PostgreSQL migrations run before a new application image is healthy, so using a
+migration to purge data could let the automatic rollback restore an image that
+writes the data again.
+
+Perform the cutover in this order:
+
+1. Deploy this release to both application services with `DEPLOY_TARGET=all`.
+2. Confirm the API and socket health checks pass, the browser's WCA authorization
+   request has exactly `scope=public`, and a new login succeeds.
+3. From `/opt/letscube`, record the running safe commit as the rollback floor:
+
+   ```bash
+   git rev-parse HEAD | tee .privacy-email-cutover
+   ```
+
+   Future `scripts/deploy.sh` runs refuse any commit that does not descend from
+   this floor. Do not use direct Compose commands or a saved image tag to restore
+   an older release after creating the marker.
+4. Run the purge from the newly deployed image:
+
+   ```bash
+   docker compose -f compose.yml -f compose.prod.yml --env-file .env.prod run --rm --no-deps api node server/privacy/purgeUserEmails.js
+   ```
+
+   The command reports record counts only. It never prints field values and
+   exits unsuccessfully if either database still has a value.
+5. Run the same command a second time. Every matched, modified, cleared, and
+   remaining count should be zero; this verifies that the operation is
+   idempotent.
+6. Create and verify a fresh post-cutover backup. Remove every pre-cutover local
+   and remote backup according to the storage provider's secure deletion
+   procedure; normal retention is not sufficient for historical private data.
+
+If the initial deployment rolls back before step 3, do not create the marker or
+run the purge. Correct the release and redeploy it first. After step 3, the
+privacy-safe commit is the minimum supported rollback image.
+
+The PostgreSQL `app.users.email` column remains temporarily as an always-empty
+compatibility column. Current dual writes never send it a value and clear a
+legacy value whenever they update an existing row. Removing the column is phase two:
+first deploy an application release that no longer references the compatibility
+column, advance every supported rollback image to that release, and only then
+apply a migration that drops it. Do not combine that drop with this cutover.
+
 ### Health Checks
 
 The API and socket processes expose dependency-aware health endpoints.

client/src/components/Header.jsx

@@ -1,5 +1,4 @@
 import React from 'react';
-import qs from 'qs';
 import PropTypes from 'prop-types';
 import { connect } from 'react-redux';
 import { Link } from 'react-router-dom';
@@ -15,6 +14,7 @@ import Button from '@material-ui/core/Button';
 import MenuItem from '@material-ui/core/MenuItem';
 import { apiOrigin } from '../lib/fetch';
 import { getNameFromId } from '../lib/events';
+import { getWcaAuthorizationUrl } from '../lib/wcaAuth';
 
 const useStyles = makeStyles(() => ({
   root: {
@@ -57,13 +57,13 @@ function Header({ user, room }) {
   };
 
   const login = () => {
-    localStorage.setItem('letscube.redirect_uri', `${document.location.origin}/wca-redirect`);
-    const url = `${process.env.REACT_APP_WCA_ORIGIN}/oauth/authorize?${qs.stringify({
-      response_type: 'code',
-      scope: 'public dob email',
-      redirect_uri: `${document.location.origin}/wca-redirect`,
-      client_id: process.env.REACT_APP_WCA_CLIENT_ID,
-    })}`;
+    const redirectUri = `${document.location.origin}/wca-redirect`;
+    localStorage.setItem('letscube.redirect_uri', redirectUri);
+    const url = getWcaAuthorizationUrl({
+      origin: process.env.REACT_APP_WCA_ORIGIN,
+      clientId: process.env.REACT_APP_WCA_CLIENT_ID,
+      redirectUri,
+    });
     window.location = url;
   };
 

client/src/components/common/Profile.jsx

@@ -19,12 +19,9 @@ const validate = (username) => {
     return '';
   }
 
-  if (username.indexOf(' ') > -1) {
-    return 'Username cannot contains spaces';
-  }
-
-  if (username.length >= 16) {
-    return 'Username cannot be longer than 16 characters';
+  const normalizedUsername = username.normalize('NFKC').trim();
+  if (normalizedUsername.length > 15 || !/^[A-Za-z0-9_-]+$/.test(normalizedUsername)) {
+    return 'Use 1–15 letters, numbers, underscores, or hyphens';
   }
 
   return '';

client/src/lib/wcaAuth.js

@@ -0,0 +1,12 @@
+import qs from 'qs';
+
+export const WCA_OAUTH_SCOPE = 'public';
+
+export const getWcaAuthorizationUrl = ({ origin, clientId, redirectUri }) => (
+  `${origin}/oauth/authorize?${qs.stringify({
+    response_type: 'code',
+    scope: WCA_OAUTH_SCOPE,
+    redirect_uri: redirectUri,
+    client_id: clientId,
+  })}`
+);

client/src/lib/wcaAuth.test.js

@@ -0,0 +1,15 @@
+import { getWcaAuthorizationUrl, WCA_OAUTH_SCOPE } from './wcaAuth';
+
+describe('WCA authorization request', () => {
+  it('requests only the public identity scope', () => {
+    const url = new URL(getWcaAuthorizationUrl({
+      origin: 'https://www.worldcubeassociation.org',
+      clientId: 'client-id',
+      redirectUri: 'https://letscube.net/wca-redirect',
+    }));
+
+    expect(WCA_OAUTH_SCOPE).toBe('public');
+    expect(url.searchParams.get('scope')).toBe('public');
+    expect(url.searchParams.get('scope')).not.toMatch(/dob|email/);
+  });
+});

docs/username-migration.md

@@ -0,0 +1,70 @@
+# Normalized username migration
+
+MongoDB remains the user source of truth. Usernames are optional and use two
+fields: `username` stores trimmed display casing, while `usernameNormalized`
+stores the lowercase lookup key. The normalized value is intentionally not
+returned by user serialization.
+
+Valid new usernames contain 1–15 ASCII letters, numbers, underscores, or
+hyphens. Empty input unsets both fields. Legacy invalid values are preserved but
+left without a normalized key, so they are not discoverable until their owner
+chooses a valid username. The privacy exception is an invalid value containing
+an email marker (`@`) after NFKC canonicalization, which is removed instead of
+retained. This includes compatibility forms such as fullwidth and small `@`.
+Case-insensitive legacy collisions are reported for explicit resolution; the
+migration never picks a winner or silently renames an account.
+
+## Production order
+
+Issue #191 / PR #192 is a hard prerequisite. Its WCA email scope removal and
+data purge must remain in the branch and be deployed before username discovery.
+
+1. Back up MongoDB and schedule a short maintenance window for username writes.
+2. Apply the committed PostgreSQL migrations. The nullable
+   `app.users.username_normalized` column and its unique index are additive, so
+   the previous application version continues to run.
+3. From the release checkout, audit MongoDB without changing data:
+
+   ```bash
+   yarn workspace letscube-server usernames:backfill
+   ```
+
+4. Save the JSON report. Resolve every reported collision with the account
+   owners before rollout. Do not deploy username writes or discovery while any
+   collision remains: index creation fails closed rather than leaving the name
+   available for a third account to claim. Invalid raw values are deliberately
+   omitted from the report, and email-like invalid values are removed rather
+   than copied or logged.
+5. After the collision report is empty, apply the idempotent backfill, reconcile
+   existing PostgreSQL usernames by WCA user ID, and create or verify the sparse
+   unique MongoDB index:
+
+   ```bash
+   yarn workspace letscube-server usernames:backfill --apply --create-index
+   ```
+
+   The PostgreSQL reconciliation uses MongoDB's planned target for each WCA user
+   and verifies exact equality without printing stored values. It clears both
+   PostgreSQL username fields for email-like legacy values. When PostgreSQL is
+   intentionally disabled with `POSTGRES_ENABLED=false`, the command records a
+   disabled status and skips that secondary store; rerun with PostgreSQL enabled
+   before bringing the mirror back into service.
+6. Run the dry-run command again. `pendingChanges`, `privacyRemoved`, and the
+   collision report must all be empty or zero. The apply command must report
+   zero MongoDB private values and zero PostgreSQL mismatches.
+7. Deploy the API and verify a casing-only username change, a conflicting
+   change (`409 USERNAME_TAKEN`), and an invalid change
+   (`400 INVALID_USERNAME`). PostgreSQL receives the normalized field through
+   the existing non-blocking dual writer whenever MongoDB users are saved.
+
+Do not enable username writes or the discovery endpoint from #82 until this
+sequence and the #191 purge have completed successfully.
+
+## Rollback
+
+Do not drop the MongoDB index or PostgreSQL column when rolling the application
+back. Both additions are backward-compatible and the previous application
+ignores them. Disable username discovery and username edits while the old code
+is serving because it does not maintain `usernameNormalized`. Before rolling
+forward again, rerun the dry-run and apply commands to reconcile any writes made
+by the old application.

scripts/backfill-normalized-usernames.js

@@ -0,0 +1,116 @@
+#!/usr/bin/env node
+/* eslint-disable import/no-extraneous-dependencies, no-console */
+const path = require('path');
+const mongoose = require('mongoose');
+
+process.env.GETCONFIG_ROOT = process.env.GETCONFIG_ROOT
+  || path.join(__dirname, '../server/config');
+
+const config = require('../server/runtimeConfig');
+const {
+  assertUsernameRolloutReady,
+  planUsernameBackfill,
+} = require('../server/usernameBackfill');
+const { reconcilePostgresUsernames } = require('../server/usernamePostgresBackfill');
+
+const args = new Set(process.argv.slice(2));
+const supportedArgs = new Set(['--apply', '--create-index']);
+const unknownArg = [...args].find((arg) => !supportedArgs.has(arg));
+
+const duplicateNormalizedUsernames = (collection) => collection.aggregate([
+  { $match: { usernameNormalized: { $exists: true } } },
+  { $group: { _id: '$usernameNormalized', count: { $sum: 1 } } },
+  { $match: { count: { $gt: 1 } } },
+  { $limit: 1 },
+]).toArray();
+
+const readUsers = (collection) => collection.find({}, {
+  projection: {
+    _id: 1,
+    id: 1,
+    username: 1,
+    usernameNormalized: 1,
+  },
+}).toArray();
+
+let postgresPool;
+
+const run = async () => {
+  if (unknownArg) {
+    throw new Error(`Unknown argument: ${unknownArg}`);
+  }
+
+  const apply = args.has('--apply');
+  const createIndex = args.has('--create-index');
+  if (createIndex && !apply) {
+    throw new Error('--create-index must be used with --apply');
+  }
+
+  mongoose.set('strictQuery', false);
+  await mongoose.connect(config.mongodb, { autoIndex: false });
+  const users = mongoose.connection.collection('users');
+  const documents = await readUsers(users);
+  const plan = planUsernameBackfill(documents);
+
+  console.log(JSON.stringify({
+    mode: apply ? 'apply' : 'dry-run',
+    ...plan.report,
+  }, null, 2));
+
+  if (!apply) {
+    return;
+  }
+
+  if (plan.operations.length) {
+    const result = await users.bulkWrite(plan.operations, { ordered: false });
+    console.log(`Updated ${result.modifiedCount} user records.`);
+  }
+
+  const verifiedPlan = planUsernameBackfill(await readUsers(users));
+  if (verifiedPlan.report.pendingChanges || verifiedPlan.report.privacyRemoved) {
+    throw new Error('MongoDB username backfill verification failed');
+  }
+
+  let postgres = { status: 'disabled' };
+  if (config.postgres.enabled) {
+    // Avoid creating a PostgreSQL pool for dry runs or explicitly disabled mirrors.
+    // eslint-disable-next-line global-require
+    ({ pool: postgresPool } = require('../server/postgres'));
+    postgres = {
+      status: 'reconciled',
+      ...await reconcilePostgresUsernames({
+        client: postgresPool,
+        users: verifiedPlan.postgresUsers,
+      }),
+    };
+  }
+
+  console.log(JSON.stringify({
+    verification: {
+      mongoPendingChanges: verifiedPlan.report.pendingChanges,
+      mongoPrivateValuesRemaining: verifiedPlan.report.privacyRemoved,
+      postgres,
+    },
+  }, null, 2));
+
+  if (createIndex) {
+    assertUsernameRolloutReady(verifiedPlan.report);
+    const duplicates = await duplicateNormalizedUsernames(users);
+    if (duplicates.length) {
+      throw new Error('Duplicate normalized usernames remain; unique index was not created');
+    }
+    const name = await users.createIndex(
+      { usernameNormalized: 1 },
+      { name: 'users_username_normalized_unique', sparse: true, unique: true },
+    );
+    console.log(`Verified MongoDB index ${name}.`);
+  }
+};
+
+run().catch((err) => {
+  console.error(err.message);
+  process.exitCode = 1;
+}).finally(() => Promise.allSettled([
+  mongoose.disconnect(),
+  ...(postgresPool ? [postgresPool.end()] : []),
+]));

scripts/deploy.sh

@@ -58,6 +58,19 @@ esac
 
 current_commit="$(git rev-parse HEAD)"
 
+privacy_cutover_file="${PRIVACY_EMAIL_CUTOVER_FILE:-$APP_DIR/.privacy-email-cutover}"
+if [ -f "$privacy_cutover_file" ]; then
+  privacy_cutover_commit="$(tr -d '[:space:]' < "$privacy_cutover_file")"
+  if ! [[ "$privacy_cutover_commit" =~ ^[0-9a-f]{40}$ ]]; then
+    echo "Invalid email privacy cutover commit in $privacy_cutover_file" >&2
+    exit 2
+  fi
+  if ! git merge-base --is-ancestor "$privacy_cutover_commit" "$current_commit"; then
+    echo "Refusing to deploy $current_commit: it predates the email privacy cutover." >&2
+    exit 1
+  fi
+fi
+
 if [ "$DEPLOY_TARGET" = "auto" ]; then
   missing_services=()
   for service in api socket nginx; do