Migrate external EPP proxy to integrated Nomulus EPP server

[ptkach]

Migrate EPP traffic handling from the external EPP proxy container to an integrated, in-process Netty-based EPP server running within the Nomulus frontend container.

1. Integrated Netty EPP Server:
   • Replaces the standalone proxy container with an in-process Netty TCP server (EppServer) listening for registrar EPP traffic.
   • Forwards EPP requests to EPP flows via a servlet-compatible bridge (EppServiceHandler -> FakeHttpServletRequest -> RegistryServlet).
   • Offloads EPP command execution to a dedicated business thread pool (businessGroup) to prevent blocking the Netty I/O loop.
2. Network Topology & Client IP Preservation:
   • Deploys a new epp-server GKE service using a Layer 4 regional Load Balancer with externalTrafficPolicy: Local to preserve the original client IP.
   • Extracts and propagates client IP and certificate hash through custom headers (Nomulus-Client-Address and X-SSL-Certificate) to EPP validation flows.
   • Configures a TCP socket readiness probe on the GKE EPP server deployment for health monitoring.
3. Throttling & Rate-Limiting:
   • Implements thread-safe local connection limiting per pod for both IP and Certificate scopes using atomic map operations.
   • Enforces distributed command-level rate limiting via QuotaManager utilizing Valkey (Redis) Lua scripts.
4. Lifecycle and Metrics Consolidation:
   • Introduces EppServerLifecycleListener to manage EPP Netty server bootstrap and teardown.
   • Consolidates the JVM-wide MetricReporter startup and shutdown under MetricsLifecycleListener to ensure JVM and EPP server metrics are correctly pushed to Stackdriver.

This change is 

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[jianglai]

FWIW I think most of the new code is copied from the proxy code base. You should use git mv, which would make this PR much easier to review, not to mention to retain the history from the proxy code base.

[ptkach]

Unfortunately that wasn't possible as we want to keep old proxy running in parallel with the new one, while we're testing the new service and it might take a while, as such the old proxy deletion will be a separate change.

@ptkach made 1 comment.
Reviewable status: 0 of 42 files reviewed, 15 unresolved discussions.

[jianglai]

That's too bad. But I think you can salvage the situation by making a copy of the current proxy code (therefore losing the history), updating your build process to build the proxy from the copy, then renaming/refactoring your existing proxy code into the epp server. There is more work, but you get a cleaner result in the end where the full history is preserved, and the refactor is easier to review as only the changes from proxy to epp server are highlighted.

I think you can ask agent to do the copying part and it should do a decent job. But it's ultimately your decision how you want to handle it.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[gbrodman]

nit: should change the commit message for more details and to use an action verb

@gbrodman reviewed 44 files and all commit messages, and made 9 comments.
Reviewable status: all files reviewed, 23 unresolved discussions (waiting on ptkach).

---

core/src/main/java/google/registry/config/files/default-config.yaml line 437 at r6 (raw file):

eppServer:
  # The integrated EPP server listens on this port.
  port: 30002

should we default to 700? i haven't read through the rest of this PR yet so i don't know if that's a valid question

---

core/src/main/java/google/registry/eppserver/EppServer.java line 80 at r6 (raw file):

        ChannelHandler handler = handlerProvider.get();
        String handlerName = handler.getClass().getSimpleName();
        if (handlerName.equals("EppServiceHandler")) {

i think we can just compare handler.getClass() to the class directly, rather than hardcoding the class simple name?

---

core/src/main/java/google/registry/eppserver/handler/EppServiceHandler.java line 146 at r6 (raw file):

    // 2. Trigger initial EPP <greeting>
    handleEppFrame(ctx, Unpooled.wrappedBuffer(helloBytes));

i think this is happening on the main IO thread right? since this just happens after we resolve the cert. I am pretty sure we want to offload this to an executor so that the initial hello isn't blocking the IO thread

---

core/src/main/java/google/registry/eppserver/quota/LocalConnectionLimiter.java line 77 at r6 (raw file):

  }

  private void release(String key, ConcurrentHashMap<String, AtomicInteger> map) {

i think we need to computeIfAbsent this too otherwise one thread could increment an AtomicInteger that has already been removed from the map

---

core/src/main/java/google/registry/eppserver/quota/QuotaManager.java line 40 at r6 (raw file):

  /** Lua script to atomically decrement a token bucket with a TTL. */
  private static final String DECR_LUA =

you could probably use triplequoted strings for these , your call if that's cleaner

---

core/src/main/java/google/registry/util/FakeHttpServletRequest.java line 1 at r6 (raw file):

// Copyright 2024 The Nomulus Authors. All Rights Reserved.

2026? worth checking copyright dates on all the files since there's a ton of variance. Though I'm not sure how to treat copyright dates on the moved proxy files

---

core/src/main/java/google/registry/util/FakeHttpServletRequest.java line 53 at r6 (raw file):

  private final Map<String, String> headers = new HashMap<>();
  private final Map<String, Object> attributes = new HashMap<>();
  private String method = "POST";

i think many of these can be final?

---

jetty/deploy-nomulus-for-env.sh line 36 at r6 (raw file):

echo "Updating cluster ${parts[0]} in location ${parts[1]}..."
gcloud container fleet memberships get-credentials "${parts[0]}" --project "${project}"
for service in frontend backend pubapi console epp-server

are we just going to have this open for anyone to connect to if they want? just double checking

[gbrodman]

@gbrodman made 1 comment.
Reviewable status: all files reviewed, 24 unresolved discussions (waiting on ptkach).

---

core/src/main/java/google/registry/util/FakeHttpServletResponse.java line 68 at r6 (raw file):

  @Override
  public void addHeader(String name, String value) {
    headers.put(name, value);

not a huge deal, but i'm p sure this should append the name+value on to the existing header

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[ptkach]

Updated PR message and description, update commit title

@ptkach made 7 comments.
Reviewable status: 38 of 44 files reviewed, 24 unresolved discussions (waiting on gbrodman).

---

core/src/main/java/google/registry/config/files/default-config.yaml line 437 at r6 (raw file):

Previously, gbrodman wrote…

should we default to 700? i haven't read through the rest of this PR yet so i don't know if that's a valid question

30002 is an internal port, external port is 700, same as the old proxy

---

core/src/main/java/google/registry/eppserver/EppServer.java line 80 at r6 (raw file):

Previously, gbrodman wrote…

i think we can just compare handler.getClass() to the class directly, rather than hardcoding the class simple name?

Done

---

core/src/main/java/google/registry/eppserver/handler/EppServiceHandler.java line 146 at r6 (raw file):

Previously, gbrodman wrote…

i think this is happening on the main IO thread right? since this just happens after we resolve the cert. I am pretty sure we want to offload this to an executor so that the initial hello isn't blocking the IO thread

Good catch, I've copied it from proxy as is, but it works differently there so now executor is required.

---

core/src/main/java/google/registry/eppserver/quota/LocalConnectionLimiter.java line 77 at r6 (raw file):

Previously, gbrodman wrote…

i think we need to computeIfAbsent this too otherwise one thread could increment an AtomicInteger that has already been removed from the map

Good point, updated

---

core/src/main/java/google/registry/util/FakeHttpServletRequest.java line 53 at r6 (raw file):

Previously, gbrodman wrote…

i think many of these can be final?

Updated servername and contentType, method, requestUri, and body can not be set as final they have setters

---

jetty/deploy-nomulus-for-env.sh line 36 at r6 (raw file):

Previously, gbrodman wrote…

are we just going to have this open for anyone to connect to if they want? just double checking

Discussed offline

[gbrodman]

@gbrodman reviewed 6 files and all commit messages, made 2 comments, and resolved 4 discussions.
Reviewable status: all files reviewed, 20 unresolved discussions (waiting on ptkach).

---

core/src/main/java/google/registry/eppserver/quota/LocalConnectionLimiter.java line 77 at r6 (raw file):

Previously, ptkach (Pavlo Tkach) wrote…

Good point, updated

whoops yeah, I meant computeIfPresent as you noted

I don't think we need or want to remove the AtomicIntegers entirely, as that makes acquire more complicated. I think you could keep acquire as is, and just change this to

computeIfPresent(
key,
(k, v) -> {
  if (v.decrementAndGet() <= 0) {
    return null;
  }
  return v;
}

that way we can be sure that the AtomicIntegers returned by computeIfPresent and computeIfAbsent will always be the "same" object.

---

core/src/main/java/google/registry/eppserver/EppServer.java line 81 at r7 (raw file):

        ChannelHandler handler = handlerProvider.get();
        String handlerName = handler.getClass().getSimpleName();
        if (handler.getClass() == EppServiceHandler.class) {

TIL you can use double-equals for class objects

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[ptkach]

@ptkach made 4 comments.
Reviewable status: 41 of 44 files reviewed, 20 unresolved discussions (waiting on gbrodman).

---

core/src/main/java/google/registry/eppserver/quota/LocalConnectionLimiter.java line 77 at r6 (raw file):

Previously, gbrodman wrote…

whoops yeah, I meant computeIfPresent as you noted

I don't think we need or want to remove the AtomicIntegers entirely, as that makes acquire more complicated. I think you could keep acquire as is, and just change this to

computeIfPresent(
key,
(k, v) -> {
  if (v.decrementAndGet() <= 0) {
    return null;
  }
  return v;
}

that way we can be sure that the AtomicIntegers returned by computeIfPresent and computeIfAbsent will always be the "same" object.

That's what I did the first time, but AI still validated this as a race condition. Integet approach makes it bullet-proof. The race condition with what you suggested is the following:

1. Initial State: Map contains {"IP_A": AtomicInteger(1)} (1 active connection).
2. Thread B (Acquire): Calls map.computeIfAbsent("IP_A", ...) which returns the reference to the existing AtomicInteger (value 1).
3. Thread A (Release): Calls map.computeIfPresent("IP_A", ...) which locks key "IP_A".
   • It decrements the AtomicInteger value to 0.
   • Since 0 <= 0, it returns null which removes "IP_A" from the map.
   • It releases the lock.
4. Thread B: Calls count.incrementAndGet() on the reference it obtained in Step 2. The value becomes 1. Thread B thinks it successfully acquired a connection.
5. Current State:
   • Connection B is active (represented by the orphaned AtomicInteger of value 1 which is no longer in the map).
   • The map has no entry for "IP_A".
6. Thread C (Acquire): Calls map.computeIfAbsent("IP_A", ...) -> since the key is missing, it creates a new AtomicInteger(0) and puts it in the map.
7. Thread C: Calls count.incrementAndGet() -> value becomes 1. (Thread C successfully acquires).
8. State: Both Connection B and Connection C are active (2 connections). The map only contains {"IP_A": AtomicInteger(1)} (representing C).
9. Thread B (Release): Calls map.computeIfPresent("IP_A", ...) -> obtains reference to Thread C's AtomicInteger (value 1).
10. Thread B: Decrements Thread C's AtomicInteger to 0 and returns null -> removes "IP_A" from the map.

---

core/src/main/java/google/registry/eppserver/quota/QuotaManager.java line 40 at r6 (raw file):

Previously, gbrodman wrote…

you could probably use triplequoted strings for these , your call if that's cleaner

Thats' fine for now

---

core/src/main/java/google/registry/util/FakeHttpServletRequest.java line 1 at r6 (raw file):

Previously, gbrodman wrote…

2026? worth checking copyright dates on all the files since there's a ton of variance. Though I'm not sure how to treat copyright dates on the moved proxy files

Repaced

---

core/src/main/java/google/registry/util/FakeHttpServletResponse.java line 68 at r6 (raw file):

Previously, gbrodman wrote…

not a huge deal, but i'm p sure this should append the name+value on to the existing header

Done.

[gbrodman]

@gbrodman reviewed 3 files and all commit messages, made 2 comments, and resolved 5 discussions.
Reviewable status: all files reviewed, 15 unresolved discussions (waiting on ptkach).

---

core/src/main/java/google/registry/eppserver/quota/LocalConnectionLimiter.java line 77 at r6 (raw file):

Previously, ptkach (Pavlo Tkach) wrote…

That's what I did the first time, but AI still validated this as a race condition. Integet approach makes it bullet-proof. The race condition with what you suggested is the following:

1. Initial State: Map contains {"IP_A": AtomicInteger(1)} (1 active connection).
2. Thread B (Acquire): Calls map.computeIfAbsent("IP_A", ...) which returns the reference to the existing AtomicInteger (value 1).
3. Thread A (Release): Calls map.computeIfPresent("IP_A", ...) which locks key "IP_A".
   • It decrements the AtomicInteger value to 0.
   • Since 0 <= 0, it returns null which removes "IP_A" from the map.
   • It releases the lock.
4. Thread B: Calls count.incrementAndGet() on the reference it obtained in Step 2. The value becomes 1. Thread B thinks it successfully acquired a connection.
5. Current State:
   • Connection B is active (represented by the orphaned AtomicInteger of value 1 which is no longer in the map).
   • The map has no entry for "IP_A".
6. Thread C (Acquire): Calls map.computeIfAbsent("IP_A", ...) -> since the key is missing, it creates a new AtomicInteger(0) and puts it in the map.
7. Thread C: Calls count.incrementAndGet() -> value becomes 1. (Thread C successfully acquires).
8. State: Both Connection B and Connection C are active (2 connections). The map only contains {"IP_A": AtomicInteger(1)} (representing C).
9. Thread B (Release): Calls map.computeIfPresent("IP_A", ...) -> obtains reference to Thread C's AtomicInteger (value 1).
10. Thread B: Decrements Thread C's AtomicInteger to 0 and returns null -> removes "IP_A" from the map.

yeah i see what you mean, we def need to essentially have the whole thing in a lock (which is what this now does).

---

core/src/main/java/google/registry/eppserver/quota/LocalConnectionLimiter.java line 78 at r8 (raw file):

            return v + 1;
          }
          acquired[0] = false;

this line isn't necessary, though idk if the compiler complains if you get rid of it

you could also replace the contents if (v == null) {...} block with just v = 0 if you want, then it'll go through the other normal v < limit path. Either way probably fine.

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

[gbrodman]

@gbrodman reviewed 7 files and all commit messages.
Reviewable status: all files reviewed, 15 unresolved discussions (waiting on ptkach).

relevant issues have been resolved

[github-actions]

⚠️ Attention Required: Lockfile Detected

This pull request contains modifications to one or more *.lockfile files. Please confirm that you have run update_dependency.sh to push new dependencies to the private repo.

Someone with Admin role must manually dismiss this review before merging.

relevant issues have been resolved