RANGER-5371: pdp server implementation by mneethiraj · Pull Request #896 · apache/ranger

mneethiraj · 2026-03-28T02:27:40Z

What changes were proposed in this pull request?

introduce Ranger PDP (policy decision point) server with REST APIs to evaluate authorization requests
updated Ranger Python client to support Ranger PDP REST APIs
updated docker setup to support ranger-pdp server

How was this patch tested?

verified that ranger-pdp server starts successfully in docker setup
verified that Ranger Python client successfully calls PDP REST APIs, and PDP server audits authorizations

Copilot

Pull request overview

This PR introduces a new Ranger PDP (Policy Decision Point) server module that exposes authorization evaluation over REST, updates the Python client to call the PDP APIs, and extends distro + docker tooling to package/run the new server.

Changes:

Added a new pdp Maven module implementing an embedded-Tomcat/Jersey REST server with authn (header/JWT/SPNEGO), request context, health endpoints, and Prometheus-style metrics.
Added a Python RangerPDPClient plus new authz request/response model classes for calling PDP REST APIs.
Updated distribution assembly and docker-compose setup to build/package/run the new ranger-pdp artifact.

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
pom.xml	Adds the new `pdp` module to the multi-module build.
pdp/src/test/java/org/apache/ranger/pdp/security/RangerPdpRequestContextFilterTest.java	Tests request-id propagation and MDC cleanup for request context filter.
pdp/src/test/java/org/apache/ranger/pdp/security/RangerPdpAuthFilterTest.java	Tests auth filter handler registration logic.
pdp/src/test/java/org/apache/ranger/pdp/security/KerberosAuthHandlerTest.java	Tests Kerberos principal short-name mapping logic.
pdp/src/test/java/org/apache/ranger/pdp/security/HttpHeaderAuthHandlerTest.java	Tests header-based authentication behavior.
pdp/src/test/java/org/apache/ranger/pdp/rest/RangerPdpRESTTest.java	Unit tests for REST endpoints and stats updates.
pdp/src/test/java/org/apache/ranger/pdp/config/RangerPdpConfigTest.java	Tests config overrides (system properties, port parsing).
pdp/src/test/java/org/apache/ranger/pdp/RangerPdpStatusServletTest.java	Tests readiness/metrics servlet output and policy-cache age logic.
pdp/src/test/java/org/apache/ranger/pdp/RangerPdpStatsTest.java	Tests stats counters and latency averaging/clamping.
pdp/src/main/resources/ranger-pdp-default.xml	Default PDP configuration (port, authn, authorizer, audit, etc.).
pdp/src/main/java/org/apache/ranger/pdp/security/RangerPdpRequestContextFilter.java	Filter to set requestId in MDC/request/response header.
pdp/src/main/java/org/apache/ranger/pdp/security/RangerPdpAuthFilter.java	Main authn filter selecting among header/JWT/SPNEGO handlers.
pdp/src/main/java/org/apache/ranger/pdp/security/PdpAuthHandler.java	Auth handler interface + result contract.
pdp/src/main/java/org/apache/ranger/pdp/security/KerberosAuthHandler.java	SPNEGO/Kerberos authentication handler implementation.
pdp/src/main/java/org/apache/ranger/pdp/security/JwtAuthHandler.java	JWT bearer-token authentication handler (delegates to ranger-authn).
pdp/src/main/java/org/apache/ranger/pdp/security/HttpHeaderAuthHandler.java	Trusted-header authentication handler.
pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java	JAX-RS resource exposing authorize/authorizeMulti/permissions endpoints.
pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpApplication.java	Jersey `ResourceConfig` wiring for REST, Jackson, and DI bindings.
pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConstants.java	PDP constants for config keys, attributes, and prefixes.
pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConfig.java	Reads/merges default + site XML config and applies `-D` overrides.
pdp/src/main/java/org/apache/ranger/pdp/RangerPdpStatusServlet.java	Live/ready/metrics endpoints for health checks and Prometheus scraping.
pdp/src/main/java/org/apache/ranger/pdp/RangerPdpStats.java	Runtime stats counters and health state flags.
pdp/src/main/java/org/apache/ranger/pdp/RangerPdpServer.java	Embedded Tomcat bootstrap + wiring (filters, Jersey servlet, endpoints).
pdp/scripts/ranger-pdp.sh	Init-style wrapper script for start/stop/restart/status.
pdp/scripts/ranger-pdp-services.sh	Service runner script (start/run/stop/restart/version).
pdp/pom.xml	New Maven module definition + dependencies for PDP server.
pdp/conf.dist/ranger-pdp-site.xml	Config template for overriding defaults.
pdp/conf.dist/logback.xml	Default logging config template for PDP distribution.
pdp/conf.dist/README-k8s.md	Kubernetes guidance for probes/metrics/configuration.
intg/src/main/python/apache_ranger/model/ranger_authz.py	Adds Python authz request/response models for PDP API payloads.
intg/src/main/python/apache_ranger/model/init.py	Re-exports new authz model types.
intg/src/main/python/apache_ranger/exceptions.py	Improves parsing of error responses (supports `message` field).
intg/src/main/python/apache_ranger/client/ranger_pdp_client.py	New Python client for PDP REST endpoints.
intg/src/main/python/apache_ranger/client/init.py	Re-exports `RangerPDPClient` and new model types.
intg/src/main/python/README.md	Adds example usage snippet for the new PDP Python client.
distro/src/main/assembly/pdp.xml	New distro assembly descriptor to package PDP artifacts + deps/config/scripts.
distro/pom.xml	Wires the new `pdp.xml` assembly and adds `ranger-pdp` dependency.
dev-support/ranger-docker/scripts/pdp/ranger-pdp.sh	Docker entrypoint script for starting PDP in container.
dev-support/ranger-docker/scripts/pdp/ranger-pdp-site.xml	Docker PDP configuration (Kerberos-enabled, admin/audit wiring).
dev-support/ranger-docker/scripts/pdp/logback.xml	Docker logback config (file + stdout).
dev-support/ranger-docker/scripts/kdc/entrypoint.sh	Adds PDP principals/keytabs and test user provisioning.
dev-support/ranger-docker/docker-compose.ranger-pdp.yml	New compose service for `ranger-pdp`.
dev-support/ranger-docker/README.md	Documents bringing up PDP alongside other Ranger docker services.
dev-support/ranger-docker/Dockerfile.ranger-pdp	Docker image build for the PDP distribution tarball.
dev-support/ranger-docker/.env	Adds `PDP_VERSION` and `DEBUG_PDP` env vars.
dev-support/ranger-docker/.dockerignore	Ensures PDP distro tarball is included in docker build context.
authz-embedded/src/main/java/org/apache/ranger/authz/embedded/RangerEmbeddedAuthorizer.java	Passes `appType/appId` into plugin construction.
authz-embedded/src/main/java/org/apache/ranger/authz/embedded/RangerAuthzPlugin.java	Updates plugin constructor to accept appId and pass into `RangerPluginConfig`.
authz-api/src/main/java/org/apache/ranger/authz/model/RangerMultiAuthzResult.java	Adds constructors supporting `(requestId, decision[, accesses])`.
authz-api/src/main/java/org/apache/ranger/authz/model/RangerAuthzResult.java	Adds constructors supporting `(requestId, decision[, permissions])`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dev-support/ranger-docker/README.md

pdp/src/test/java/org/apache/ranger/pdp/security/RangerPdpAuthNFilterTest.java

pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java

pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConstants.java

pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConfig.java

dev-support/ranger-docker/README.md

pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java

pdp/src/main/resources/ranger-pdp-default.xml

intg/src/main/python/apache_ranger/client/ranger_pdp_client.py

Copilot

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java

pdp/src/main/java/org/apache/ranger/pdp/security/RangerPdpAuthNFilter.java

pdp/src/main/resources/ranger-pdp-default.xml

pdp/conf.dist/ranger-pdp-site.xml

pdp/src/main/resources/ranger-pdp-default.xml

pdp/src/main/java/org/apache/ranger/pdp/security/JwtAuthHandler.java

pdp/scripts/ranger-pdp-services.sh

dev-support/ranger-docker/Dockerfile.ranger-pdp

pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConfig.java

Copilot

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 8 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pdp/src/main/java/org/apache/ranger/pdp/RangerPdpServer.java

pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java

pdp/src/main/java/org/apache/ranger/pdp/security/RangerPdpAuthNFilter.java

pdp/src/main/java/org/apache/ranger/pdp/security/KerberosAuthHandler.java

pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConfig.java

pdp/conf.dist/logback.xml

intg/src/main/python/apache_ranger/model/ranger_authz.py

intg/src/main/python/apache_ranger/exceptions.py

Copilot

Pull request overview

Copilot reviewed 52 out of 52 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pdp/src/main/java/org/apache/ranger/pdp/security/RangerPdpAuthNFilter.java

pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java

pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConfig.java

intg/src/main/python/apache_ranger/exceptions.py

dev-support/ranger-docker/scripts/pdp/ranger-pdp-site.xml

pdp/src/main/java/org/apache/ranger/pdp/security/KerberosAuthNHandler.java

Copilot

Pull request overview

Copilot reviewed 52 out of 52 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-29T02:55:01Z

pdp/src/main/java/org/apache/ranger/pdp/RangerPdpStatusServlet.java

+        sb.append("# TYPE ranger_pdp_requests_total counter\n");
+        sb.append("ranger_pdp_requests_total ").append(runtimeState.getTotalRequests()).append('\n');


ranger_pdp_requests_total is sourced from runtimeState.getTotalRequests(), but totalRequests is only incremented for 2xx/400/5xx authz outcomes and not for 401/403 auth failures. This makes the metric name misleading (it’s not actually total HTTP requests). Consider incrementing totalRequests in recordAuthFailure() (or in recordRequestMetrics() for 401/403) or renaming the metric to reflect that it excludes auth failures.

Suggested change

sb.append("# TYPE ranger_pdp_requests_total counter\n");

sb.append("ranger_pdp_requests_total ").append(runtimeState.getTotalRequests()).append('\n');

sb.append("# TYPE ranger_pdp_authz_requests_total counter\n");

sb.append("ranger_pdp_authz_requests_total ").append(runtimeState.getTotalRequests()).append('\n');

pdp/src/main/resources/ranger-pdp-default.xml

pdp/conf.dist/ranger-pdp-site.xml

ranger-examples/sample-client/src/main/python/sample_pdp_client.py

pdp/src/test/java/org/apache/ranger/pdp/security/RangerPdpAuthNFilterTest.java

Copilot

Pull request overview

Copilot reviewed 52 out of 52 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pdp/src/main/resources/ranger-pdp-default.xml

pdp/src/main/java/org/apache/ranger/pdp/security/RangerPdpAuthNFilter.java

pdp/src/main/java/org/apache/ranger/pdp/security/KerberosAuthNHandler.java

pdp/conf.dist/logback.xml

pdp/src/main/java/org/apache/ranger/pdp/RangerPdpServer.java

Copilot

Pull request overview

Copilot reviewed 53 out of 53 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-29T18:26:56Z

pdp/src/main/java/org/apache/ranger/pdp/config/RangerPdpConfig.java

+    private void parseHadoopXml(InputStream in, String source) {
+        LOG.info("Loading from {}. Properties count {}", source, props.size());
+
+        XMLUtils.loadConfig(in, props);
+
+        LOG.info("Loaded from {}. Properties count {}", source, props.size());
+    }


parseHadoopXml() logs config-load progress at INFO for every config source, which will add noisy startup/runtime logs (and is inconsistent with other Ranger config loaders that don’t log each load at INFO). Consider downgrading these messages to DEBUG (or logging only on failures) to keep production logs clean.

Copilot · 2026-03-29T18:26:57Z

pdp/src/main/java/org/apache/ranger/pdp/RangerPdpStatusServlet.java

+    private long getPolicyCacheAgeMs() {
+        String cacheDir = config.get(RangerPdpConstants.PROP_AUTHZ_POLICY_CACHE_DIR, null);
+
+        if (StringUtils.isNotBlank(cacheDir)) {
+            File dir = new File(cacheDir);
+
+            if (dir.exists() && dir.isDirectory()) {
+                File[] files = dir.listFiles((d, name) -> name.endsWith(".json"));
+
+                if (files != null && files.length > 0) {
+                    long latestMtime = 0L;
+
+                    for (File f : files) {
+                        latestMtime = Math.max(latestMtime, f.lastModified());
+                    }
+
+                    if (latestMtime > 0L) {
+                        return Math.max(0L, System.currentTimeMillis() - latestMtime);
+                    }
+                }
+            }
+        }
+
+        return -1;


getPolicyCacheAgeMs() scans the policy-cache directory and stats every *.json file on each /health/ready and /metrics request. With frequent Prometheus scraping (and/or large cache dirs) this can become unnecessary filesystem overhead. Consider caching the computed value for a short TTL, tracking latest mtime incrementally, or moving this expensive check behind a lower-frequency refresh.

Copilot · 2026-03-29T18:26:57Z

pdp/src/main/java/org/apache/ranger/pdp/security/KerberosAuthNHandler.java

+        String nameRules     = config.getProperty(RangerPdpConstants.PROP_AUTHN_KERBEROS_NAME_RULES, "DEFAULT");
+        String tokenValidity = config.getProperty(RangerPdpConstants.PROP_AUTHN_KERBEROS_KRB_TOKEN_VALIDITY);
+
+        int tokenLifetime = StringUtils.isBlank(tokenValidity) ? GSSCredential.INDEFINITE_LIFETIME : Integer.parseInt(tokenValidity);
+


token.valid.seconds is parsed with Integer.parseInt() without validation. A malformed value will throw NumberFormatException during filter init and effectively disable Kerberos auth (and can prevent the server from starting if no other handlers are enabled). Consider using a safe parse with a default/fallback (and an explicit log message) similar to RangerPdpConfig#getInt().

Copilot · 2026-03-29T18:26:57Z

intg/src/main/python/apache_ranger/exceptions.py

+                    self.msgDesc     = response.content
+                    self.messageList = [ response.content ]


In the JSON parse failure path, msgDesc/messageList are populated from response.content, which is bytes in requests. This produces exception messages like b'...' and can leak raw bytes; it’s better to use response.text (or decode response.content with a safe charset/replace strategy).

Suggested change

self.msgDesc = response.content

self.messageList = [ response.content ]

self.msgDesc = response.text

self.messageList = [ response.text ]

Copilot · 2026-03-29T18:26:57Z

pdp/src/main/java/org/apache/ranger/pdp/rest/RangerPdpREST.java

+            ret = validateCaller(caller, user, access, serviceName);
+
+            if (RESPONSE_OK.equals(ret)) {
+                try {


Using a concrete Response instance (RESPONSE_OK) as a sentinel and checking RESPONSE_OK.equals(ret) is brittle because Response equality is effectively object identity here. If validateCaller() is ever refactored to return a newly-built 200 response, this check will silently fail. Consider returning a boolean/enum from validateCaller() (or checking ret.getStatus()), and building the actual response in the caller.

mneethiraj · 2026-03-30T02:50:39Z

Will create a new PR for this feature.

mneethiraj added 2 commits March 27, 2026 19:00

RANGER-5371: pdp server implementation

bcd5e5e

RANGER-5371: fix unit test failure

90176fb

mneethiraj requested review from Copilot, dhavalshah9131, kumaab, mehulbparikh, pradeepagrawal8184, rameeshm, sneethiraj, spolavarpau1 and vperiasamy March 28, 2026 02:27

Copilot started reviewing on behalf of mneethiraj March 28, 2026 02:28 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

mneethiraj added 2 commits March 27, 2026 19:37

RANGER-5371: fix checkstyle violation

24234ec

RANGER-5371: addressed review suggestions

a37f433

mneethiraj requested a review from Copilot March 28, 2026 09:25

Copilot started reviewing on behalf of mneethiraj March 28, 2026 09:26 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

RANGER-5371: addressed review suggestions

658e90f

mneethiraj requested a review from Copilot March 28, 2026 10:33

Copilot started reviewing on behalf of mneethiraj March 28, 2026 10:33 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

RANGER-5371: addressed review suggestions, fixed kerberos authentication

9146fda

mneethiraj requested a review from Copilot March 29, 2026 00:22

Copilot started reviewing on behalf of mneethiraj March 29, 2026 00:22 View session

Copilot AI reviewed Mar 29, 2026

View reviewed changes

RANGER-5371: addressed review comments

2cba769

mneethiraj requested a review from Copilot March 29, 2026 02:24

Copilot started reviewing on behalf of mneethiraj March 29, 2026 02:50 View session

Copilot AI reviewed Mar 29, 2026

View reviewed changes

RANGER-5371: fix stats handling of authN failures

ffe01ea

mneethiraj requested a review from Copilot March 29, 2026 07:18

Copilot started reviewing on behalf of mneethiraj March 29, 2026 07:18 View session

Copilot AI reviewed Mar 29, 2026

View reviewed changes

RANGER-5371: addressed review suggestions

4421984

mneethiraj requested a review from Copilot March 29, 2026 18:21

Copilot started reviewing on behalf of mneethiraj March 29, 2026 18:22 View session

Copilot AI reviewed Mar 29, 2026

View reviewed changes

RANGER-5371: addressed review comments

bf4f4ae

mneethiraj closed this Mar 30, 2026

mneethiraj deleted the RANGER-5371 branch March 30, 2026 02:51

		sb.append("# TYPE ranger_pdp_requests_total counter\n");
		sb.append("ranger_pdp_requests_total ").append(runtimeState.getTotalRequests()).append('\n');

		self.msgDesc = response.content
		self.messageList = [ response.content ]

Conversation

mneethiraj commented Mar 28, 2026

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!