doc: druid 37 release notes by 317brian · Pull Request #19287 · apache/druid

317brian · 2026-04-09T19:13:34Z

Fixes #XXXX.

Description

Fixed the bug ...

Renamed the class ...

Added a forbidden-apis entry ...

Release note

Key changed/added classes in this PR

MyFoo
OurBar
TheirBaz

This PR has:

techdocsmith

This is looking good. all my suggestions are stylistic

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

;

cecemei

request some change regarding incremental metadata cache and supervisor based compaction, we made several changes so some notes are outdated in the same release.

cecemei · 2026-04-20T21:36:10Z

+
+#### Segment locking
+
+Segment locking and `NumberedOverwriteShardSpec` are deprecated and will be removed in a future release. Use time chunk locking instead. You can make sure only time chunk locking is used by setting `druid.indexer.tasklock.forceTimeChunkLock` to `true`.


maybe worth mentioning druid.indexer.tasklock.forceTimeChunkLock is true by default, i guess we should have deprecate segment locking long long time ago, cc: @clintropolis .

cecemei

overall lgtm, left a few minor comments

cecemei · 2026-04-21T00:20:08Z

+#### Changed storage column displays
+
+- Improved the compaction config view to 
+- Renamed **Current size** to **Assigned size**.
+- Renamed **Max size** to **Effective size**. It now displays the smaller value between `max_size` and `storage_size`. The max size is still shown as a tooltip.
+- Changed usage calculation to use `effective_size`
+
+[#19007](https://github.com/apache/druid/pull/19007)
+


this part doesnt read smooth to me, this is part of showing storage metric for data nodes i believe, this change is also related with vsf, do you mind take a look at this, @clintropolis ?

yea, this is related to vsf mode where the actual size disk size is separate from the amount of data the node is responsible for, so we updated this UI to be able to show the ratio of assigned segment size to actual disk size. No real change other than labels if not vsf mode

clintropolis · 2026-04-21T00:46:44Z

+
+### Query blocklist
+
+You can now use the using the `/druid/coordinator/v1/config/broker` API to create a query blocklist to dynamically block queries by datasource, query type, or query context. The blocklist takes effect without a restarting Druid. Block rules use `AND` logic, which means all criteria must match.


You can now use the using the ...

this seems to be missing something

clintropolis · 2026-04-21T00:48:34Z

+
+### Thrift input format
+
+As part of the Thrift contributor extension, Druid now supports Thrift-encoded data for Kafka and Kinesis streaming ingestion.


technically we already supported it before, but only when using the deprecated parser/parsespec stuff that has been removed in this release, so the new thing is that it supports the modern InputFormat stuff

clintropolis · 2026-04-21T00:53:12Z


+- Added the `druid.storage.transfer.asyncHttpClientType` config that specifies which async HTTP client to use for S3 transfers: `crt` for Amazon CRT or `netty` for Netty NIO [#19249](https://github.com/apache/druid/pull/19249)
+- Added a mechanism to automatically clean up intermediary files on HDFS storage [#19187](https://github.com/apache/druid/pull/19187)
+- Changed the default value for `druid.indexing.formats.maxStringLength` to null from 0 [#19198](https://github.com/apache/druid/pull/19198)


this probably doesn't need documented separately from columnFormatSpec on the string column since it is related to that

clintropolis · 2026-04-21T00:54:42Z

+
 ### Ingestion

+- Added the `maxStringLength` configuration for string dimensions that truncates values exceeding the specified length during ingestion. You can set the length globally using `druid.indexing.formats.maxStringLength` or per-dimension in the ingestion spec [#19146](https://github.com/apache/druid/pull/19146)


this was refactored into columnFormatSpec on the string column in #19258, this and the other related bullet should all be combined

clintropolis · 2026-04-21T00:57:05Z

+
+### Incompatible changes
+
+#### Removed `ParseSpec` and deprecated parsers


this should be combined or at least next to the Parsers section since it is related (also related to hadoop removal since hadoop ingestion only supported parser/parseSpec which is why we had to drag them around for so long)

clintropolis · 2026-04-21T00:59:06Z

+
+### Tombstones
+
+Tombstones for JSON-based native batch ingestion (the `dropExisting` flag for `ioConfig`) are now generally available.


this should probably indicate that this was more of an oversight/mistake of not updating docs than really anything changing since these have been used in production for quite a long time, and are not even optional in msq

clintropolis · 2026-04-21T05:49:07Z

+- `bytebuddy` from `1.18.3` to `1.18.5` [#19145](https://github.com/apache/druid/pull/19145)
+- Added `objenesis` `3.5` [#19145](https://github.com/apache/druid/pull/19145)
+- `org.apache.zookeeper` from 3.8.4 to 3.8.6 [#19135](https://github.com/apache/druid/pull/19135)
+- Added AWS SDK `2.40.0` [#18891](https://github.com/apache/druid/pull/18891)


this is a pretty big update to aws sdk libraries going from 1.x to 2.x, maybe deserves more callout

+1 IMHO, we should call this out in upgrade notes.

abhishekrb19 · 2026-04-22T05:03:57Z

+
+[#18966](https://github.com/apache/druid/pull/18966)
+
+#### Other metrics and monitoring improvements


I think this is worth calling out as an experimental perf feature for task init speedup:
#19022

do you mean a different PR? #19022 is not merged

abhishekrb19 · 2026-04-22T05:05:11Z

+
+Added `segment/schemaCache/rowSignature/changed` and `segment/schemaCache/rowSignature/column/count` metrics to expose events when the Broker initializes and updates the row signature in the the segment metadata cache for each datasource.
+
+[#18966](https://github.com/apache/druid/pull/18966)


Bug fix #19162 appears to be missing from the release notes

I can’t recall whether both a “Release note” section in the PR summary and a release note GitHub label are required for changes to appear in the release notes docs

We typically don't include bug fixes in the release notes unless requested. I'm not sure why historically, but that's been the process since I've worked on Druid

abhishekrb19 · 2026-04-22T05:08:29Z

+
 #### Other cluster management improvements

+- Added a `ReadOnly` authorizer that allows all READ operations but denies any other operation, such as WRITE [#19243](https://github.com/apache/druid/pull/19243)


@aho135 @317brian what do you think about adding a brief note linking #19272 for the read only authorizer (as a fix to an issue), since the fix isn’t included in 37.0.0?

I probably should have included it in rc2, maybe we could add this:

This is the first read-only authorizer in Druid, it enforces a global restriction by denying all non-READ operations, but also means API-level permission misconfigurations cannot be corrected via access configuration alone (e.g., GET /druid/indexer/v1/supervisor endpoint incorrectly requiring WRITE access, which is fixed in PR #19272 but not included in this release).

Co-authored-by: Cece Mei <yingqian.mei@gmail.com>

Co-authored-by: Abhishek Radhakrishnan <abhishek.rb19@gmail.com>

cecemei · 2026-04-23T23:49:54Z

+
+#### Filtering metrics
+
+Operators can set `druid.emitter.logging.shouldFilterMetrics=true` to limit which metrics the logging emitter writes. Optionally, they can set `druid.emitter.logging.allowedMetricsPath` to a JSON object file where the keys are metric names. A missing custom file results in a warning and use of the bundled `defaultMetrics.json`. Alerts and other non-metric events are always logged.


defaultMetrics.json has been renamed to loggingEmitterAllowedMetrics.json in #19359. Backport PR: #19371

cecemei

minor comment

cecemei · 2026-05-07T21:47:39Z

+- Added compaction mode to the `compact/task/count` metric [#19151](https://github.com/apache/druid/pull/19151)
+- Added support for logging and emitting SQL dynamic parameter values [#19067](https://github.com/apache/druid/pull/19067)
+- Added `ingest/rows/published`, which all task types emit to denote the total row count of successfully published segments [#19177](https://github.com/apache/druid/pull/19177)
+- Added `queries` and `totalQueries` counters, which reflect queries made to realtime servers to retrieve realtime data


this is from #19196?

cecemei · 2026-05-07T22:08:18Z

+
 #### Other cluster management improvements

+- Added a `ReadOnly` authorizer that allows all READ operations but denies any other operation, such as WRITE [#19243](https://github.com/apache/druid/pull/19243)


I probably should have included it in rc2, maybe we could add this:

This is the first read-only authorizer in Druid, it enforces a global restriction by denying all non-READ operations, but also means API-level permission misconfigurations cannot be corrected via access configuration alone (e.g., GET /druid/indexer/v1/supervisor endpoint incorrectly requiring WRITE access, which is fixed in PR #19272 but not included in this release).

cecemei · 2026-05-07T22:14:06Z

+
+[#18966](https://github.com/apache/druid/pull/18966)
+
+#### Other metrics and monitoring improvements


do you mean a different PR? #19022 is not merged

docs: druid 37 release notes

8559deb

github-actions Bot added the Area - Documentation label Apr 9, 2026

317brian added 3 commits April 9, 2026 12:14

upgrade notes

0a084f0

docs: 37.0.0 release notes

2c20784

fix spelling

3378d78

317brian changed the title ~~docs: druid 37 release notes~~ doc: druid 37 release notes Apr 10, 2026

317brian added this to the 37.0.0 milestone Apr 10, 2026

techdocsmith requested changes Apr 10, 2026

View reviewed changes

317brian and others added 7 commits April 13, 2026 15:16

add suggested revisions

87ff8d5

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

update 19071

dd02c60

add cascading reindexing

4cffd61

fix typos

722d4d2

;

Merge branch '37.0.0' into 37-rn

04c3338

add 19258

78ef933

Merge branch '37.0.0' into 37-rn

76a20e3

cecemei requested changes Apr 20, 2026

View reviewed changes

317brian commented Apr 20, 2026

View reviewed changes

Comment thread docs/release-info/release-notes.md Outdated

317brian and others added 4 commits April 20, 2026 15:19

Apply suggestion from @317brian

ad3f309

fix 19146

239fcf1

update 19050

c691d1b

update metadata storage upgrade note

69555c5

cecemei approved these changes Apr 21, 2026

View reviewed changes

clintropolis reviewed Apr 21, 2026

View reviewed changes

address comments

22f07ec

abhishekrb19 reviewed Apr 22, 2026

View reviewed changes

317brian and others added 4 commits April 22, 2026 14:08

add 18983

3958f33

Apply suggestion from @cecemei

8a0c99e

Co-authored-by: Cece Mei <yingqian.mei@gmail.com>

cleanup 19146

65fc1e1

Apply suggestion from @abhishekrb19

68d05a6

Co-authored-by: Abhishek Radhakrishnan <abhishek.rb19@gmail.com>

317brian and others added 3 commits April 23, 2026 09:44

Apply suggestions from code review

155e2e3

Co-authored-by: Abhishek Radhakrishnan <abhishek.rb19@gmail.com>

typo

aa5c682

typo

3fa6979

cecemei reviewed Apr 23, 2026

View reviewed changes

317brian added 2 commits April 28, 2026 11:02

add 19359

5efab65

fix

1e969bd

317brian marked this pull request as ready for review May 6, 2026 19:44

cecemei approved these changes May 7, 2026

View reviewed changes

317brian added 2 commits May 8, 2026 13:41

address comments

a479152

typo

adf6028

cecemei merged commit 7168f3f into apache:37.0.0 May 8, 2026
8 checks passed


		#### Segment locking

		Segment locking and `NumberedOverwriteShardSpec` are deprecated and will be removed in a future release. Use time chunk locking instead. You can make sure only time chunk locking is used by setting `druid.indexer.tasklock.forceTimeChunkLock` to `true`.


		### Query blocklist

		You can now use the using the `/druid/coordinator/v1/config/broker` API to create a query blocklist to dynamically block queries by datasource, query type, or query context. The blocklist takes effect without a restarting Druid. Block rules use `AND` logic, which means all criteria must match.


		### Thrift input format

		As part of the Thrift contributor extension, Druid now supports Thrift-encoded data for Kafka and Kinesis streaming ingestion.


		### Ingestion

		- Added the `maxStringLength` configuration for string dimensions that truncates values exceeding the specified length during ingestion. You can set the length globally using `druid.indexing.formats.maxStringLength` or per-dimension in the ingestion spec [#19146](https://github.com/apache/druid/pull/19146)


		### Incompatible changes

		#### Removed `ParseSpec` and deprecated parsers


		### Tombstones

		Tombstones for JSON-based native batch ingestion (the `dropExisting` flag for `ioConfig`) are now generally available.


		[#18966](https://github.com/apache/druid/pull/18966)

		#### Other metrics and monitoring improvements


		Added `segment/schemaCache/rowSignature/changed` and `segment/schemaCache/rowSignature/column/count` metrics to expose events when the Broker initializes and updates the row signature in the the segment metadata cache for each datasource.

		[#18966](https://github.com/apache/druid/pull/18966)


		#### Other cluster management improvements

		- Added a `ReadOnly` authorizer that allows all READ operations but denies any other operation, such as WRITE [#19243](https://github.com/apache/druid/pull/19243)


		#### Filtering metrics

		Operators can set `druid.emitter.logging.shouldFilterMetrics=true` to limit which metrics the logging emitter writes. Optionally, they can set `druid.emitter.logging.allowedMetricsPath` to a JSON object file where the keys are metric names. A missing custom file results in a warning and use of the bundled `defaultMetrics.json`. Alerts and other non-metric events are always logged.

Conversation

317brian commented Apr 9, 2026

Description

Fixed the bug ...

Renamed the class ...

Added a forbidden-apis entry ...

Release note

Key changed/added classes in this PR

Uh oh!

techdocsmith left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cecemei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cecemei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clintropolis Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cecemei left a comment

Choose a reason for hiding this comment

clintropolis Apr 21, 2026 •

edited

Loading