Skip to content

Add --unique-app-id parameter#197

Closed
philipp94831 wants to merge 14 commits intomasterfrom
feature/application-id-param
Closed

Add --unique-app-id parameter#197
philipp94831 wants to merge 14 commits intomasterfrom
feature/application-id-param

Conversation

@philipp94831
Copy link
Copy Markdown
Member

No description provided.

Comment thread streams-bootstrap/src/main/java/com/bakdata/kafka/KafkaStreamsApplication.java Outdated
Comment thread streams-bootstrap/src/main/java/com/bakdata/kafka/KafkaStreamsApplication.java Outdated
@philipp94831 philipp94831 changed the title Add --application-id parameter Add --unique-app-id parameter Jan 26, 2024
Copy link
Copy Markdown
Contributor

@torbsto torbsto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would slightly prefer a new parameter in streams that lets you set the consumer group instead of the flag, and I really don't see why it would break anything.

streams:
# brokers: "test:9092"
# schemaRegistryUrl: "url:1234"
passApplicationId: true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the default should be false if your goal is that it is non-breaking

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New helm chart should have it as default but old apps can still use new chart. We did a similar thing for optimizeLeaveGroupBehavior

Comment thread streams-bootstrap/build.gradle.kts
return kafkaConfig;
}

private String getApplicationId() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it would clutter the API even more and lead to confusion together with getUniqueAppId(), but I feel like we will need to call it elsewhere

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this method does the validation whenever Kafka properties are created. So getUniqueAppId() should still return the correct result unless a user explicitly overrides the method and it returns null. I don't think this will ever happen

return kafkaConfig;
}

private String getApplicationId() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I think we need a test for this method

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add tests, just wanted to have your thoughts first

@torbsto
Copy link
Copy Markdown
Contributor

torbsto commented Feb 20, 2024

@raminqaf @philipp94831 Like I said above, I'd prefer using a separate argument for the app id and not reusing the autoscaling one. See feature/application-id-param...feature/helm-app-id for how I imagine it could work

@philipp94831
Copy link
Copy Markdown
Member Author

@raminqaf @philipp94831 Like I said above, I'd prefer using a separate argument for the app id and not reusing the autoscaling one. See feature/application-id-param...feature/helm-app-id for how I imagine it could work

I feel like having two parameters is more confusing. Then we should document it well and probably deprecat the old way of configuring it

@torbsto
Copy link
Copy Markdown
Contributor

torbsto commented Feb 22, 2024

I would be fine with deprecating it, but I'm not sure if there might be a situation in which someone wants to use KEDA but not set the consumer group via Helm.
I also added this check so that we fail as early as possible:

{{- if and (.Values.streams.appId) (.Values.autoscaling.consumerGroup) -}}
{{- fail "streams.appId and autoscaling.consumerGroup are mutually exclusive. You can use use streams.appId for apps 2.17.0+. If you don't want to set your app id via Helm or you're using an older version, but still want to use KEDA autoscaling, set autoscaling.consumerGroup" -}}
{{- end -}}

But I agree that we should also document it properly in the values.yaml and README.md

@philipp94831
Copy link
Copy Markdown
Member Author

@raminqaf @philipp94831 Like I said above, I'd prefer using a separate argument for the app id and not reusing the autoscaling one. See feature/application-id-param...feature/helm-app-id for how I imagine it could work

There is also this annotation we should not forget about https://github.com/bakdata/streams-bootstrap/blob/master/charts/streams-app/templates/deployment.yaml#L20

@philipp94831
Copy link
Copy Markdown
Member Author

Closing in favor of #243

@philipp94831 philipp94831 deleted the feature/application-id-param branch January 28, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants