[Doc] Transaction

[Anonymitaet]

Recently, Pulsar 2.8 has been released and one of the highlights is transaction API, which supports atomic writes and acknowledgments across multiple topics. It completes our vision of making Pulsar a complete unified messaging and streaming platform.

The transaction docs need to be improved, so I've re-architectured the doc and drafted one here.

Feel free to comment, thanks.

[Anonymitaet]

@congbobo184 since you are the lead developer of the transaction, could you please help review overall and answer the questions marked in the doc?

It would be much appreciated if you could provide any feedback by EOD 6/23 GMT+8, many thanks!

[newur]

Really good idea to update the docu on this one. I was wondering a lot about how this feature works and missing examples, after reading the release notes and the blog post.

For me the last chapter in the draft, called 'Integration' only made it clear in which case the transactional feature applies. Maybe wanna move that chapter/picture a bit up?

[Anonymitaet]

@newur thanks for your feedback. Does it make sense to move the contents of integration to the what feature do transactions provide section under the overview chapter?

[newur]

@Anonymitaet I would say so, yes. For me transactions are all about the boundaries, so I want to know where does it start and till which point can it rollback. And pictures are pretty useful to illustrate borders. :D

Btw. I would drop statements like "Today’s businesses need to handle ever-increasing amounts of live data." This probably sounds great in marketing, but is fluff for most people who will actually read the documentation. Or actually implement a transactional boundary. ^^ Just my 2 cents of course

[Anonymitaet]

Hi @newur thanks for your suggestions.

After discussing w/ @congbobo184, we've adjusted the doc architecture and want to make the following changes:

1. Move the following contents to the What is txn chapter since they are general explanations and appropriate for stream processing (like Pulsar Flink connector, Pulsar Spark connector, and Function).

2. Add Use Case chapter

(1) Move the following contents to the Pulsar Flink connector doc and add the link here.

(2) Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future. At that time, we'll add txn explanations for them to their corresponding doc and then add links to them here.

Does this make sense to you?

Besides, "Today’s businesses need to handle ever-increasing amounts of live data." demonstrates the fact (processing a lot of streaming data is in great demand), I think it is OK to say that. :-D

[newur]

Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).

Nit: I think writes the resulting messages to another topic is more like writes the result to one or more topics? At least there are multiple topics in the picture, and the sentence about the source is in a similar fashion.

Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.

Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?

[Anonymitaet]

Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).

Thanks for your suggestion. I think the directions of these arrows already point out the start and end (from left to right), correct?

Nit: I think writes the resulting messages to another topic is more like writes the result to one or more topics? At least there are multiple topics in the picture, and the sentence about the source is in a similar fashion.

Thanks for reminding me, I've updated it to writes the result to one or more topics.

Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.
Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?

Hi @congbobo184 for this question, could you please help explain? thanks