diff --git a/eng/spotbugs-aggregate-report/pom.xml b/eng/spotbugs-aggregate-report/pom.xml index 2b3a3d4b9ef0..6ba14ab80173 100644 --- a/eng/spotbugs-aggregate-report/pom.xml +++ b/eng/spotbugs-aggregate-report/pom.xml @@ -16,8 +16,8 @@ 5.0.1 - 2.3.0 - 2.5.0 + 2.3.1 + 2.5.1 1.2.0 2.0.0 10.5.0 diff --git a/eventhubs/data-plane/ConsumingEvents.md b/eventhubs/data-plane/ConsumingEvents.md index 450247efcb71..1c56521668a9 100644 --- a/eventhubs/data-plane/ConsumingEvents.md +++ b/eventhubs/data-plane/ConsumingEvents.md @@ -1,37 +1,37 @@ -# Consuming Events with the Java client for Azure Event Hubs +# Consuming Events with the Java client for Azure Event Hubs -Consuming events from Event Hubs is different from typical messaging infrastuctures like queues or topic +Consuming events from Event Hubs is different from typical messaging infrastuctures like queues or topic subscriptions where a consumer simply fetches the "next" message. -Event Hubs puts the consumer in control of the offset from which the log shall be read,and the consumer can repeatedly pick a different or the same offset and read the event stream from chosen offsets while the events are being retained. Each partition is therefore loosely analogous to a tape drive that you can wind back to a particular mark and then play back to the freshest data available. +Event Hubs puts the consumer in control of the offset from which the log shall be read,and the consumer can repeatedly pick a different or the same offset and read the event stream from chosen offsets while the events are being retained. Each partition is therefore loosely analogous to a tape drive that you can wind back to a particular mark and then play back to the freshest data available. -Azure Event Hubs consumers need to be aware of the partitioning model chosen for an Event Hub as receivers explicitly -interact with partitions. Any Event Hub's event store is split up into at least 4 partitions, each maintaining a separate event log. You can think of partitions like lanes on a highway. The more events the Event Hub needs to handle, the more lanes (partitions) you have -to add. Each partition can handle at most the equivalent of 1 "throughput unit", equivalent to at most 1000 events per +Azure Event Hubs consumers need to be aware of the partitioning model chosen for an Event Hub as receivers explicitly +interact with partitions. Any Event Hub's event store is split up into at least 4 partitions, each maintaining a separate event log. You can think of partitions like lanes on a highway. The more events the Event Hub needs to handle, the more lanes (partitions) you have +to add. Each partition can handle at most the equivalent of 1 "throughput unit", equivalent to at most 1000 events per second and at most 1 Megabyte per second. -The common consumption model for Event Hubs is that multiple consumers (threads, processes, compute nodes) process events -from a single Event Hub in parallel, and coordinate which consumer is responsible for pulling events from which partition. +The common consumption model for Event Hubs is that multiple consumers (threads, processes, compute nodes) process events +from a single Event Hub in parallel, and coordinate which consumer is responsible for pulling events from which partition. + +> An upcoming update for this client will also bring the popular and powerful "event processor host" from C# to Java. +> The event processor host dramatically simplifies writing high-scale, high-throughput event consumer applications +> that distribute the processing load over a dynamic cluster of machines. -> An upcoming update for this client will also bring the popular and powerful "event processor host" from C# to Java. -> The event processor host dramatically simplifies writing high-scale, high-throughput event consumer applications -> that distribute the processing load over a dynamic cluster of machines. - ## Getting Started This library is available for use in Maven projects from the Maven Central Repository, and can be referenced using the -following dependency declaration inside of your Maven project file: +following dependency declaration inside of your Maven project file: ```XML - com.microsoft.azure + com.microsoft.azure azure-eventhubs - 2.3.0 + 2.3.1 ``` - -For different types of build environments, the latest released JAR files can also be [explicitly obtained from the -Maven Central Repository]() or from [the Release distribution point on GitHub](). + +For different types of build environments, the latest released JAR files can also be [explicitly obtained from the +Maven Central Repository]() or from [the Release distribution point on GitHub](). For a simple event consumer, you'll need to import the *com.microsoft.azure.eventhubs* package for the Event Hub client classes. @@ -39,7 +39,7 @@ For a simple event consumer, you'll need to import the *com.microsoft.azure.even import com.microsoft.azure.eventhubs.*; ``` -Event Hubs client library uses qpid proton reactor framework which exposes AMQP connection and message delivery related +Event Hubs client library uses qpid proton reactor framework which exposes AMQP connection and message delivery related state transitions as reactive events. In the process, the library will need to run many asynchronous tasks while sending and receiving messages to Event Hubs. So, `EventHubClient` requires an instance of `ScheduledExecutorService`, where all these tasks are run. @@ -50,171 +50,171 @@ So, `EventHubClient` requires an instance of `ScheduledExecutorService`, where a ``` The receiver code creates an *EventHubClient* from a given connecting string - + ```Java ConnectionStringBuilder connStr = new ConnectionStringBuilder() .setNamespaceName("----ServiceBusNamespaceName-----") .setEventHubName("----EventHubName-----") .setSasKeyName("-----SharedAccessSignatureKeyName-----") - .setSasKey("---SharedAccessSignatureKey----"); - + .setSasKey("---SharedAccessSignatureKey----"); + EventHubClient ehClient = EventHubClient.createSync(connStr.toString(), executor); -``` +``` + +The receiver code then creates (at least) one *PartitionReceiver* that will receive the data. The receiver is seeded with an offset, in the snippet below it's simply the start of the log. -The receiver code then creates (at least) one *PartitionReceiver* that will receive the data. The receiver is seeded with an offset, in the snippet below it's simply the start of the log. - ```Java String partitionId = "0"; PartitionReceiver receiver = ehClient.createReceiverSync( - EventHubClient.DEFAULT_CONSUMER_GROUP_NAME, - partitionId, + EventHubClient.DEFAULT_CONSUMER_GROUP_NAME, + partitionId, EventPosition.fromStartOfStream()); receiver.setReceiveTimeout(Duration.ofSeconds(20)); -``` +``` -Once the receiver is initialized, getting events is just a matter of calling the *receive()* method in a loop. Each call -to *receive()* will fetch an iterable batch of events to process. - -```Java - Iterable receivedEvents = receiver.receiveSync(maxEventCount); +Once the receiver is initialized, getting events is just a matter of calling the *receive()* method in a loop. Each call +to *receive()* will fetch an iterable batch of events to process. + +```Java + Iterable receivedEvents = receiver.receiveSync(maxEventCount); ``` -## Consumer Groups +## Consumer Groups Event Hub receivers always receive via Consumer Groups. A consumer group is a named entity on an Event Hub that is conceptually similar to a Messaging Topic subscription, even though it provides no server-side filtering capabilities. -Each Event Hub has a "default consumer group" that is created with the Event Hub, which is also the one used in -the samples. +Each Event Hub has a "default consumer group" that is created with the Event Hub, which is also the one used in +the samples. -The primary function of consumers groups is to provide a shared coordination context for multiple concurrent consumers -processing the same event stream in parallel. There can be at most 5 concurrent readers on a partition per consumer group; -it is however *recommended* that there is only one active receiver on a partition per consumer group. The [Ownership, Failover, -and Epochs](#ownership-failover-and-epochs) section below explains how to ensure this. +The primary function of consumers groups is to provide a shared coordination context for multiple concurrent consumers +processing the same event stream in parallel. There can be at most 5 concurrent readers on a partition per consumer group; +it is however *recommended* that there is only one active receiver on a partition per consumer group. The [Ownership, Failover, +and Epochs](#ownership-failover-and-epochs) section below explains how to ensure this. You can create up to 20 such consumer groups on an Event Hub via the Azure portal or the HTTP API. ## Using Offsets -Each Event Hub has a configurable event retention period, which defaults to one day and can be extended to seven days. +Each Event Hub has a configurable event retention period, which defaults to one day and can be extended to seven days. By contacting Microsoft product support you can ask for further extend the retention period to up to 30 days. -There are several options for a consumer to pick at which point into the retained event stream it wants to +There are several options for a consumer to pick at which point into the retained event stream it wants to begin receiving events: -1. **Start of stream** Receive from the start of the retained stream, as shown in the example above. This option will start - with the oldest available retained event in the partition and then continuously deliver events until all available events +1. **Start of stream** Receive from the start of the retained stream, as shown in the example above. This option will start + with the oldest available retained event in the partition and then continuously deliver events until all available events have been read. -2. **Time offset**. This option will start with the oldest event in the partition that has been received into the Event Hub +2. **Time offset**. This option will start with the oldest event in the partition that has been received into the Event Hub after the given instant. - + ``` Java PartitionReceiver receiver = ehClient.createReceiverSync( - EventHubClient.DEFAULT_CONSUMER_GROUP_NAME, - partitionId, + EventHubClient.DEFAULT_CONSUMER_GROUP_NAME, + partitionId, EventPosition.fromEnqueuedTime(instant)); - ``` + ``` -3. **Absolute offset** This option is commonly used to resume receiving events after a previous receiver on the partition +3. **Absolute offset** This option is commonly used to resume receiving events after a previous receiver on the partition has been aborted or suspended for any reason. The offset is a system-supplied string that should not be interpreted by the application. The next section will discuss scenarios for using this option. - + ``` Java PartitionReceiver receiver = ehClient.createReceiverSync( - EventHubClient.DEFAULT_CONSUMER_GROUP_NAME, - partitionId, + EventHubClient.DEFAULT_CONSUMER_GROUP_NAME, + partitionId, EventPosition.fromOffset(savedOffset)); - ``` + ``` 4. **End of stream** While this option is self explanatory, one point to remember here is that, this call is designed to be more performant than using `EventPosition.fromEnqueuedTime(Instant.now())`. -5. **Sequence number** This option is baked into the API to provide better integration with stream processing technologies - (ex: APACHE SPARK). +5. **Sequence number** This option is baked into the API to provide better integration with stream processing technologies + (ex: APACHE SPARK). ## Ownership, Failover, and Epochs -As mentioned in the overview above, the common consumption model for Event Hubs is that multiple consumers process events -from a single Event Hub in parallel. Depending on the amount of processing work required and the data volume that has to be -worked through, and also dependent on how resilient the system needs to be against failures, these consumers may be spread -across multiple different compute nodes (VMs). +As mentioned in the overview above, the common consumption model for Event Hubs is that multiple consumers process events +from a single Event Hub in parallel. Depending on the amount of processing work required and the data volume that has to be +worked through, and also dependent on how resilient the system needs to be against failures, these consumers may be spread +across multiple different compute nodes (VMs). -A simple setup for this is to create a fixed assignment of Event Hub partitions to compute nodes. For instance, you -could have two compute nodes handling events from 8 Event Hub partitions, assigning the first 4 partitions to the -first node and assigning the second set of 4 to the second node. +A simple setup for this is to create a fixed assignment of Event Hub partitions to compute nodes. For instance, you +could have two compute nodes handling events from 8 Event Hub partitions, assigning the first 4 partitions to the +first node and assigning the second set of 4 to the second node. -The downside of such a simple model with fixed assignments is that if one of the compute nodes becomes unavailable, no events -get processed for the partitions owned by that node. +The downside of such a simple model with fixed assignments is that if one of the compute nodes becomes unavailable, no events +get processed for the partitions owned by that node. The alternative is to make ownership dynamic and have all processing nodes reach consensus about who owns which partition, -which is referred to as "[leader election](https://en.wikipedia.org/wiki/Leader_election)" or "consensus" in literature. -One infrastructure for negotiating leaders is [Apache Zookeeper] (https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection), +which is referred to as "[leader election](https://en.wikipedia.org/wiki/Leader_election)" or "consensus" in literature. +One infrastructure for negotiating leaders is [Apache Zookeeper] (https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection), another one one is [leader election over Azure Blobs](https://msdn.microsoft.com/de-de/library/dn568104.aspx). -> The "event processor host" is a forthcoming extension to this Java client that provides an implementation of leader -> election over Azure blobs. The event processor host for Java is very similar to the respective implementation available -> for C# clients. +> The "event processor host" is a forthcoming extension to this Java client that provides an implementation of leader +> election over Azure blobs. The event processor host for Java is very similar to the respective implementation available +> for C# clients. As the number of event processor nodes grows or shrinks, a leader election model will yield a redistribution of partition -ownership. More nodes each own fewer partitions, fewer nodes each own more partitions. Since leader election occurs -external to the Event Hub clients, there's a mechanism needed to allow a new leader for a partition to force the old leader +ownership. More nodes each own fewer partitions, fewer nodes each own more partitions. Since leader election occurs +external to the Event Hub clients, there's a mechanism needed to allow a new leader for a partition to force the old leader to let go of the partition, meaning it must be forced to stop receiving and processing events from the partition. That mechanism is called **epochs**. An epoch is an integer value that acts as a label for the time period during which the -"current" leader for the partition retains its ownership. The epoch value is provided as an argument to the -*EventHubClient::createEpochReciver* method. +"current" leader for the partition retains its ownership. The epoch value is provided as an argument to the +*EventHubClient::createEpochReciver* method. ``` Java epochValue = 1 PartitionReceiver receiver1 = ehClient.createEpochReceiverSync( - EventHubClient.DefaultConsumerGroupName, - partitionId, + EventHubClient.DefaultConsumerGroupName, + partitionId, EventPosition.fromOffset(savedOffset), epochValue); ``` - + When a new partition owner takes over, it creates a receiver for the same partition, but with a greater epoch value. This will instantly cause the previous receiver to be dropped (the service initiates a shutdown of the link) and the new receiver to take over. - + ``` Java /* obtain checkpoint data */ epochValue = 2 PartitionReceiver receiver2 = ehClient.createEpochReceiverSync( - EventHubClient.DefaultConsumerGroupName, - partitionId, + EventHubClient.DefaultConsumerGroupName, + partitionId, EventPosition.fromOffset(savedOffset), epochValue); - ``` - -The new reader obviously also needs to know at which offset processing shall continue. For this, the current owner of a partition should -periodically record its progress on the event stream to a shared location, tracking the offset of the last processed message. This is -called "checkpointing". In case of the aforementioned Azure Blob lease election model, the blob itself is a great place to keep this information. + ``` + +The new reader obviously also needs to know at which offset processing shall continue. For this, the current owner of a partition should +periodically record its progress on the event stream to a shared location, tracking the offset of the last processed message. This is +called "checkpointing". In case of the aforementioned Azure Blob lease election model, the blob itself is a great place to keep this information. -How often an event processor writes checkpoint information depends on the use-case. Frequent checkpointing may cause excessive writes to -the checkpoint state location. Too infrequent checkpointing may cause too many events to be re-processed as the new onwer picks up from -an outdated offset. +How often an event processor writes checkpoint information depends on the use-case. Frequent checkpointing may cause excessive writes to +the checkpoint state location. Too infrequent checkpointing may cause too many events to be re-processed as the new onwer picks up from +an outdated offset. ## AMQP 1.0 -Azure Event Hubs requires using the AMQP 1.0 protocol for consuming events. +Azure Event Hubs requires using the AMQP 1.0 protocol for consuming events. -AMQP 1.0 is a TCP based protocol. For Azure Event Hubs, all traffic *must* be protected using TLS (SSL) and is using -TCP port 5671. For the WebSocket binding of AMQP, traffic flows via port 443. +AMQP 1.0 is a TCP based protocol. For Azure Event Hubs, all traffic *must* be protected using TLS (SSL) and is using +TCP port 5671. For the WebSocket binding of AMQP, traffic flows via port 443. ## Connection Strings Azure Event Hubs and Azure Service Bus share a common format for connection strings. A connection string holds all required -information to set up a connection with an Event Hub. The format is a simple property/value list of the form -{property}={value} with pairs separated by ampersands (&). +information to set up a connection with an Event Hub. The format is a simple property/value list of the form +{property}={value} with pairs separated by ampersands (&). | Property | Description | -|-----------------------|------------------------------------------------------------| +|-----------------------|------------------------------------------------------------| | Endpoint | URI for the Event Hubs namespace. Typically has the form *sb://{namespace}.servicebus.windows.net/* | -| EntityPath | Relative path of the Event Hub in the namespace. Commonly this is just the Event Hub name | +| EntityPath | Relative path of the Event Hub in the namespace. Commonly this is just the Event Hub name | | SharedAccessKeyName | Name of a Shared Access Signature rule configured for the Event Hub or the Event Hub name. For publishers, the rule must include "Send" permissions. | | SharedAccessKey | Base64-encoded value of the Shared Access Key for the rule | | SharedAccessSignature | A previously issued [Shared Access Signature token](https://azure.microsoft.com/en-us/documentation/articles/service-bus-sas-overview/) | - + A connection string will therefore have the following form: ``` @@ -225,19 +225,19 @@ A connection string will therefore have the following form: If you want to read Device to Cloud (D2C) messages sent to **Azure IoT Hub**, [IoT Hub Event Hub-compatible endpoint](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-messages-read-builtin#read-from-the-built-in-endpoint) covers the connection string to be used in this case in detail. -Consumers generally have a different relationship with the Event Hub than publishers. Usually there are relatively few consumers +Consumers generally have a different relationship with the Event Hub than publishers. Usually there are relatively few consumers and those consumers enjoy a high level of trust within the context of a system. The relationship between an event consumer and the Event Hub is commonly also much longer-lived. -It's therefore more common for a consumer to be directly configured with a SAS key rule name and key as part of the +It's therefore more common for a consumer to be directly configured with a SAS key rule name and key as part of the connection string. In order to prevent the SAS key from leaking, it is still advisable to use a long-lived token rather than the naked key. -A generated token will be configured into the connection string with the *SharedAccessSignature* property. - -More information about Shared Access Signature in Service Bus and Event Hubs about about how to generate the required tokens -in a range of languages [can be found on the Azure site.](https://azure.microsoft.com/en-us/documentation/articles/service-bus-sas-overview/) +A generated token will be configured into the connection string with the *SharedAccessSignature* property. + +More information about Shared Access Signature in Service Bus and Event Hubs about about how to generate the required tokens +in a range of languages [can be found on the Azure site.](https://azure.microsoft.com/en-us/documentation/articles/service-bus-sas-overview/) The easiest way to obtain a token for development purposes is to copy the connection string from the Azure portal. These tokens -do include key name and key value outright. The portal does not issue tokens. +do include key name and key value outright. The portal does not issue tokens. diff --git a/eventhubs/data-plane/PublishingEvents.md b/eventhubs/data-plane/PublishingEvents.md index c83007c0174b..4002eff0173a 100644 --- a/eventhubs/data-plane/PublishingEvents.md +++ b/eventhubs/data-plane/PublishingEvents.md @@ -1,33 +1,33 @@ -# Publishing Events with the Java client for Azure Event Hubs +# Publishing Events with the Java client for Azure Event Hubs -The vast majority of Event Hub applications using this and the other client libraries are and will be event publishers. +The vast majority of Event Hub applications using this and the other client libraries are and will be event publishers. And for most of these publishers, publishing events is extremely simple and handled with just a few API gestures. ## Getting Started This library is available for use in Maven projects from the Maven Central Repository, and can be referenced using the -following dependency declaration inside of your Maven project file: +following dependency declaration inside of your Maven project file: ```XML - com.microsoft.azure + com.microsoft.azure azure-eventhubs - 2.3.0 + 2.3.1 ``` - - For different types of build environments, the latest released JAR files can also be [explicitly obtained from the - Maven Central Repository](https://search.maven.org/#search%7Cga%7C1%7Ca%3A%22azure-eventhubs%22) or from [the Release distribution point on GitHub](https://github.com/Azure/azure-event-hubs/releases). + + For different types of build environments, the latest released JAR files can also be [explicitly obtained from the + Maven Central Repository](https://search.maven.org/#search%7Cga%7C1%7Ca%3A%22azure-eventhubs%22) or from [the Release distribution point on GitHub](https://github.com/Azure/azure-event-hubs/releases). + + +For a simple event publisher, you'll need to import the *com.microsoft.azure.eventhubs* package for the Event Hub client classes. -For a simple event publisher, you'll need to import the *com.microsoft.azure.eventhubs* package for the Event Hub client classes. - - ```Java import com.microsoft.azure.eventhubs.*; ``` -Event Hubs client library uses qpid proton reactor framework which exposes AMQP connection and message delivery related +Event Hubs client library uses qpid proton reactor framework which exposes AMQP connection and message delivery related state transitions as reactive events. In the process, the library will need to run many asynchronous tasks while sending and receiving messages to Event Hubs. So, `EventHubClient` requires an instance of `Executor`, where all these tasks are run. @@ -37,63 +37,63 @@ So, `EventHubClient` requires an instance of `Executor`, where all these tasks a ScheduledExecutorService executor = Executors.newScheduledThreadPool(8) ``` -Using an Event Hub connection string, which holds all required connection information including an authorization key or token -(see [Connection Strings](#connection-strings)), you then create an *EventHubClient* instance. - +Using an Event Hub connection string, which holds all required connection information including an authorization key or token +(see [Connection Strings](#connection-strings)), you then create an *EventHubClient* instance. + ```Java ConnectionStringBuilder connStr = new ConnectionStringBuilder() .setNamespaceName("----ServiceBusNamespaceName-----") .setEventHubName("----EventHubName-----") .setSasKeyName("-----SharedAccessSignatureKeyName-----") - .setSasKey("---SharedAccessSignatureKey----"); - + .setSasKey("---SharedAccessSignatureKey----"); + EventHubClient ehClient = EventHubClient.createSync(connStr.toString(), executor); ``` -Once you have the client in hands, you can package any arbitrary payload as a plain array of bytes and send it. The samples -we use to illustrate the functionality send a UTF-8 encoded JSON data, but you can transfer any format you wish. +Once you have the client in hands, you can package any arbitrary payload as a plain array of bytes and send it. The samples +we use to illustrate the functionality send a UTF-8 encoded JSON data, but you can transfer any format you wish. ```Java EventData sendEvent = EventData.create(payloadBytes); ehClient.sendSync(sendEvent); ``` - -The entire client API is built for Java 8's concurrent task model, generally returning + +The entire client API is built for Java 8's concurrent task model, generally returning [*CompletableFuture*](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html), so the library has these methods suffixed with *Sync* as their Synchronous counterparts/varaints. ## AMQP 1.0 Azure Event Hubs allows for publishing events using the HTTPS and AMQP 1.0 protocols. The Azure Event Hub endpoints -also support AMQP over the WebSocket protocol, allowing event traffic to leverage the same outbound TCP port as -HTTPS. +also support AMQP over the WebSocket protocol, allowing event traffic to leverage the same outbound TCP port as +HTTPS. -This client library is built on top of the [Apache Qpid Proton-J]() libraries and supports AMQP, which is significantly -more efficient at publishing event streams than HTTPS. AMQP 1.0 is an international standard published as ISO/IEC 19464:2014. +This client library is built on top of the [Apache Qpid Proton-J]() libraries and supports AMQP, which is significantly +more efficient at publishing event streams than HTTPS. AMQP 1.0 is an international standard published as ISO/IEC 19464:2014. -AMQP is session-oriented and sets up the required addressing information and authorization information just once for each -send link, while HTTPS requires doing so with each sent message. AMQP also has a compact binary format to express common -event properties, while HTTPS requires passing message metadata in a verbose text format. AMQP can also keep a significant -number of events "in flight" with asynchronous and robust acknowledgement flow, while HTTPS enforces a strict request-reply +AMQP is session-oriented and sets up the required addressing information and authorization information just once for each +send link, while HTTPS requires doing so with each sent message. AMQP also has a compact binary format to express common +event properties, while HTTPS requires passing message metadata in a verbose text format. AMQP can also keep a significant +number of events "in flight" with asynchronous and robust acknowledgement flow, while HTTPS enforces a strict request-reply pattern. -AMQP 1.0 is a TCP based protocol. For Azure Event Hubs, all traffic *must* be protected using TLS (SSL) and is using -TCP port 5671. +AMQP 1.0 is a TCP based protocol. For Azure Event Hubs, all traffic *must* be protected using TLS (SSL) and is using +TCP port 5671. This library will provide HTTPS support via WebSockets when Proton-J supports HTTPS. ## Connection Strings Azure Event Hubs and Azure Service Bus share a common format for connection strings. A connection string holds all required -information to set up a connection with an Event Hub. The format is a simple property/value list of the form -{property}={value} with pairs separated by ampersands (&). +information to set up a connection with an Event Hub. The format is a simple property/value list of the form +{property}={value} with pairs separated by ampersands (&). | Property | Description | -|-----------------------|------------------------------------------------------------| +|-----------------------|------------------------------------------------------------| | Endpoint | URI for the Event Hubs namespace. Typically has the form *sb://{namespace}.servicebus.windows.net/* | -| EntityPath | Relative path of the Event Hub in the namespace. Commonly this is just the Event Hub name | +| EntityPath | Relative path of the Event Hub in the namespace. Commonly this is just the Event Hub name | | SharedAccessKeyName | Name of a Shared Access Signature rule configured for the Event Hub or the Event Hub name. For publishers, the rule must include "Send" permissions. | | SharedAccessKey | Base64-encoded value of the Shared Access Key for the rule | | SharedAccessSignature | A previously issued Shared Access Signature token (not yet supported; will be soon) | - + A connection string will therefore have the following form: ``` @@ -102,42 +102,42 @@ A connection string will therefore have the following form: ## Advanced Operations -The publisher example shown in the overview above sends an event into the Event Hub without further qualification. This is -the preferred and most flexible and reliable option. For specific needs, Event Hubs offers two extra options to -qualify send operations: Publisher policies and partion addressing. +The publisher example shown in the overview above sends an event into the Event Hub without further qualification. This is +the preferred and most flexible and reliable option. For specific needs, Event Hubs offers two extra options to +qualify send operations: Publisher policies and partion addressing. ### Partition Addressing -Any Event Hub's event store is split up into at least 4 partitions, each maintaining a separate event log. You can think -of partitions like lanes on a highway. The more events the Event Hub needs to handle, the more lanes (partitions) you have -to add. Each partition can handle at most the equivalent of 1 "throughput unit", equivalent to at most 1000 events per +Any Event Hub's event store is split up into at least 4 partitions, each maintaining a separate event log. You can think +of partitions like lanes on a highway. The more events the Event Hub needs to handle, the more lanes (partitions) you have +to add. Each partition can handle at most the equivalent of 1 "throughput unit", equivalent to at most 1000 events per second and at most 1 Megabyte per second. In some cases, publisher applications need to address partitions directly in order to pre-categorize events for consumption. -A partition is directly addressed either by using the partition's identifier or by using some string (partition key) that gets +A partition is directly addressed either by using the partition's identifier or by using some string (partition key) that gets consistently hashed to a particular partition. -This capability, paired with a large number of partitions, may appear attractive for implementing a fine grained, per publisher +This capability, paired with a large number of partitions, may appear attractive for implementing a fine grained, per publisher subscription scheme similar to what Topics offer in Service Bus Messaging - but it's not at all how the capability should be used -and it's likely not going to yield satisfying results. - -Partition addressing is designed as a routing capability that consistently assigns events from the same sources to the same partition allowing -downstream consumer systems to be optimized, but under the assumption of very many of such sources (hundreds, thousands) share -the same partition. If you need fine-grained content-based routing, Service Bus Topics might be the better option. +and it's likely not going to yield satisfying results. + +Partition addressing is designed as a routing capability that consistently assigns events from the same sources to the same partition allowing +downstream consumer systems to be optimized, but under the assumption of very many of such sources (hundreds, thousands) share +the same partition. If you need fine-grained content-based routing, Service Bus Topics might be the better option. #### Using Partition Keys Of the two addressing options, the preferable one is to let the hash algorithm map the event to the appropriate partition. -The gesture is a straightforward extra override to the send operation supplying the partition key: +The gesture is a straightforward extra override to the send operation supplying the partition key: ```Java EventData sendEvent = EventData.create(payloadBytes); > ehClient.sendSync(sendEvent, partitionKey); ``` - + #### Using Partition Ids -If you indeed need to target a specific partition, for instance because you must use a particular distribution strategy, +If you indeed need to target a specific partition, for instance because you must use a particular distribution strategy, you can send directly to the partition, but doing so requires an extra gesture so that you don't accidentally choose this option. To send to a partition you explicitly need to create a client object that is tied to the partition as shown below: @@ -151,24 +151,24 @@ option. To send to a partition you explicitly need to create a client object tha #### Publisher Policies Event Hub Publisher Policies are not yet supported by this client and will be supported in a future release. - + #### Special considerations for partitions and publisher policies -Using partitions or publisher policies (which are effectively a special kind of partition key) may impact throughput -and availability of your Event Hub solution. +Using partitions or publisher policies (which are effectively a special kind of partition key) may impact throughput +and availability of your Event Hub solution. -When you do a regular send operation that does not prescribe a particular partition, the Event Hub will choose a -partition at random, ensuring about equal distribution of events across partitions. Sticking with the above analogy, -all highway lanes get the same traffic. +When you do a regular send operation that does not prescribe a particular partition, the Event Hub will choose a +partition at random, ensuring about equal distribution of events across partitions. Sticking with the above analogy, +all highway lanes get the same traffic. -If you explicitly choose the partition key or partition-id, it's up to you to take care that traffic is evenly -distributed, otherwise you may end up with a traffic jam (in the form of throttling) on one partition while there's -little or no traffic on another partition. +If you explicitly choose the partition key or partition-id, it's up to you to take care that traffic is evenly +distributed, otherwise you may end up with a traffic jam (in the form of throttling) on one partition while there's +little or no traffic on another partition. -Also, like every other aspect of distributed systems, the log storage backing any partition may rarely and briefly slow +Also, like every other aspect of distributed systems, the log storage backing any partition may rarely and briefly slow down or experience congestion. If you leave choosing the target partition for an event to Event Hubs, it can flexibly -react to such availability blips for publishers. +react to such availability blips for publishers. -Generally, you should *not* use partitioning as a traffic prioritization scheme, and you should *not* use it -for fine grained assignment of particular kinds of events to a particular partitions. *Partitions are a load +Generally, you should *not* use partitioning as a traffic prioritization scheme, and you should *not* use it +for fine grained assignment of particular kinds of events to a particular partitions. *Partitions are a load distribution mechanism, not a filtering model*. diff --git a/eventhubs/data-plane/azure-eventhubs-eph/pom.xml b/eventhubs/data-plane/azure-eventhubs-eph/pom.xml index 649dc2322e69..0a3bd414a75e 100644 --- a/eventhubs/data-plane/azure-eventhubs-eph/pom.xml +++ b/eventhubs/data-plane/azure-eventhubs-eph/pom.xml @@ -7,14 +7,14 @@ com.microsoft.azure azure-eventhubs-clients - 2.3.0 + 2.3.1 ../pom.xml 4.0.0 com.microsoft.azure azure-eventhubs-eph - 2.5.0 + 2.5.1 Microsoft Azure SDK for Event Hubs Event Processor Host(EPH) EPH is built on top of the Azure Event Hubs Client and provides a number of features not present in that lower layer diff --git a/eventhubs/data-plane/azure-eventhubs-extensions/pom.xml b/eventhubs/data-plane/azure-eventhubs-extensions/pom.xml index 45bbd35f6b58..a2a85d70883b 100644 --- a/eventhubs/data-plane/azure-eventhubs-extensions/pom.xml +++ b/eventhubs/data-plane/azure-eventhubs-extensions/pom.xml @@ -7,7 +7,7 @@ com.microsoft.azure azure-eventhubs-clients - 2.3.0 + 2.3.1 ../pom.xml diff --git a/eventhubs/data-plane/azure-eventhubs/pom.xml b/eventhubs/data-plane/azure-eventhubs/pom.xml index c7a1e80f3f5a..6f567e713080 100644 --- a/eventhubs/data-plane/azure-eventhubs/pom.xml +++ b/eventhubs/data-plane/azure-eventhubs/pom.xml @@ -7,7 +7,7 @@ com.microsoft.azure azure-eventhubs-clients - 2.3.0 + 2.3.1 ../pom.xml diff --git a/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/ConnectionStringBuilder.java b/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/ConnectionStringBuilder.java index 380d2ac9cacd..d203026ddec2 100644 --- a/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/ConnectionStringBuilder.java +++ b/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/ConnectionStringBuilder.java @@ -406,7 +406,7 @@ private void parseConnectionString(final String connectionString) { this.transportType = TransportType.fromString(values[valueIndex]); } catch (IllegalArgumentException exception) { throw new IllegalConnectionStringFormatException( - String.format("Invalid value specified for property '%s' in the ConnectionString.", TRANSPORT_TYPE_CONFIG_NAME), + String.format(Locale.US, "Invalid value specified for property '%s' in the ConnectionString.", TRANSPORT_TYPE_CONFIG_NAME), exception); } } else { diff --git a/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/impl/ClientConstants.java b/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/impl/ClientConstants.java index 2fb6bee48ebe..0cd1b70972cf 100644 --- a/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/impl/ClientConstants.java +++ b/eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/impl/ClientConstants.java @@ -37,7 +37,7 @@ public final class ClientConstants { public static final String NO_RETRY = "NoRetry"; public static final String DEFAULT_RETRY = "Default"; public static final String PRODUCT_NAME = "MSJavaClient"; - public static final String CURRENT_JAVACLIENT_VERSION = "2.3.0"; + public static final String CURRENT_JAVACLIENT_VERSION = "2.3.1"; public static final String PLATFORM_INFO = getPlatformInfo(); public static final String FRAMEWORK_INFO = getFrameworkInfo(); public static final String CBS_ADDRESS = "$cbs"; diff --git a/eventhubs/data-plane/pom.xml b/eventhubs/data-plane/pom.xml index 93d1e09126b5..5d96107a7309 100644 --- a/eventhubs/data-plane/pom.xml +++ b/eventhubs/data-plane/pom.xml @@ -15,7 +15,7 @@ com.microsoft.azure azure-eventhubs-clients pom - 2.3.0 + 2.3.1 Microsoft Azure Event Hubs SDK Parent Java libraries for talking to Windows Azure Event Hubs diff --git a/eventhubs/data-plane/readme.md b/eventhubs/data-plane/readme.md index 38634dbb2979..a5e1c46c481e 100644 --- a/eventhubs/data-plane/readme.md +++ b/eventhubs/data-plane/readme.md @@ -21,7 +21,7 @@ Azure Event Hubs is a hyper-scale data ingestion service, fully-managed by Micro Refer to the [online documentation](https://azure.microsoft.com/services/event-hubs/) to learn more about Event Hubs in general and [General Overview document](Overview.md) for an overview of Event Hubs Client for Java. -## Using the library +## Using the library ### Samples @@ -33,8 +33,8 @@ Two java packages are released to Maven Central Repository from this GitHub repo #### Microsoft Azure EventHubs Java Client -This library exposes the send and receive APIs. This library will in turn pull further required dependencies, specifically -the required versions of Apache Qpid Proton-J, and the cryptography library BCPKIX by the Legion of Bouncy Castle. +This library exposes the send and receive APIs. This library will in turn pull further required dependencies, specifically +the required versions of Apache Qpid Proton-J, and the cryptography library BCPKIX by the Legion of Bouncy Castle. |Package|Package Version| |--------|------------------| @@ -42,9 +42,9 @@ the required versions of Apache Qpid Proton-J, and the cryptography library BCPK ```XML - com.microsoft.azure - azure-eventhubs - 2.3.0 + com.microsoft.azure + azure-eventhubs + 2.3.1 ``` @@ -59,26 +59,26 @@ It pulls the required versions of Event Hubs, Azure Storage and GSon libraries. ```XML - com.microsoft.azure - azure-eventhubs-eph - 2.5.0 + com.microsoft.azure + azure-eventhubs-eph + 2.5.1 ``` ## How to provide feedback First, if you experience any issues with the runtime behavior of the Azure Event Hubs service, please consider filing a support request -right away. Your options for [getting support are enumerated here](https://azure.microsoft.com/support/options/). In the Azure portal, -you can file a support request from the "Help and support" menu in the upper right hand corner of the page. +right away. Your options for [getting support are enumerated here](https://azure.microsoft.com/support/options/). In the Azure portal, +you can file a support request from the "Help and support" menu in the upper right hand corner of the page. -If you find issues in this library or have suggestions for improvement of code or documentation, you can [file an issue in the project's -GitHub repository](https://github.com/Azure/azure-event-hubs/issues) or send across a pull request - see our [Contribution Guidelines](./.github/CONTRIBUTING.md). +If you find issues in this library or have suggestions for improvement of code or documentation, you can [file an issue in the project's +GitHub repository](https://github.com/Azure/azure-event-hubs/issues) or send across a pull request - see our [Contribution Guidelines](./.github/CONTRIBUTING.md). Issues related to runtime behavior of the service, such as sporadic exceptions or apparent service-side performance or reliability issues can not be handled here. -Generally, if you want to discuss Azure Event Hubs or this client library with the community and the maintainers, you can turn to -[stackoverflow.com under the #azure-eventhub tag](http://stackoverflow.com/questions/tagged/azure-eventhub) or the -[MSDN Service Bus Forum](https://social.msdn.microsoft.com/Forums/en-US/home?forum=servbus). +Generally, if you want to discuss Azure Event Hubs or this client library with the community and the maintainers, you can turn to +[stackoverflow.com under the #azure-eventhub tag](http://stackoverflow.com/questions/tagged/azure-eventhub) or the +[MSDN Service Bus Forum](https://social.msdn.microsoft.com/Forums/en-US/home?forum=servbus). ## Build & contribute to the library @@ -86,7 +86,7 @@ You will generally not have to build this client library yourself - this library If you have any specific requirement for which you want to contribute or need to generate a SNAPSHOT version, this section is for you. **Your contributions are welcome and encouraged!** -We adopted maven build model and strive to keep the project model intuitive enough to developers. +We adopted maven build model and strive to keep the project model intuitive enough to developers. If you need any help with any specific IDE or cannot get the build going in any environment - please open an issue. Here are few general topics, which we thought developers would need help with: