From 1afdb976819bc433c1e6f1fdd60e448e570218cc Mon Sep 17 00:00:00 2001 From: momo-jun Date: Tue, 10 May 2022 17:45:14 +0800 Subject: [PATCH 1/2] add the scope of keys for different clients --- site2/docs/security-encryption.md | 45 +++++++++++++++++-------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/site2/docs/security-encryption.md b/site2/docs/security-encryption.md index 2229e590914b4..f9ed240e2db2b 100644 --- a/site2/docs/security-encryption.md +++ b/site2/docs/security-encryption.md @@ -4,11 +4,11 @@ title: Pulsar Encryption sidebar_label: End-to-End Encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -26,18 +26,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Configure a `CryptoKeyReader` to a producer, consumer or reader. @@ -123,7 +128,7 @@ reader = client.create_reader( client.close() ``` - + ```nodejs const Pulsar = require('pulsar-client'); @@ -244,8 +249,8 @@ Besides, you can use the **default** implementation of `CryptoKeyReader` by spec Currently, **customized** `CryptoKeyReader` implementation is not supported in Python. However, you can use the **default** implementation by specifying the path of `private key` and `public key`. - -Currently, **customized** `CryptoKeyReader` implementation is not supported in Node.JS. However, you can use the **default** implementation by specifying the path of `private key` and `public key`. + +Currently, **customized** `CryptoKeyReader` implementation is not supported in Node.js. However, you can use the **default** implementation by specifying the path of `private key` and `public key`. @@ -259,18 +264,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. From 1ede022353ef16d7d4eab803dd027c38645890f9 Mon Sep 17 00:00:00 2001 From: momo-jun Date: Fri, 13 May 2022 15:02:42 +0800 Subject: [PATCH 2/2] apply changes to more versioned docs --- .../version-2.10.0/security-encryption.md | 45 ++++++++++--------- .../version-2.7.2/security-encryption.md | 39 +++++++++------- .../version-2.7.3/security-encryption.md | 39 +++++++++------- .../version-2.7.4/security-encryption.md | 39 +++++++++------- .../version-2.8.0/security-encryption.md | 39 +++++++++------- .../version-2.8.1/security-encryption.md | 39 +++++++++------- .../version-2.8.3/security-encryption.md | 39 +++++++++------- .../version-2.9.0/security-encryption.md | 39 +++++++++------- .../version-2.9.1/security-encryption.md | 27 ++++++----- .../version-2.9.2/security-encryption.md | 39 +++++++++------- 10 files changed, 217 insertions(+), 167 deletions(-) diff --git a/site2/website/versioned_docs/version-2.10.0/security-encryption.md b/site2/website/versioned_docs/version-2.10.0/security-encryption.md index 84ca0676f3359..aa503685aa977 100644 --- a/site2/website/versioned_docs/version-2.10.0/security-encryption.md +++ b/site2/website/versioned_docs/version-2.10.0/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Configure a `CryptoKeyReader` to a producer, consumer or reader. @@ -124,7 +129,7 @@ reader = client.create_reader( client.close() ``` - + ```nodejs const Pulsar = require('pulsar-client'); @@ -245,8 +250,8 @@ Besides, you can use the **default** implementation of `CryptoKeyReader` by spec Currently, **customized** `CryptoKeyReader` implementation is not supported in Python. However, you can use the **default** implementation by specifying the path of `private key` and `public key`. - -Currently, **customized** `CryptoKeyReader` implementation is not supported in Node.JS. However, you can use the **default** implementation by specifying the path of `private key` and `public key`. + +Currently, **customized** `CryptoKeyReader` implementation is not supported in Node.js. However, you can use the **default** implementation by specifying the path of `private key` and `public key`. @@ -260,18 +265,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.7.2/security-encryption.md b/site2/website/versioned_docs/version-2.7.2/security-encryption.md index 8f1b2934d4ab9..5276bafe494a2 100644 --- a/site2/website/versioned_docs/version-2.7.2/security-encryption.md +++ b/site2/website/versioned_docs/version-2.7.2/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.7.3/security-encryption.md b/site2/website/versioned_docs/version-2.7.3/security-encryption.md index 19e6e04ecf48e..f67d196f185a3 100644 --- a/site2/website/versioned_docs/version-2.7.3/security-encryption.md +++ b/site2/website/versioned_docs/version-2.7.3/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.7.4/security-encryption.md b/site2/website/versioned_docs/version-2.7.4/security-encryption.md index 1ace155117307..11fdc6dc57c9c 100644 --- a/site2/website/versioned_docs/version-2.7.4/security-encryption.md +++ b/site2/website/versioned_docs/version-2.7.4/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.8.0/security-encryption.md b/site2/website/versioned_docs/version-2.8.0/security-encryption.md index 57680dd903c58..b5144c892ac77 100644 --- a/site2/website/versioned_docs/version-2.8.0/security-encryption.md +++ b/site2/website/versioned_docs/version-2.8.0/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.8.1/security-encryption.md b/site2/website/versioned_docs/version-2.8.1/security-encryption.md index 2e7f7ed4b5480..0c9ca66c15a79 100644 --- a/site2/website/versioned_docs/version-2.8.1/security-encryption.md +++ b/site2/website/versioned_docs/version-2.8.1/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.8.3/security-encryption.md b/site2/website/versioned_docs/version-2.8.3/security-encryption.md index 8e647add39a95..048b29cfca847 100644 --- a/site2/website/versioned_docs/version-2.8.3/security-encryption.md +++ b/site2/website/versioned_docs/version-2.8.3/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Configure a `CryptoKeyReader` to a producer, consumer or reader. @@ -263,18 +268,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.9.0/security-encryption.md b/site2/website/versioned_docs/version-2.9.0/security-encryption.md index 2aeb95a1c520d..f10316fa87563 100644 --- a/site2/website/versioned_docs/version-2.9.0/security-encryption.md +++ b/site2/website/versioned_docs/version-2.9.0/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. diff --git a/site2/website/versioned_docs/version-2.9.1/security-encryption.md b/site2/website/versioned_docs/version-2.9.1/security-encryption.md index ce447da12dc36..ef8ed10bf5f3c 100644 --- a/site2/website/versioned_docs/version-2.9.1/security-encryption.md +++ b/site2/website/versioned_docs/version-2.9.1/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,7 +168,7 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); diff --git a/site2/website/versioned_docs/version-2.9.2/security-encryption.md b/site2/website/versioned_docs/version-2.9.2/security-encryption.md index 75c80246bdacf..0b688793c00b4 100644 --- a/site2/website/versioned_docs/version-2.9.2/security-encryption.md +++ b/site2/website/versioned_docs/version-2.9.2/security-encryption.md @@ -5,11 +5,11 @@ sidebar_label: End-to-End Encryption original_id: security-encryption --- -Applications can use Pulsar encryption to encrypt messages at the producer side and decrypt messages at the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. +Applications can use Pulsar encryption to encrypt messages on the producer side and decrypt messages on the consumer side. You can use the public and private key pair that the application configures to perform encryption. Only the consumers with a valid key can decrypt the encrypted messages. ## Asymmetric and symmetric encryption -Pulsar uses dynamically generated symmetric AES key to encrypt messages(data). You can use the application provided ECDSA/RSA key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. +Pulsar uses a dynamically generated symmetric AES key to encrypt messages(data). You can use the application-provided ECDSA (Elliptic Curve Digital Signature Algorithm) or RSA (Rivest–Shamir–Adleman) key pair to encrypt the AES key(data key), so you do not have to share the secret with everyone. Key is a public and private key pair used for encryption or decryption. The producer key is the public key of the key pair, and the consumer key is the private key of the key pair. @@ -27,18 +27,23 @@ Pulsar does not store the encryption key anywhere in the Pulsar service. If you ## Get started -1. Enter the commands below to create your ECDSA or RSA public and private key pair. - -```shell -openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem -openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem -``` +1. Create your ECDSA or RSA public and private key pair by using the following commands. + * ECDSA（for Java clients only) + ```shell + openssl ecparam -name secp521r1 -genkey -param_enc explicit -out test_ecdsa_privkey.pem + openssl ec -in test_ecdsa_privkey.pem -pubout -outform pem -out test_ecdsa_pubkey.pem + ``` + * RSA (for C++, Python and Node.js clients) + ```shell + openssl genrsa -out test_rsa_privkey.pem 2048 + openssl rsa -in test_rsa_privkey.pem -pubout -outform pkcs8 -out test_rsa_pubkey.pem + ``` 2. Add the public and private key to the key management and configure your producers to retrieve public keys and consumers clients to retrieve private keys. -3. Implement the CryptoKeyReader interface, specifically CryptoKeyReader.getPublicKey() for producer and CryptoKeyReader.getPrivateKey() for consumer, which Pulsar client invokes to load the key. +3. Implement the `CryptoKeyReader` interface, specifically `CryptoKeyReader.getPublicKey()` for producer and `CryptoKeyReader.getPrivateKey()` for consumer, which Pulsar client invokes to load the key. -4. Add encryption key name to producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). +4. Add the encryption key name to the producer builder: PulsarClient.newProducer().addEncryptionKey("myapp.key"). 5. Add CryptoKeyReader implementation to producer or consumer builder: PulsarClient.newProducer().cryptoKeyReader(keyReader) / PulsarClient.newConsumer().cryptoKeyReader(keyReader). @@ -163,18 +168,18 @@ If you produce messages that are consumed across application boundaries, you nee When producers want to encrypt the messages with multiple keys, producers add all such keys to the config. Consumer can decrypt the message as long as the consumer has access to at least one of the keys. -If you need to encrypt the messages using 2 keys (myapp.messagekey1 and myapp.messagekey2), refer to the following example. +If you need to encrypt the messages using 2 keys (`myapp.messagekey1` and `myapp.messagekey2`), refer to the following example. ```java PulsarClient.newProducer().addEncryptionKey("myapp.messagekey1").addEncryptionKey("myapp.messagekey2"); ``` ## Decrypt encrypted messages at the consumer application -Consumers require access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. +Consumers require to access one of the private keys to decrypt messages that the producer produces. If you want to receive encrypted messages, create a public or private key and give your public key to the producer application to encrypt messages using your public key. ## Handle failures -* Producer/ Consumer loses access to the key - * Producer action fails indicating the cause of the failure. Application has the option to proceed with sending unencrypted message in such cases. Call PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction) to control the producer behavior. The default behavior is to fail the request. - * If consumption fails due to decryption failure or missing keys in consumer, application has the option to consume the encrypted message or discard it. Call PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction) to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. +* Producer/Consumer loses access to the key + * Producer action fails to indicate the cause of the failure. Application has the option to proceed with sending unencrypted messages in such cases. Call `PulsarClient.newProducer().cryptoFailureAction(ProducerCryptoFailureAction)` to control the producer behavior. The default behavior is to fail the request. + * If consumption fails due to decryption failure or missing keys in consumer, the application has the option to consume the encrypted message or discard it. Call `PulsarClient.newConsumer().cryptoFailureAction(ConsumerCryptoFailureAction)` to control the consumer behavior. The default behavior is to fail the request. Application is never able to decrypt the messages if the private key is permanently lost. * Batch messaging - * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to ConsumerCryptoFailureAction.CONSUME. -* If decryption fails, the message consumption stops and application notices backlog growth in addition to decryption failure messages in the client log. If application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages. + * If decryption fails and the message contains batch messages, client is not able to retrieve individual messages in the batch, hence message consumption fails even if cryptoFailureAction() is set to `ConsumerCryptoFailureAction.CONSUME`. +* If decryption fails, the message consumption stops and the application notices backlog growth in addition to decryption failure messages in the client log. If the application does not have access to the private key to decrypt the message, the only option is to skip or discard backlogged messages.