From 82c6b169048007162a64aef00b72e27fa5249214 Mon Sep 17 00:00:00 2001 From: Kieran Brantner-Magee Date: Wed, 9 Dec 2020 00:17:39 -0800 Subject: [PATCH 1/3] Adjust README verbiage in preparation for UX pilot study with feedback - improve clarity of "host" vs topic endpoint (which is the verbiage actually seen in the portal) - Add breadcrumbs to generate_sas and sas based auth - change api key verbiage to access key --- sdk/eventgrid/azure-eventgrid/README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/sdk/eventgrid/azure-eventgrid/README.md b/sdk/eventgrid/azure-eventgrid/README.md index 25f18e85d907..dceb812ee5f7 100644 --- a/sdk/eventgrid/azure-eventgrid/README.md +++ b/sdk/eventgrid/azure-eventgrid/README.md @@ -38,11 +38,12 @@ In order to interact with the Event Grid service, you will need to create an ins A **topic_hostname** and **credential** are necessary to instantiate the client object. #### Looking up the endpoint -You can find the endpoint and the hostname on the Azure portal. +You can find the topic endpoint within the Event Grid Topic resource on the Azure portal. +The topic hostname is the URL host component of this endpoint. (Everything up-to and including "eventgrid.azure.net".) #### Create the client with AzureKeyCredential -To use an API key as the `credential` parameter, +To use an Access Key as the `credential` parameter, pass the key as a string into an instance of [AzureKeyCredential][azure-key-credential]. ```python @@ -50,10 +51,12 @@ from azure.core.credentials import AzureKeyCredential from azure.eventgrid import EventGridPublisherClient topic_hostname = "https://..eventgrid.azure.net" -credential = AzureKeyCredential("") +credential = AzureKeyCredential("") eg_publisher_client = EventGridPublisherClient(topic_hostname, credential) ``` +> **Note:** A client may also be authenticated via SAS signature, using the `EventGridSharedAccessSignatureCredential`. A sample demonstrating this, as well as how to generate that signature utilizing `generate_shared_access_signature` is available [here][python-eg-sample-publish-sas-signature]. + ## Key concepts Information about the key concepts on Event Grid, see [Concepts in Azure Event Grid][publisher-service-doc] @@ -250,6 +253,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con [python-eg-sample-consume-customevent]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/eventgrid/azure-eventgrid/samples/champion_scenarios/cs4_consume_custom_events.py [python-eg-sample-send-cloudevent]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/eventgrid/azure-eventgrid/samples/champion_scenarios/cs5_publish_events_using_cloud_events_1.0_schema.py [python-eg-sample-consume-cloudevent]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/eventgrid/azure-eventgrid/samples/champion_scenarios/cs6_consume_events_using_cloud_events_1.0_schema.py +[python-eg-sample-publish-sas-signature]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/eventgrid/azure-eventgrid/samples/publish_samples/publish_with_shared_access_signature_sample.py [publisher-service-doc]: https://docs.microsoft.com/azure/event-grid/concepts [cla]: https://cla.microsoft.com From 54cef2b9bf3e91aed110672d02f8611e6f2df50e Mon Sep 17 00:00:00 2001 From: Kieran Brantner-Magee Date: Thu, 10 Dec 2020 08:46:09 -0800 Subject: [PATCH 2/3] Add UX study feedback in light of pilot study - clarify some readme sections (access keys, endpoint) - make data be populated even if it's not a recognized strong type, was previously none. (this appears to be what JS/dotnet more or less do) - clarify data_version as required in egevent - clarify some docstrings, such as topic being required for domain. - adjust dev reqs so that mgmt endpoints aren't broken by their going to a beta release. --- sdk/eventgrid/azure-eventgrid/README.md | 38 ++++++++++++++++++- .../azure/eventgrid/_models.py | 11 +++--- .../azure-eventgrid/dev_requirements.txt | 2 +- sdk/eventgrid/azure-eventgrid/tests/_mocks.py | 8 ++++ .../azure-eventgrid/tests/test_eg_consumer.py | 8 +++- 5 files changed, 58 insertions(+), 9 deletions(-) diff --git a/sdk/eventgrid/azure-eventgrid/README.md b/sdk/eventgrid/azure-eventgrid/README.md index dceb812ee5f7..d559b6312189 100644 --- a/sdk/eventgrid/azure-eventgrid/README.md +++ b/sdk/eventgrid/azure-eventgrid/README.md @@ -46,6 +46,8 @@ The topic hostname is the URL host component of this endpoint. (Everything up-to To use an Access Key as the `credential` parameter, pass the key as a string into an instance of [AzureKeyCredential][azure-key-credential]. +> **Note:** The Access Key may be found in the azure portal in the "Access Keys" menu of the Event Grid Topic resource. They may also be obtained via the azure CLI, or the `azure-mgmt-eventgrid` library. + ```python from azure.core.credentials import AzureKeyCredential from azure.eventgrid import EventGridPublisherClient @@ -74,6 +76,7 @@ The following sections provide several code snippets covering some of the most c * [Send an Event Grid Event](#send-an-event-grid-event) * [Send a Cloud Event](#send-a-cloud-event) +* [Send to a Domain](#send-to-a-domain) * [Consume an eventgrid Event](#consume-an-event-grid-event) * [Consume a cloud Event](#consume-a-cloud-event) @@ -100,6 +103,7 @@ credential = AzureKeyCredential(key) client = EventGridPublisherClient(topic_hostname, credential) client.send(event) +# Note: Events may also be sent as a list, e.g: client.send([event]) ``` ### Send a Cloud Event @@ -124,6 +128,36 @@ credential = AzureKeyCredential(key) client = EventGridPublisherClient(topic_hostname, credential) client.send(event) +# Note: Events may also be sent as a list, e.g: client.send([event]) +``` + +### Send to a Domain + +This example illustrates sending to a Domain, whereas the earlier samples sent to a Topic. + +The primary difference is that for `EventGridEvents` a topic must be defined in the event being sent, such that it can be routed within the Domain. + +```Python +import os +from azure.core.credentials import AzureKeyCredential +from azure.eventgrid import EventGridPublisherClient, EventGridEvent + +key = os.environ["EG_ACCESS_KEY"] +domain_hostname = os.environ["EG_DOMAIN_HOSTNAME"] + +event = EventGridEvent( + subject="Door1", + data={"team": "azure-sdk"}, + event_type="Azure.Sdk.Demo", + data_version="2.0", + topic="test" +) + +credential = AzureKeyCredential(key) +client = EventGridPublisherClient(domain_hostname, credential) + +client.send(event) +# Note: Events may also be sent as a list, e.g: client.send([event]) ``` ### Consume an Event Grid Event @@ -146,7 +180,7 @@ eg_storage_dict = { "dataVersion":"2.0", "metadataVersion":"1", "eventTime":"2020-08-07T02:28:23.867525Z", - "topic":"/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.EventGrid/topics/eventgridegsub" + "topic":"/subscriptions/fake-subscription-id/resourceGroups/fake-resource-group/providers/Microsoft.EventGrid/topics/eventgridegsub" } deserialized_event = consumer.decode_eventgrid_event(eg_storage_dict) @@ -167,7 +201,7 @@ consumer = EventGridConsumer() cloud_storage_dict = { "id":"a0517898-9fa4-4e70-b4a3-afda1dd68672", - "source":"/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Storage/storageAccounts/{storage-account}", + "source":"/subscriptions/fake-subscription-id/resourceGroups/fake-resource-group/providers/Microsoft.Storage/storageAccounts/fake-storage-account", "data":{ "api":"PutBlockList", }, diff --git a/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py b/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py index 304b4aa19677..811d22fe9fdf 100644 --- a/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py +++ b/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py @@ -22,15 +22,15 @@ class EventMixin(object): def _deserialize_data(event, event_type): """ Sets the data of the desrialized event to strongly typed event object if event type exists in _event_mappings. - Otherwise, sets it to None. + Otherwise, leaves it as-is. :param str event_type: The event_type of the EventGridEvent object or the type of the CloudEvent object. """ # if system event type defined, set event.data to system event object try: event.data = (_event_mappings[event_type]).deserialize(event.data) - except KeyError: # else, if custom event, then event.data is dict and should be set to None - event.data = None + except KeyError: # else, if custom event, then event.data is dict and is left as-is. + pass @staticmethod def _from_json(event, encode): @@ -137,6 +137,7 @@ class EventGridEvent(InternalEventGridEvent, EventMixin): All required parameters must be populated in order to send to Azure. :param topic: The resource path of the event source. If not provided, Event Grid will stamp onto the event. + Is required for Domain topics. :type topic: str :param subject: Required. A resource path relative to the topic path. :type subject: str @@ -144,11 +145,11 @@ class EventGridEvent(InternalEventGridEvent, EventMixin): :type data: object :param event_type: Required. The type of the event that occurred. :type event_type: str + :param data_version: Required. The schema version of the data object. + :type data_version: str :ivar metadata_version: The schema version of the event metadata. If provided, must match Event Grid Schema exactly. If not provided, EventGrid will stamp onto event. :vartype metadata_version: str - :param data_version: The schema version of the data object. If not provided, will be stamped with an empty value. - :type data_version: str :param id: Optional. An identifier for the event. The combination of id and source must be unique for each distinct event. :type id: Optional[str] diff --git a/sdk/eventgrid/azure-eventgrid/dev_requirements.txt b/sdk/eventgrid/azure-eventgrid/dev_requirements.txt index 2d79b6ebd683..87e0a688c693 100644 --- a/sdk/eventgrid/azure-eventgrid/dev_requirements.txt +++ b/sdk/eventgrid/azure-eventgrid/dev_requirements.txt @@ -2,4 +2,4 @@ -e ../../../tools/azure-sdk-tools -e ../../core/azure-core -e ../../identity/azure-identity --e ../azure-mgmt-eventgrid +azure-mgmt-eventgrid==3.0.0rc8 diff --git a/sdk/eventgrid/azure-eventgrid/tests/_mocks.py b/sdk/eventgrid/azure-eventgrid/tests/_mocks.py index 107d6d6175ed..00492ccced98 100644 --- a/sdk/eventgrid/azure-eventgrid/tests/_mocks.py +++ b/sdk/eventgrid/azure-eventgrid/tests/_mocks.py @@ -24,6 +24,14 @@ cloud_storage_string = json.dumps(cloud_storage_dict) cloud_storage_bytes = cloud_storage_string.encode("utf-8") +# A representative sample from an E2E use case of storage queues, used because it once reprod a failure. +cloud_storage_queue_json = json.dumps({"id":"96ff7bc3-ff73-4eb9-aa1b-26f354610fe3", + "source":"datapoint", + "data":{"event":"added","SKU":"ABK8765","status":"success"}, + "type":"test_data", + "time":"2020-12-09T21:14:40.000973Z", + "specversion":"1.0"}) + # custom cloud event cloud_custom_dict = { "id":"de0fd76c-4ef4-4dfb-ab3a-8f24a307e033", diff --git a/sdk/eventgrid/azure-eventgrid/tests/test_eg_consumer.py b/sdk/eventgrid/azure-eventgrid/tests/test_eg_consumer.py index 450910d936a8..f6b06f49e867 100644 --- a/sdk/eventgrid/azure-eventgrid/tests/test_eg_consumer.py +++ b/sdk/eventgrid/azure-eventgrid/tests/test_eg_consumer.py @@ -19,6 +19,7 @@ cloud_storage_dict, cloud_storage_string, cloud_storage_bytes, + cloud_storage_queue_json, cloud_custom_dict, cloud_custom_string, cloud_custom_bytes, @@ -50,7 +51,12 @@ def test_eg_consumer_cloud_storage_bytes(self, **kwargs): deserialized_event = client.decode_cloud_event(cloud_storage_bytes) assert deserialized_event.__class__ == CloudEvent assert deserialized_event.data.__class__ == StorageBlobCreatedEventData - + + def test_eg_consumer_cloud_storage_queue_json(self, **kwargs): + client = EventGridConsumer() + deserialized_event = client.decode_cloud_event(cloud_storage_queue_json) + assert deserialized_event.__class__ == CloudEvent + assert deserialized_event.data def test_eg_consumer_cloud_custom_dict(self, **kwargs): client = EventGridConsumer() From 88dc4cbaed10a642bb99adf8e37669f70aecba7d Mon Sep 17 00:00:00 2001 From: Kieran Brantner-Magee Date: Fri, 11 Dec 2020 09:24:56 -0800 Subject: [PATCH 3/3] Round 2 UX study feedback. clarify some model comments, and add some guiding snippets to readme on core concepts and egevent/cloudevent usage. --- sdk/eventgrid/azure-eventgrid/README.md | 8 ++++++++ .../azure-eventgrid/azure/eventgrid/_models.py | 13 ++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/sdk/eventgrid/azure-eventgrid/README.md b/sdk/eventgrid/azure-eventgrid/README.md index d559b6312189..5d6a1e24cf97 100644 --- a/sdk/eventgrid/azure-eventgrid/README.md +++ b/sdk/eventgrid/azure-eventgrid/README.md @@ -63,6 +63,12 @@ eg_publisher_client = EventGridPublisherClient(topic_hostname, credential) Information about the key concepts on Event Grid, see [Concepts in Azure Event Grid][publisher-service-doc] +### Topic +A channel within the EventGrid service to send events. Must be of CloudEvent or EventGridEvent schema, (decided at creation time) and the corrosponding event type must be used. + +### Domain Topic +A domain exists to rout arbitrary named topics within it. Topics must not exist beforehand if using a domain, topic name can be provided at send time. Simply provide the domain endpoint at client construction instead of topic endpoint. + ### EventGridPublisherClient `EventGridPublisherClient` provides operations to send event data to topic hostname specified during client initialization. Either a list or a single instance of CloudEvent/EventGridEvent/CustomEvent can be sent. @@ -83,6 +89,7 @@ The following sections provide several code snippets covering some of the most c ### Send an Event Grid Event This example publishes an Event Grid event. +> **Note:** It is important to know if your topic supports Cloud or EventGrid events beforehand, otherwise send() will throw an exception. ```Python import os @@ -109,6 +116,7 @@ client.send(event) ### Send a Cloud Event This example publishes a Cloud event. +> **Note:** It is important to know if your topic supports Cloud or EventGrid events beforehand, otherwise send() will throw an exception. ```Python import os diff --git a/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py b/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py index 811d22fe9fdf..17729896024e 100644 --- a/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py +++ b/sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py @@ -49,11 +49,13 @@ def _from_json(event, encode): class CloudEvent(EventMixin): #pylint:disable=too-many-instance-attributes """Properties of an event published to an Event Grid topic using the CloudEvent 1.0 Schema. + Note: CloudEvents can only be sent to a topic of CloudEvent schema. For EventGrid schema, use EventGridEvent. All required parameters must be populated in order to send to Azure. :param source: Required. Identifies the context in which an event happened. The combination of id and source must - be unique for each distinct event. If publishing to a domain topic, source must be the domain name. + be unique for each distinct event, and otherwise can be any user-defined string. + If publishing to a domain topic, source must be the domain name. :type source: str :param data: Event data specific to the event type. :type data: object @@ -131,21 +133,22 @@ def _to_generated(self, **kwargs): class EventGridEvent(InternalEventGridEvent, EventMixin): """Properties of an event published to an Event Grid topic using the EventGrid Schema. + Note: EventGridEvents can only be sent to a topic of EventGridEvent schema. For CloudEvent schema, use CloudEvent. Variables are only populated by the server, and will be ignored when sending a request. All required parameters must be populated in order to send to Azure. :param topic: The resource path of the event source. If not provided, Event Grid will stamp onto the event. - Is required for Domain topics. + Is required for Domain topics, and a domain topic may be specified which doesn't yet exist. :type topic: str - :param subject: Required. A resource path relative to the topic path. + :param subject: Required. A resource path relative to the topic path. Can be any user-defined string. :type subject: str :param data: Event data specific to the event type. :type data: object - :param event_type: Required. The type of the event that occurred. + :param event_type: Required. The type of the event that occurred. Can be any user-defined string. :type event_type: str - :param data_version: Required. The schema version of the data object. + :param data_version: Required. The schema version of the data object. Can be any user-defined string. :type data_version: str :ivar metadata_version: The schema version of the event metadata. If provided, must match Event Grid Schema exactly. If not provided, EventGrid will stamp onto event.