Skip to content

[Cosmos] split proof queries async client - #22261

Merged
simorenoh merged 83 commits into
Azure:mainfrom
simorenoh:split-proof-async
Feb 7, 2022
Merged

[Cosmos] split proof queries async client#22261
simorenoh merged 83 commits into
Azure:mainfrom
simorenoh:split-proof-async

Conversation

@simorenoh

@simorenoh simorenoh commented Dec 24, 2021

Copy link
Copy Markdown
Member

This is the PR to address the current gap in the cosmos sdk not being split proof

The gone_retry_policy is used exclusively to refresh the partition key range cache and raise the exception immediately in order to repair the document producers and not waste time retrying several times.

One note on these changes is that I still need to remove certain print statements and comments but want to get input first so I can keep logging the right things in case some further iteration is needed.

simorenoh and others added 30 commits August 13, 2021 13:14
* Removed some stuff

* Looking at constructors

* Updated request

* Added client close

* working client creation

Co-authored-by: simorenoh <simonmorenohe@gmail.com>
database read works, but ignored exception is returned:
Fatal error on SSL transport
NoneType has no attribute 'send' (_loop._proactor.send)
RuntimeError: Event loop is closed
Unclosed connector/ connection
Added methods needed to use async with when initializing client, but logs output "Exception ignored... Runtime Error: Event loop is closed"
missing upsert and other resources
missing read_all_items and both query methods for container class
Complete item CRUD functionality - only missing queries
Query functionality and container query methods
also fixed README and added examples_async
@simorenoh
simorenoh requested a review from annatisch January 24, 2022 22:23
@simorenoh
simorenoh marked this pull request as draft January 24, 2022 22:29
Comment thread sdk/cosmos/azure-cosmos/azure/cosmos/_execution_context/aio/document_producer.py Outdated
Comment thread sdk/cosmos/azure-cosmos/azure/cosmos/_execution_context/aio/document_producer.py Outdated
Comment thread sdk/cosmos/azure-cosmos/azure/cosmos/aio/_retry_utility_async.py
Comment thread sdk/cosmos/azure-cosmos/azure/cosmos/aio/_retry_utility_async.py
@simorenoh
simorenoh requested a review from annatisch January 25, 2022 17:10
@azure-sdk

Copy link
Copy Markdown
Collaborator

API changes have been detected in azure-cosmos. You can review API changes here

Comment thread sdk/cosmos/azure-cosmos/azure/cosmos/exceptions.py Outdated
@simorenoh
simorenoh requested a review from annatisch January 25, 2022 22:59
@azure-sdk

Copy link
Copy Markdown
Collaborator

API changes have been detected in azure-cosmos. You can review API changes here

@simorenoh simorenoh added Client This issue points to a problem in the data-plane of the library. and removed Do Not Merge labels Jan 28, 2022
@simorenoh
simorenoh marked this pull request as ready for review February 2, 2022 18:15
Comment thread sdk/cosmos/azure-cosmos/CHANGELOG.md Outdated
@simorenoh
simorenoh requested review from kushagraThapar and removed request for moderakh February 2, 2022 21:21
@azure-sdk

Copy link
Copy Markdown
Collaborator

API changes have been detected in azure-cosmos. You can review API changes here

@simorenoh
simorenoh merged commit b7f6afc into Azure:main Feb 7, 2022
@simorenoh
simorenoh deleted the split-proof-async branch February 7, 2022 17:00
rakshith91 pushed a commit to rakshith91/azure-sdk-for-python that referenced this pull request Apr 10, 2022
* initial commit

* Client Constructor (Azure#20310)

* Removed some stuff

* Looking at constructors

* Updated request

* Added client close

* working client creation

Co-authored-by: simorenoh <simonmorenohe@gmail.com>

* read database

database read works, but ignored exception is returned:
Fatal error on SSL transport
NoneType has no attribute 'send' (_loop._proactor.send)
RuntimeError: Event loop is closed
Unclosed connector/ connection

* Update simon_testfile.py

* with coroutine

Added methods needed to use async with when initializing client, but logs output "Exception ignored... Runtime Error: Event loop is closed"

* Update simon_testfile.py

* small changes

* async with returns no exceptions

* async read container

* async item read

* cleaning up

* create item/ database methods

* item delete working

* docs replace functionality

missing upsert and other resources

* upsert functionality

missing read_all_items and both query methods for container class

* missing query methods

* CRUD for udf, sproc, triggers

* initial query logic + container methods

* missing some execution logic and tests

* oops

* fully working queries

* small fix to query_items()

also fixed README and added examples_async

* Update _cosmos_client_connection_async.py

* Update _cosmos_client_connection.py

* documentation update

* updated MIT dates and get_user_client() description

* Update CHANGELOG.md

* Delete simon_testfile.py

* leftover retry utility

* Update README.md

* docs and removed six package

* changes based on comments

still missing discussion resolution on SSL verification and tests for async functionality under test module (apart from samples which are basically end to end tests)

* small change in type hints

* updated readme

* fixes based on conversations

* added missing type comments

* update changelog for ci pipeline

* added typehints, moved params into keywords, added decorators, made _connection_policy private

* changes based on sync with central sdk

* remove is_system_key from scripts (only used in execute_sproc)

is_system_key verifies that an empty partition key is properly dealt with if ['partitionKey']['systemKey'] exists in the container options - however, we do not allow containers to be created with empty partition key values in the python sdk, so the functionality is needless

* Revert "remove is_system_key from scripts (only used in execute_sproc)"

Reverting last commit, will find way to init is_system_key for now

* async script proxy using composition

* pylint

* capitalized constants

* Apply suggestions from code review

Clarifying comments for README

Co-authored-by: Gahl Levy <75269480+gahl-levy@users.noreply.github.com>

* closing python code snippet

* last doc updates

* Update sdk/cosmos/azure-cosmos/CHANGELOG.md

Co-authored-by: Simon Moreno <30335873+simorenoh@users.noreply.github.com>

* version update

* cosmos updates for release

* working split proof for async client, need to remove prints() and make better comments

* remove print statements and improve comments

* Update CHANGELOG.md

* pylint

* address Annie's comments on sync split proof

* parity with sync client

* async comparing of document producers

* removed unneeded logic/imports, made compares async for pylint attempt

* spelling mistake and making private

* making sync client private too

* Update CHANGELOG.md

* Delete test_axq.py

Co-authored-by: annatisch <antisch@microsoft.com>
Co-authored-by: Gahl Levy <75269480+gahl-levy@users.noreply.github.com>
Co-authored-by: Travis Prescott <tjprescott@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Client This issue points to a problem in the data-plane of the library. Cosmos

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants