Skip to content

ADLS Gen 2 Append and Lease Operations#31263

Merged
ibrahimrabab merged 4 commits into
Azure:feature/storage/stg85basefrom
ibrahimrabab:stg85base/appendLease
Oct 18, 2022
Merged

ADLS Gen 2 Append and Lease Operations#31263
ibrahimrabab merged 4 commits into
Azure:feature/storage/stg85basefrom
ibrahimrabab:stg85base/appendLease

Conversation

@ibrahimrabab

@ibrahimrabab ibrahimrabab commented Oct 4, 2022

Copy link
Copy Markdown
Contributor

Description

This PR aims to:

  1. Acquisition & renewal of lease will be available with append API from HDFS endpoint.

  2. Release of lease with flush on append.

  3. Acquire-Release Mode introduced in append for small file write scenarios. Auto-renew, Acquire & Acquire-Release Mode will be introduced for flush so that intermittent/slow flushes succeed.

More information on this feature can be found in the following spec

@ghost ghost added the Storage Storage Service (Queues, Blobs, Files) label Oct 4, 2022
@azure-sdk

azure-sdk commented Oct 4, 2022

Copy link
Copy Markdown
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-storage-file-datalake
azure-storage-file-share

/**
* @return the lease duration in seconds.
*/
public Integer getLeaseDuration() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather see Duration used here to allow this to be more flexible on usage and avoid time as a number ambiguity (is it milliseconds, is it seconds, is it something else)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears we are using int for lease duration in the DataLakeLeaseClient -

. I think we should be internally consistent throughout the SDK.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good with using Integer if we want to maintain internal consistency. In the future if we get feedback about it another overload using Duration could always be added, similar to what was done with RequestRetryOptions (#17628).

/**
* @return the lease duration in seconds.
*/
public Integer getLeaseDuration() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears we are using int for lease duration in the DataLakeLeaseClient -

. I think we should be internally consistent throughout the SDK.

@ibrahimrabab ibrahimrabab merged commit 2622ec9 into Azure:feature/storage/stg85base Oct 18, 2022
@ibrahimrabab ibrahimrabab deleted the stg85base/appendLease branch October 18, 2022 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants