[FEATURE REQ] Content-Length -1 BlockBlobClient.upload

[ryan-murphy-sc]

Is your feature request related to a problem? Please describe.
I am trying to migrate from storage sdk v8 to storage-blob sdk v12, and found that v12 does not allow -1 content-length when uploading block blobs. This is an important feature for our use cases since the size is not always known.

Describe the solution you'd like
If possible, would like the ability to provide -1 as length argument in BlockBlobClient.upload().

Describe alternatives you've considered

• Remaining on v8.
• Writing stream to temporary file before upload.

Additional context

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Description Added
• Expected solution specified

Thanks for consideration.

[rickle-msft]

Hi, @ryan-murphy-sc thank you for reporting this. We do have an option for upload on BlobClient that doesn't accept a length parameter and can handle arbitrary length streams. For now, I hope it is sufficient to be able to select the correct method based on whether or not the length was known.

The idea behind the types in the specialized package (BlockBlobClient included) is that their apis map one to one with the rest api so that it is clear how many io operations are happening. Because of that, it is unlikely we will add this option to BlockBlobClient.upload. However, if you feel the two separate upload methods do not meet your needs, we can continue to look for a solution that works for you and fits with the architecture of this sdk. Please let me know how you would like to proceed.

[ryan-murphy-sc]

Hi @rickle-msft , thanks for the quick response.

I could not find a method matching the one you described on BlobClient. The only upload methods I could find on BlobClient were uploadFromFile. I was looking at azure-storage-blob version 12.1.0 and also checked 12.2.0-beta.1.

[rickle-msft]

@ryan-murphy-sc You're right. My apologies; I was thinking about BlobAsyncClient that has the different upload method which is why you can't find it.

The sync equivalent of this right now is opening a BlobOutputStream and writing to that if the length of the data is unknown. We are close to submitting some performance improvements for that as well in #7067. You should be able to use that for now as it is functional. We realize that this is a bit inconvenient in cases where the length may or may not be known. We have an open issue at #5722 to add an upload method to BlobClient that accepts an arbitrary length stream so that the workflow can be more uniform.

[ryan-murphy-sc]

Thanks @rickle-msft . #5722 certainly sounds like what I'm looking for, so great to see it on the radar.

I'll try the BlobOutputStream solution in the meantime. I took a quick look at BlobOutputStream, and the corresponding BlockBlobOutputStream but it looks like it needs BlockBlobAsyncClient in the constructor rather than BlockBlobClient. I'll give it a shot, and in the worst case I'll use a temp file until #5722 is ready, or maybe even switch to using the async clients altogether.

Shall I close this myself or do you want to mark as duplicate and then close?

Thanks!

[rickle-msft]

A BlockBlobAsyncClient is configured the exact same way as a BlockBlobClient. The only difference is calling buildAsync on your builder instead of build, so hopefully that doesn't present too much of an obstacle.

If you think you're good to go, you can go ahead and close it. If you want to leave it open in case you hit more issues, that's fine, too. Or you can just reopen if something comes up. Happy coding!

[gapra-msft]

Hi @ryan-murphy-sc,

You actually shouldn't have to build a BlockBlobAsyncClient to use the BlobOutputStream.

Using a BlockBlobClient you can just call one of the getOutputStream methods to obtain the associated BlobOutputStream.

Let us know if you have any issues locating the method!

[ryan-murphy-sc]

Thanks for the direction @gapra-msft . I found and tried the getOutputStream method and it is working well for my use case.

I'll close this feature request for now. Thanks for the support.