Update LargeFileUploadTask.java#1531
Conversation
Suggested modification to the chunkInputStream method: From the second pass through this method, the code in the line "int lengthAssert = stream.read(buffer, begin, length);" throws the exception java.lang.IndexOutOfBoundsException when trying to assign the read bytes to a non-existent position in the buffer. The suggested change implies changing InputStream to FileInputStream throughout the LargeFileUploadTask class.
|
@microsoft-github-policy-service agree |
According to the documentation: public int read(byte[] b, int off, int len) throws IOException The first byte read is stored into element b[off], the next one into b[off+1], and so on. However, from the second pass through the chunkInputStream method, off is greater than any position of b and therefore, b[off] does not exist.
|
Since there seems to be a follow-up issue, I think we can do without the version bump to prevent releasing something broken? |
|
@Ndiritu we've had multiple complaints about this task, and attempted multiple fixes. (we still have to review this one that attempts fixing a regression microsoft/kiota-java#1109) I think we're probably missing parts of the equation here. I you could take time to run integration tests for that task (without this PR) against exchange and ODSP (I seem to remember the behaviour was slightly different between the two services), I think it'd go a long way to:
A couple of important things to consider during this integration test:
|
|
This is the reason why I suggested integration testing against both services. I seemed like we still have issues with ODSP, but maybe I misread that comment? |
|
also, on a somewhat related matter, we should plan a path of graph core and update it all the way (with kiota dependencies) in the service libraries. To ensure nobody is on outdated dependencies moving forward and reporting things we've already fixed. As you mentioned, it's probably part of the confusion around that task (and batching) at the moment. |
No we're on the same page now. |
|
I have a question about the upload task. Are all the chunk lengths guaranteed to exactly match what the reader has available for every chunk? The reason I ask is that (according to the docs), the |
There's no guarantee that we'll read the expected chunk length from the stream (from my understanding of how Then set the relevant content-range headers based on this and rely on the API response to determine the next byte position to start from. We currently pre-calculate the ranges before the upload actually begins which might be another bug. |
|
@Ndiritu, thanks for the info. I actually asked because of the assert. I wondered if it would fail for a valid scenario. Maybe the integration tests can help us pick up situations where the read fails but shouldn't. |
Changed chunkInputStream method in LargeFileUploadTask to resolve IndexOutOfBoundsException when uploading large files
|
@ffcdf thank you very much for your contribution! I'll merge this to a separate branch and build on your changes with a few additional fixes. |
|
Deferring to @Ndiritu who's back an in charge of tackling this issue end to end. |
Hi @ihudedi, we have kept this generic as an |
5 participants
Suggested modification to the chunkInputStream method:
From the second pass through this method, the code in the line "int lengthAssert = stream.read(buffer, begin, length);" throws the exception java.lang.IndexOutOfBoundsException when trying to assign the read bytes to a non-existent position in the buffer.
The suggested change implies changing InputStream to FileInputStream throughout the LargeFileUploadTask class.
Fixes # IndexOutOfBoundException in method chunkInputStream
Changes proposed in this pull request
byte[] buffer = new byte[length]; int lengthAssert = stream.read(buffer, begin, length);to
ByteBuffer byteBuffer = ByteBuffer.allocate(length); int lengthAssert = stream.getChannel().read(byteBuffer); byteBuffer.flip(); byte[] buffer = byteBuffer.array(); byteBuffer.clear();Other links