Skip to content

Gdrive lib update#37265

Merged
phil-davis merged 7 commits into
masterfrom
gdrive_lib_update
May 11, 2020
Merged

Gdrive lib update#37265
phil-davis merged 7 commits into
masterfrom
gdrive_lib_update

Conversation

@jvillafanez

Copy link
Copy Markdown
Member

Description

  • Update google library to version 2.4.1
  • There are some update of the library code for icewind's 3.1.2 (code in the repo seems older than the last composer update. Maybe the code wasn't updated. No real update in the composer's dependency)
  • In addition, google's new version uses guzzle, so try to enforce guzzle version to 5.3.1 (matching core's version). Using guzzle 6 might reduce the amount of dependencies brought by google, but could be problematic for core.

First commit is the library update, both google and icewind.
The second commit contains the actual changes to adapt to the API changes.
The third commit is just readability improvements.

Note that change detection has been changed because the API no longer provides a id per change, so using the google's change API was a problem. It probably isn't suited for the use case implemented in the code.

Related Issue

#36509 (comment)

Motivation and Context

Update google's library version to be compatible with latest php 7.4 deprecations.

How Has This Been Tested?

Tested manually basic file operations through ownCloud.

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Database schema changes (next release will require increase of minor version instead of patch)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation ticket raised:
  • Changelog item, see TEMPLATE

Requires testing for gdrive and SMB connectors due to library updates

@update-docs

update-docs Bot commented Apr 15, 2020

Copy link
Copy Markdown

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

@phil-davis

phil-davis commented Apr 16, 2020

Copy link
Copy Markdown
Contributor

Note: I adjusted PR #37186 the other day to get the latest icewind dependencies, with the actual files committed. Maybe that can be reviewed and merged, and will mean less things to be bundled in this PR.

Also PR #37011 "Guzzle6 for core" - we need to decide if we need/have/want to do that for PHP 7.4 or not, and get it passing. That would be another "independent" change that could happen. - not needed

@codecov

codecov Bot commented Apr 16, 2020

Copy link
Copy Markdown

Codecov Report

Merging #37265 into master will decrease coverage by 0.00%.
The diff coverage is 80.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #37265      +/-   ##
============================================
- Coverage     64.56%   64.56%   -0.01%     
- Complexity    19218    19220       +2     
============================================
  Files          1268     1268              
  Lines         75104    75091      -13     
  Branches       1331     1331              
============================================
- Hits          48493    48484       -9     
+ Misses        26219    26215       -4     
  Partials        392      392              
Flag Coverage Δ Complexity Δ
#javascript 54.14% <ø> (ø) 0.00 <ø> (ø)
#phpunit 65.72% <80.00%> (-0.01%) 19220.00 <0.00> (+2.00) ⬇️
Impacted Files Coverage Δ Complexity Δ
apps/dav/lib/Connector/Sabre/Node.php 80.80% <75.00%> (-0.35%) 51.00 <0.00> (+1.00) ⬇️
apps/dav/lib/Connector/Sabre/File.php 83.97% <100.00%> (+0.05%) 117.00 <0.00> (+1.00)
core/routes.php 48.71% <0.00%> (-1.29%) 0.00% <0.00%> (ø%)
apps/files_sharing/appinfo/routes.php 92.85% <0.00%> (-0.48%) 0.00% <0.00%> (ø%)
lib/private/Setup.php 25.94% <0.00%> (-0.31%) 58.00% <0.00%> (ø%)
core/Controller/AvatarController.php 90.57% <0.00%> (-0.07%) 31.00% <0.00%> (ø%)
lib/private/legacy/util.php 74.44% <0.00%> (-0.05%) 219.00% <0.00%> (ø%)
settings/routes.php 100.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
lib/private/OCS/Provider.php 100.00% <0.00%> (ø) 5.00% <0.00%> (ø%)
apps/files/appinfo/routes.php 100.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2bb3236...4c13db1. Read the comment docs.

@jvillafanez

jvillafanez commented Apr 17, 2020

Copy link
Copy Markdown
Member Author

❗ Please double-check "[gdrive_lib_update bcbe4f7] Fix google docs downloads" is there ❗ (should be fixed now)

@jvillafanez

Copy link
Copy Markdown
Member Author

There is a known issue when a file is re-uploaded (just upload the same file twice or more): the file is duplicated in the gdrive with the same name, which will be problematic (we don't handle multiple files with the same name)

It seems the problem is with the ownCloud's .part file. Disabling the .part files in gdrive in https://github.com/owncloud/core/blob/master/apps/dav/lib/Connector/Sabre/File.php#L637 would solve the problem because the file will be uploaded directly to the previous file, overwriting it.
We need to decide how to proceed with this issue. Adding the gdrive storage there is the simplest solution, but a bad approach.

@mmattel

mmattel commented Apr 17, 2020

Copy link
Copy Markdown
Contributor

I am adding @micbar so he knows about this PR and may add his thoughts

@micbar

micbar commented Apr 17, 2020

Copy link
Copy Markdown
Contributor

@jvillafanez what is the question?

@jvillafanez

Copy link
Copy Markdown
Member Author

The question is if we want to spend time refactoring https://github.com/owncloud/core/blob/master/apps/dav/lib/Connector/Sabre/File.php#L637 in order to provide a good solution, or if we include directly the storage name there to prevent using .part files in google drive.

@jvillafanez
jvillafanez force-pushed the gdrive_lib_update branch 2 times, most recently from 2deb9ff to 3b1ce44 Compare April 24, 2020 07:23
@cdamken

cdamken commented Apr 27, 2020

Copy link
Copy Markdown
Contributor

IMHO .part files should be always in the temp folder and then upload to the external storage once is finished (and even checked by the antivirus if enabled)

@jvillafanez

Copy link
Copy Markdown
Member Author

Decided to add the google storage to the core's ".part file exclusion list" for now in order to progress with the PR.
Opened #37313 to discuss a possible refactor.

@phil-davis

Copy link
Copy Markdown
Contributor

This is also related to #36509 (comment) for PHP 7.4 support.

@phil-davis

Copy link
Copy Markdown
Contributor

I added this to the 10.5.0 project. It needs review and moving forward.

@phil-davis
phil-davis force-pushed the gdrive_lib_update branch from e0be20b to b14e8e4 Compare May 8, 2020 09:05
@phil-davis

phil-davis commented May 8, 2020

Copy link
Copy Markdown
Contributor

I rebased to get this up-to-date with other stuff that has happened in core master in the last week.

@jvillafanez @micbar what happens next?

This needs to move forward - it is becoming a blocker if we need it for 10.5.0

@jvillafanez

Copy link
Copy Markdown
Member Author

Code-wise this is ready. It still need review.
Ignoring the first commit which contains both library update, the rest of the commits should be easy to understand.

@micbar
micbar requested review from VicDeo, karakayasemi and mrow4a May 8, 2020 11:01
@micbar

micbar commented May 8, 2020

Copy link
Copy Markdown
Contributor

@jvillafanez I requested a review. Hope that helps ;-)


private function getDefaultFieldsForFile() {
return 'id,name,mimeType,parents,capabilities/canEdit,size,viewedByMeTime,createdTime,modifiedTime';
static $fileFields = null; // keep static variable to avoid recreating the string

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO static is kinda functional thing. class constant suits better here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what it the best option. The problem with the class constant is that is easy that the field list becomes unreadable if a string is used. If we use an array, we'll have to build the string every time.
Another alternative could be to use a class attribute, but for me it would be polluting the class with information that isn't part of the class. In addition, the attribute could be changed in any part of the class.

I think this is a way the keep balance between readability and performance. The string is built only once when needed, and using an array makes it easier to read. In addition, all this complexity is isolated inside the method, without polluting the class.


private function getDefaultFieldsForFolderScan() {
return 'incompleteSearch,nextPageToken,files(id,name,mimeType,parents,capabilities/canEdit,size,viewedByMeTime,createdTime,modifiedTime)';
static $folderFields = null; // keep static variable to avoid recreating the string

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same as above. could be a class property.

@karakayasemi karakayasemi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewing deeply is not easy. The changeset looks okay, basic file operations and authorization are working.

Comment thread apps/files_external/lib/Lib/Storage/Google.php Outdated
return $this->info->getSize();
$size = $this->info->getSize();
if ($size < 0) {
return null;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this invalidates the current PHPDoc for getSize

@phil-davis

Copy link
Copy Markdown
Contributor

codecov/project fails because of the change in size of all the "crud" here.... - I will override that.

@phil-davis
phil-davis merged commit 143f67f into master May 11, 2020
@delete-merged-branch
delete-merged-branch Bot deleted the gdrive_lib_update branch May 11, 2020 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants