Skip to content

Ensure backend is verified on sync. #30412

Merged
tomneedham merged 3 commits into
masterfrom
sync-dedup
Mar 2, 2018
Merged

Ensure backend is verified on sync. #30412
tomneedham merged 3 commits into
masterfrom
sync-dedup

Conversation

@tomneedham

Copy link
Copy Markdown
Contributor

Description

When we sync with an existing account entity, we must always check that we are syncing against the same backend as when the account was created.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@tomneedham tomneedham added this to the development milestone Feb 8, 2018
@tomneedham tomneedham self-assigned this Feb 8, 2018
@tomneedham tomneedham requested a review from mrow4a February 8, 2018 07:16
@tomneedham tomneedham force-pushed the sync-dedup branch 3 times, most recently from debd466 to df6dc1e Compare February 8, 2018 07:50
} catch (DoesNotExistException $e) {
// Create a new account for this uid and backend pairing and sync
$account = $this->createNewAccount(get_class($backend), $uid);
} catch (MultipleObjectsReturnedException $e) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generaly this would be a serious bug

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - can happen with mysql though - and I was fed up with phpstorm complaining at me

@mrow4a

mrow4a commented Feb 8, 2018

Copy link
Copy Markdown
Contributor

As discussed, we need to have unit test covering this scenario

@tomneedham

Copy link
Copy Markdown
Contributor Author

Added test and fixed issue in the logging statement.

@codecov

codecov Bot commented Feb 13, 2018

Copy link
Copy Markdown

Codecov Report

Merging #30412 into master will increase coverage by 0.03%.
The diff coverage is 75%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #30412      +/-   ##
============================================
+ Coverage      61.9%   61.93%   +0.03%     
- Complexity    19061    19085      +24     
============================================
  Files          1091     1092       +1     
  Lines         61473    61531      +58     
============================================
+ Hits          38052    38109      +57     
- Misses        23421    23422       +1
Impacted Files Coverage Δ Complexity Δ
lib/private/User/SyncService.php 73.12% <75%> (+2.31%) 57 <0> (+2) ⬆️
core/register_command.php 0% <0%> (ø) 0% <0%> (ø) ⬇️
core/Command/User/Modify.php 89.65% <0%> (ø) 22% <0%> (?)
lib/public/App/ManagerEvent.php 54.54% <0%> (+18.18%) 4% <0%> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 881f595...413a86a. Read the comment docs.

@PVince81

Copy link
Copy Markdown
Contributor

test failures related to federated shares

@PVince81

Copy link
Copy Markdown
Contributor

as discussed use a hack to change the alternate home from the testing app ?

@tomneedham

Copy link
Copy Markdown
Contributor Author

as discussed use a hack to change the alternate home from the testing app ?

Added it. But not sure where the backend values should be set back? Where exactly is the testing app used?

@phil-davis

Copy link
Copy Markdown
Contributor

The testing app is used by the API "integration" tests and the UI tests as an endpoint that can switch on/off capabilities, run an occ command... so that tests setup settings...

Comment thread apps/testing/lib/Application.php Outdated
// Make existing accounts use the new alternative home backed
$q = $c->getServer()->getDatabaseConnection()->getQueryBuilder();
$q->update('*PREFIX*accounts')
->set('backend', AlternativeHomeUserBackend::class)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

->set('backend', $q->expr()->literal(AlternativeHomeUserBackend::class))

@tomneedham

Copy link
Copy Markdown
Contributor Author

Doesn't necessarily need a LDAP app to work. You would utilise the existing AlternativeHomeUserBackend but the testing needs to be more explicit about creating a user for that backend, and using that user - not hot swapping the backend class.

@PVince81

Copy link
Copy Markdown
Contributor

the testing needs to be more explicit about creating a user for that backend

We'd likely need to adjust all tests that create users, or the code that calls the provisioning API to create them. Currently provisioning API doesn't allow for specifying a backend and I believe it's only designed for DB backend.

In any case, it feels like everything related to that is a hack.

Another alternative would be to modify the actual DB user backend to also support specifying a home folder, officially, when the user gets created.

@tomneedham

Copy link
Copy Markdown
Contributor Author

Currently provisioning API doesn't allow for specifying a backend and I believe it's only designed for DB backend.

This is fine - since the testing app removes the backends and adds the AltHomeBackend - which is extended from database and works exactly the same - so no issue here.

The problem is that this home testing code is hot swapping the backend which we no longer tolerate in core. I still dont understand where this testing app is enabled, and when and why, and when disabled? If we understood that, we could simply recreate the users it needs so they have the right backend, and remove them afterwards.

Another alternative would be to modify the actual DB user backend to also support specifying a home folder, officially, when the user gets created.

Couldn't we essentially achieve this by changing the data directory value in config for the usermanager?

@PVince81

Copy link
Copy Markdown
Contributor

Couldn't we essentially achieve this by changing the data directory value in config for the usermanager?

Maybe, but need to first create the users and then swap the data directory. Not sure at what moment exactly during the tests. And when to set it back ? Sounds like a similar problem.

@phil-davis

Copy link
Copy Markdown
Contributor

The testing app is enabled once in tests/integration/run.sh, the whole behat run of test scenarios is done and then the testing app is put back to its previous state on exit from the whole test run.

During each scenario the needed "ordinary" users are created, and an AfterScenario step automagically deletes the users/groups... that were created in the scenario.

But as you have seen, it is assumed that the ownCloud install comes with an existing admin acount "for free" and that is "just used" by the tests for any scenarios that need admin privs.

@PVince81

Copy link
Copy Markdown
Contributor

based on what @phil-davis which I think is correct I don't understand why we're getting the above errors then

@PVince81

Copy link
Copy Markdown
Contributor

Should we then exclude the admin user from the alternate home since it's the only user that is always there ? From what I remember most tests are anyway creating separate users.

@PVince81

Copy link
Copy Markdown
Contributor

pushed... let's find out

@PVince81

Copy link
Copy Markdown
Contributor

how come... 14:49:15 [Wed Feb 28 14:49:15 2018] User <admin> already provided by another backend(OC\User\Database !== OCA\Testing\AlternativeHomeUserBackend), skipping.

@PVince81

Copy link
Copy Markdown
Contributor

I think the problem is that the "admin" account already exists before the testing app is enabled, since we're using a formerly set up system. I'm now struggling to find a way to make the testing app change all backends once at the time the app gets enabled.

@phil-davis

Copy link
Copy Markdown
Contributor

Will life be easier if we create an admin2 account with admin privs at the start of a whole test run (and remove it at the end). Then use admin2 in all the tests.

That way the original admin account will only be used as the authentication for creating admin2 and after that will not be needed/touched.

@PVince81

Copy link
Copy Markdown
Contributor

A fresh new hack has arrived: now at the time where the testing app gets enabled, it will adjust all DB backends to alternate home backend. This makes sure that existing users like the admin gets their backend converted. We only do it once to avoid the "database is locked" error we had before.

  • we could also revert the backend names when disabling the app

@tomneedham please check if this additional fix is ok, I keep seeing this in the logs for empty string: 1d7276a

@PVince81

Copy link
Copy Markdown
Contributor

if my last hack doesn't work we could try #30412 (comment) next @phil-davis

@PVince81

PVince81 commented Mar 1, 2018

Copy link
Copy Markdown
Contributor

ohh look... it worked... ok, will resolve conflict and clean up

Tom Needham and others added 3 commits March 1, 2018 10:10
For example if enabling an app failed, it must fail and bail out
Fixes some database locking errors with federated sharing tests.
@PVince81

PVince81 commented Mar 1, 2018

Copy link
Copy Markdown
Contributor

Squashed and rebased.

@tomneedham please review my fix for the testing app and we can move forward with the review of your part.

@tomneedham

Copy link
Copy Markdown
Contributor Author

fine but fragile - no more than it was before. At some point I suggest we remove this - and get tests to configure their own admin user when necessary.

@tomneedham tomneedham merged commit 8389096 into master Mar 2, 2018
@phil-davis

phil-davis commented Mar 2, 2018

Copy link
Copy Markdown
Contributor

See #30679 for a little tidy up of the messages in run.sh
I will make a combined backport of both in a moment...
Maybe not - there are conflicts in the backport because #29669 has not been backported. So someone else can sort that out - is #29669 to be backported? If so, backport it first.

OK, I backported the 3rd commit here, along with the backport of #30679 - see #30683
The first 2 commits still need to be backported some day.

@phil-davis

Copy link
Copy Markdown
Contributor

Backport stable10 - the 3 commits here are backported in #30638

@lock

lock Bot commented Jul 31, 2019

Copy link
Copy Markdown

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock Bot locked as resolved and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants