Skip to content

Force alphabetical ordering of app installation to fix on osx 10.13#29142

Merged
PVince81 merged 2 commits into
masterfrom
fix-install-app-order
Oct 17, 2017
Merged

Force alphabetical ordering of app installation to fix on osx 10.13#29142
PVince81 merged 2 commits into
masterfrom
fix-install-app-order

Conversation

@tomneedham

Copy link
Copy Markdown
Contributor

Description

Well this was an interesting one to find. PHP readdir returns in order that the filesystem stores a directory - which is not necessarily alphabetical ordering. However, apparently we require this ordering, since apps with names files_ typically rely on files to already be installed.

Since updating to OSX High Sierra 10.13, they now use a new 'optimised filesystem' which appears to affect the listing order. This caused the install to fail because trashbin was installed before files and there was an autoloading exception.

The fix forces the installer to use an alphabetical ordering when listing the contents of app folders.

Motivation and Context

Install of core fails since OSX High Sierra upgrade.

How Has This Been Tested?

UI and OCC installs on OSX High Sierra.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@tomneedham tomneedham added this to the development milestone Oct 4, 2017
@tomneedham tomneedham self-assigned this Oct 4, 2017
@tomneedham

Copy link
Copy Markdown
Contributor Author

Now what did I break find on Jenkins

Comment thread lib/private/Installer.php
while( false !== ( $filename = readdir( $dir ))) {
$nodes = scandir($dir);
foreach($nodes as $filename) {
if( substr( $filename, 0, 1 ) != '.' and is_dir($app_dir['path']."/$filename") ) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also moderately dodgy. There is no check for the parent dir entry .. - I think it will pass this "if" test.
The test on the next line will (thankfully) be false, because some file like apps/../appinfo/info.xml is not going to exist.
But not directly relevant to the reason for this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. is caught by the first condition, since it is checking only the first character of the filename

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes! I skimmed over the detail too quickly. And I guess in this context we do not care if there is a folder ".hidden-stuff" that is skipped over - that should not have an app in it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly

@tomneedham

Copy link
Copy Markdown
Contributor Author

just grepping for other areas this may effect

@tomneedham

Copy link
Copy Markdown
Contributor Author

oc_properties is missing when the core tests are run, because dav app is not installed.

diff --git a/tests/enable_all.php b/tests/enable_all.php
index 11b983eb78..67de8c04ff 100644
--- a/tests/enable_all.php
+++ b/tests/enable_all.php
@@ -16,6 +16,7 @@ function enableApp($app) {
        }
 }

+enableApp('dav');
 enableApp('files_sharing');
 enableApp('files_trashbin');
 enableApp('encryption');

fixes - but why is this now a problem

@DeepDiver1975

Copy link
Copy Markdown
Member

fixes - but why is this now a problem

because the load order of apps has changed? we might simply got lucky in the past because dav sorts up pretty hight .... no idea

@tomneedham

tomneedham commented Oct 4, 2017

Copy link
Copy Markdown
Contributor Author

we might simply got lucky

my brain was avoiding this theory 😒 digging...

@phil-davis

Copy link
Copy Markdown
Contributor

On my Ubuntu 16.04 with PHP 7.0.22

<?php
$dir = opendir('core/apps');
while( false !== ( $filename = readdir( $dir ))) {
  echo $filename . "\n";
}

the output is:

files_trashbin
files
updatenotification
federation
dav
files_sharing
..
files_versions
systemtags
comments
federatedfilesharing
testing
provisioning_api
files_texteditor
files_external
.
firewall
theme-example

so the old way is "random order" on *nix also.

@tomneedham

Copy link
Copy Markdown
Contributor Author

Just be lucky that your files app comes first ;)

@tomneedham tomneedham force-pushed the fix-install-app-order branch from f455b35 to 7f45658 Compare October 9, 2017 12:45
@PVince81

Copy link
Copy Markdown
Contributor

Could this explain the wrong migration orders in some setups or is this unrelated ?

Ref #28285

@tomneedham

Copy link
Copy Markdown
Contributor Author

Could this explain the wrong migration orders in some setups or is this unrelated ?

I thought so - but they are explicitly sorted by their numbers before being applied.

But could still be related - as apps are installing in different orders which could cause weird effects when they are trying to apply schema changes based on what is present

@PVince81

Copy link
Copy Markdown
Contributor

@tomneedham your attempt at making the universe deterministic has failed:

14:49:19 There were 10 errors:
14:49:19 
14:49:19 1) Test\DB\QueryBuilder\QueryBuilderTest::testGetLastInsertId
14:49:19 Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing 'INSERT INTO "oc_properties" ("fileid", "propertyname", "propertyvalue") VALUES('100500', 'testing', 'testing')':
14:49:19 
14:49:19 SQLSTATE[HY000]: General error: 1 no such table: oc_properties
14:49:19 
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractSQLiteDriver.php:58
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:128
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:1015
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/lib/private/DB/Connection.php:211
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php:208
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/lib/private/DB/QueryBuilder/QueryBuilder.php:141
14:49:19 /var/lib/jenkins/workspace/owncloud-core_core_PR-29142-JO7HXKWBQT42X32D2PGSRR4QVWOAPQRCZP75FRKHSWHVTLPQOWIQ/tests/lib/DB/QueryBuilder/QueryBuilderTest.php:1133
14:49:19 
14:49:19 Caused by
14:49:19 Doctrine\DBAL\Driver\PDOException: SQLSTATE[HY000]: General error: 1 no such table: oc_properties
14:49:19 

@tomneedham tomneedham force-pushed the fix-install-app-order branch from 0488a7e to d54b938 Compare October 16, 2017 11:12
@tomneedham

Copy link
Copy Markdown
Contributor Author

Explicitly setting files, dav as the first two apps seems to work - otherwise somehow dav gets installed without the migrations running 😟

@PVince81 PVince81 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍, assuming there is no adjustable unit test due to staticness.

Not happy about the hard-coded bits but there is no better way. At least app loading will be deterministic.

@tomneedham

Copy link
Copy Markdown
Contributor Author

Not happy about the hard-coded bits but there is no better way

Likewise - but at least it is stable

@PVince81 PVince81 merged commit 0fdedfa into master Oct 17, 2017
@PVince81 PVince81 deleted the fix-install-app-order branch October 17, 2017 09:11
@PVince81

Copy link
Copy Markdown
Contributor

@tom please backport to stable10

@lock

lock Bot commented Aug 2, 2019

Copy link
Copy Markdown

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock Bot locked as resolved and limited conversation to collaborators Aug 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants