External storage tables are now created in a migration by PVince81 · Pull Request #26923 · owncloud/core

PVince81 · 2017-01-11T11:00:22Z

Description

Reimplement the old files_external DB structure as a Doctrine migration.

Related Issue

Fixes #26907

Motivation and Context

Because starting with 10.0 db_structure.xml is not used any more on updates and we need those tables.

How Has This Been Tested?

TEST: setup this branch from scratch: tables properly created
TEST: setup OC 9.1 without files_external, migrate to this branch: tables properly created
TEST: setup OC 9.1 with files_external enabled and a configured ext storage, migrate to this branch: existing tables were kept with contents

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

TODOs:

⚠️ make sure the table structure is exactly the same as it was when created from db_structure

mention-bot · 2017-01-11T11:00:24Z

@PVince81, thanks for your PR! By analyzing the history of the files in this pull request, we identified @tanghus, @bantu and @butonic to be potential reviewers.

PVince81 · 2017-01-11T11:02:33Z

BUG: wrong integer type
BUG: wrong collation

left is 9.1, right is this branch:

441,442c464,465
<   `applicable_id` bigint(20) NOT NULL AUTO_INCREMENT,
<   `mount_id` bigint(20) NOT NULL,
---
>   `applicable_id` int(11) NOT NULL AUTO_INCREMENT,
>   `mount_id` int(11) NOT NULL,
444c467
<   `value` varchar(64) COLLATE utf8_bin DEFAULT NULL,
---
>   `value` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
449c472
< ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
---
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

PVince81 · 2017-01-11T11:52:35Z

Added explicit collation and explicit bigint(20) values.
I hope this is correct...

I think I shouldn't need to explicitly specify the collation, see #26925

Also I'm a bit surprised that the original schema had 256 chars for "mountpoint" specified, but the resulting schema in Mysql only had 128 chars defined. That's why I also set it to 128 here in the migration...

Basically all I tried to do is to prevent any differences between the 9.1 schema and the current one by diffing the schema dumps.

@DeepDiver1975 @butonic please review

PVince81 · 2017-01-11T18:00:31Z

    Test\Files\External\Service\DBConfigServiceTest.testAddApplicable
    Test\Files\External\Service\DBConfigServiceTest.testRemoveApplicableGlobal
    Test\Files\External\Service\DBConfigServiceTest.testGetAdminMountsForGlobal
    Test\Files\External\Service\GlobalStoragesServiceTest::testAddStorage.testAddStorage with data set #0
    Test\Files\External\Service\GlobalStoragesServiceTest::testUpdateStorage.testUpdateStorage with data set #0
    Test\Files\External\Service\GlobalStoragesServiceTest::testUpdateStorage.testUpdateStorage with data set #1
    Test\Files\External\Service\GlobalStoragesServiceTest::testUpdateStorage.testUpdateStorage with data set #2
    Test\Files\External\Service\GlobalStoragesServiceTest::testUpdateStorage.testUpdateStorage with data set #3
    Test\Files\External\Service\GlobalStoragesServiceTest::testHooksAddStorage.testHooksAddStorage with data set #0
    Test\Files\External\Service\GlobalStoragesServiceTest::testHooksUpdateStorage.testHooksUpdateStorage with data set #0
    Test\Files\External\Service\GlobalStoragesServiceTest::testHooksUpdateStorage.testHooksUpdateStorage with data set #3
    Test\Files\External\Service\GlobalStoragesServiceTest::testHooksDeleteStorage.testHooksDeleteStorage with data set #1
    Test\Files\External\Service\GlobalStoragesServiceTest::testDeleteStorage.testDeleteStorage with data set #0
    Test\Files\External\Service\GlobalStoragesServiceTest::testDeleteStorage.testDeleteStorage with data set #1
    Test\Files\External\Service\GlobalStoragesServiceTest.testGetStoragesBackendNotVisible
    Test\Files\External\Service\GlobalStoragesServiceTest.testGetStoragesAuthMechanismNotVisible
    Test\Files\External\Service\GlobalStoragesServiceTest.testUpdateStorageMountPoint
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetStorageWithApplicable.testGetStorageWithApplicable with data set #0
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testDeleteStorage.testDeleteStorage with data set #0
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testDeleteStorage.testDeleteStorage with data set #1
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetUniqueStorages.testGetUniqueStorages with data set #0
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetUniqueStorages.testGetUniqueStorages with data set #1
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetUniqueStorages.testGetUniqueStorages with data set #2
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetUniqueStorages.testGetUniqueStorages with data set #3
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetUniqueStorages.testGetUniqueStorages with data set #8
    Test\Files\External\Service\UserGlobalStoragesServiceTest::testGetUniqueStorages.testGetUniqueStorages with data set #9
    Test\Files\External\Service\UserGlobalStoragesServiceTest.testUpdateStorage

Looks like I messed up some indices:

Integrity constraint violation: 19 NOT NULL constraint failed: oc_external_applicable.value

PVince81 · 2017-01-12T08:41:23Z

Either Oracle had a hiccup or the migrations stuff doesn't work properly with Oracle:

19:32:57   [Doctrine\DBAL\Exception\TableNotFoundException]                                                                                                                                                                                                                                                                              
19:32:57   An exception occurred while executing 'SELECT m."mount_id", "mount_point", "storage_backend", "auth_backend", "priority", m."type" FROM "oc_external_mounts" m INNER JOIN "oc_external_applicable" a ON m."mount_id" = a."mount_id" WHERE (a."type" = ?) AND (a."value" = ?) AND (m."type" = 1)' with params [3, "admin23"]:  
19:32:57   ORA-00942: table or view does not exist

PVince81 · 2017-01-12T08:44:39Z

Rebased + squashed. Let's hope the Oracle tests work now.

VicDeo · 2017-01-16T17:46:20Z

+	 * @param Schema $schema
+	 */
+	public function up(Schema $schema) {
+		$prefix = $this->connection->getPrefix();


@PVince81 $this->connection is an instance of \Doctrine\DBAL\Connection and has no method getPrefix

No, it's the ownCloud connection. Else this PR would not work at all at setup time. Try it 😉

@PVince81 I was kicked in the a** by @DeepDiver1975 several times for exactly that kind of magic

I'm very aware of this - thanks for fighting me with my own weapons 🔪 .... maybe we need to come up with our own AbstractMigration class to allow proper handling of things like this .... let's keep this out of this pr for now.

PVince81 · 2017-01-18T11:15:20Z

👍 for @DeepDiver1975's fix

PVince81 · 2017-01-18T11:16:30Z

@DeepDiver1975 @VicDeo is my commit acceptable then and can we merge it ?

PVince81 · 2017-01-23T06:28:11Z

@DeepDiver1975 looked into the Oracle problem last week and said we'll need our own wrapper classes to Oraclify whatever input is coming in, like for example backtick-escaping stuff.

PVince81 · 2017-01-30T09:22:16Z

Superseded by #27041

lock · 2019-08-04T00:02:42Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

PVince81 added 2 - Developing sev1-critical labels Jan 11, 2017

PVince81 added this to the 10.0 milestone Jan 11, 2017

PVince81 self-assigned this Jan 11, 2017

This was referenced Jan 11, 2017

Upgrade from 9.1.3 to master 💣 #26907

Closed

Doctrine migration wrong default collation #26925

Closed

PVince81 added 3 - To Review and removed 2 - Developing labels Jan 11, 2017

External storage tables are now created in a migration

294cca5

PVince81 force-pushed the migrations-extstorage-tables branch from 3dbab44 to 294cca5 Compare January 12, 2017 08:44

VicDeo reviewed Jan 17, 2017

View reviewed changes

The default collation is set in the connection for schema operations

8e6f512

PVince81 closed this Jan 30, 2017

PVince81 deleted the migrations-extstorage-tables branch January 30, 2017 09:22

lock Bot locked as resolved and limited conversation to collaborators Aug 4, 2019

Uh oh!

Conversation

PVince81 commented Jan 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

TODOs:

Uh oh!

mention-bot commented Jan 11, 2017

Uh oh!

PVince81 commented Jan 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PVince81 commented Jan 11, 2017

Uh oh!

PVince81 commented Jan 11, 2017

Uh oh!

PVince81 commented Jan 12, 2017

Uh oh!

PVince81 commented Jan 12, 2017

Uh oh!

VicDeo Jan 16, 2017

Choose a reason for hiding this comment

Uh oh!

PVince81 Jan 18, 2017

Choose a reason for hiding this comment

Uh oh!

VicDeo Jan 18, 2017

Choose a reason for hiding this comment

Uh oh!

DeepDiver1975 Jan 18, 2017

Choose a reason for hiding this comment

Uh oh!

PVince81 commented Jan 18, 2017

Uh oh!

PVince81 commented Jan 18, 2017

Uh oh!

PVince81 commented Jan 23, 2017

Uh oh!

PVince81 commented Jan 30, 2017

Uh oh!

lock Bot commented Aug 4, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PVince81 commented Jan 11, 2017 •

edited

Loading

PVince81 commented Jan 11, 2017 •

edited

Loading