[9.0][account] Analytic line views not removed during migration

[JordiBForgeFlow]

Impacted versions: 9.0

Steps to reproduce:
Migrate a demo database in v8 with modules 'analytic' and 'account' installed.

Current behavior:
The form, search and tree views for model 'account.analytic.line.tree' are not deleted.

Depending on how Odoo decides to render the views, this may result in an error such as:
File "/home/jordi/Odoo9/openerp/addons/base/ir/ir_ui_view.py", line 463, in raise_view_error raise AttributeError(message) AttributeError: Field journal_id does not exist Error context: View account.analytic.line.form [view_id: 566, xml_id: n/a, model: account.analytic.line, parent_id: n/a]

Expected behavior:
The following views should be deleted during the migration process, or during the database cleanup:
account.view_account_analytic_line_tree
account.view_account_analytic_line_form
account.view_account_analytic_line_filter

Video/Screenshot link (optional):

[hbrunn]

have a look at the log of a successful migration: https://travis-ci.org/OCA/OpenUpgrade/builds/147954920#L4416

At the end, odoo will delete all records pointed to by obsolete xmlids from migrated models. Check out how this looks in your migration, probably you'll find a line where odoo tries to do this, but there are other views inheriting from it or some other reason leads to the view not being deleted.

[JordiBForgeFlow]

Looking at Travis I can see:
2016-07-28 07:14:48,586 24245 INFO openupgrade openerp.addons.base.ir.ir_model: Deleting 824@ir.ui.view
2016-07-28 07:14:48,589 24245 INFO openupgrade openerp.sql_db: bad query: delete from ir_ui_view where id IN (824)
2016-07-28 07:14:48,590 24245 WARNING openupgrade openerp.addons.base.ir.ir_model: Could not delete obsolete record with id: 824 of model ir.ui.view

Them the following error message at the end of the log:
Error context:
View account.analytic.line.form
[view_id: 824, xml_id: n/a, model: account.analytic.line, parent_id: n/a]
2016-07-28 07:14:58,348 24245 INFO openupgrade openerp.addons.base.ir.ir_ui_view: Field journal_id does not exist

This seems to indicate that openupgrade tried to, but could not delete the view, and is causing trouble later on.

[hbrunn]

so deleting the views is inhibited, which is a good think. Look at this:

odoo8_demo=> select id from ir_model_data where name in ('view_account_analytic_line_form');
  id  
------
 824
(1 row)

odoo8_demo=> delete from ir_ui_view where id=824;
ERROR:  update or delete on table "ir_ui_view" violates foreign key constraint "ir_ui_view_inherit_id_fkey" on table "ir_ui_view"
DETAIL:  Key (id)=(824) is still referenced from table "ir_ui_view".
odoo8_demo=> select id from ir_ui_view where inherit_id=824;
  id  
------
 1574
 2180
(2 rows)

odoo8_demo=> select module || '.' || name from ir_model_data where model='ir.ui.view' and res_id in (select id from ir_ui_view where inherit_id=824);
                               ?column?                               
----------------------------------------------------------------------
 hr_timesheet_invoice.view_account_analytic_line_form_inherit
 project_timesheet.view_account_analytic_line_form_inherit_account_id

So it's a matter of patience until those modules are migrated too, then the inheriting views will be gone too, and we should be fine.

[JordiBForgeFlow]

hmm, thanks. I will then speed up the migration of those modules. Because otherwise it gives me trouble.

[hbrunn]

database cleanup should be your friend here too

[JordiBForgeFlow]

project_timesheet is already migrated, and hr_timesheet_invoice does not migrate at all, so the problem should be solved by now.

When you purge obsolete views with database_cleanup, the views associated to 'hr_timesheet_invoice' are deleted, but not the views from 'project_timesheet'.

During the migration the views are losing the external ID. See below. When I exported the views, they generated a new external ID.

This would explain why updating the modules and performing a database cleanup is not removing these obsolete views.

[hbrunn]

hmmm, yes. The obsolete module is the problem, didn't pay attention to that. And indeed the xmlids will be purged, after that we don't really have a chance to detect views to be deleted.

@OCA/openupgrade-maintainers how should we handle this kind of situation? I'd be in favor of adding obsolete modules in openupgrade with the old dependencies, but further empty manifest and init. Then odoo's xmlid deletion mechanism should do the rest. @jbeficent could you try if that works for you?

Another approach would be to patch the server not to delete the xmlid when it is impossible to delete the record pointed to (is this not actually a bug in odoo?). Then database_cleanup can do its work. This would involve removing the id in question when we arrive in https://github.com/odoo/odoo/blob/9.0/openerp/addons/base/ir/ir_model.py#L1248 from ids, so that it is not unlinked in https://github.com/odoo/odoo/blob/9.0/openerp/addons/base/ir/ir_model.py#L1278 - this might actually be the better approach.

[pedrobaeza]

I prefer to add the removed addons from version to version as dummy modules (without content), or add content if needed for migration purposes. In the documentation, they should also be included in the module coverage, because when you check if your DB can be migrated from v8 to v9, you are seeing if v8 modules are covered in this table. If you don't see the entry in the table, then you don't know what to do with them.

[hbrunn]

@pedrobaeza my difficulty with this is that the side effect of the dummy module is that all the module's columns/tables will be dropped, which contradicts our stance of keeping as much data intact as possible. And moving all columns/tables away in such a case also seems a bit untenable. What's you opinion about the second option? The more I think about this, the more I think this is a bug that should be fixed in upstream anyways, independently of openupgrade. For example when you uninstall a module that adds some record on a table with a uniqueness constraint. For whatever reason this record cannot be deleted, but the rest goes fine, so the action succeeds. Then when reinstalling the module, you'll get an exception because odoo will try to recreate the record that couldn't be deleted, uniqueness constraint, boom.
Maybe we can find some upstream module that adds categories, where it should be simple to create such a constellation. With this, we can go to odoo/odoo and have it merged there, and remerge here (yes, I understand how optimistic/naive this is)

[pedrobaeza]

OK, I see the problem. Yes, you're right, we can't do that.

Let's try then the second path.

Anyway, I think the covered modules issue should be treated as mentioned. Putting for example "Merged and treated in module X", or "Nothing to do", or "Migrated module in OCA/repository", what do you think about this part?