[PATCH] [fix] allow to run migrations of CW database before 3.24

Laurent Peuch cortex at worlddomination.be
Tue Jun 4 10:41:35 CEST 2019


On Tue, Jun 04, 2019 at 10:17:10AM +0200, Denis Laxalde wrote:
> Laurent Peuch a écrit :
> > On Wed, May 29, 2019 at 11:01:00AM +0200, Denis Laxalde wrote:
> > > Laurent Peuch a écrit :
> > > > # HG changeset patch
> > > > # User Laurent Peuch <cortex at worlddomination.be>
> > > > # Date 1559087484 -7200
> > > > #      Wed May 29 01:51:24 2019 +0200
> > > > # Node ID 580c0a8681050ed789fbe1837c6cf2b9f45f9a21
> > > > # Parent  6b314fc558ed33e09ff453053c6d0097e6232f95
> > > > [fix] allow to run migrations of CW database before 3.24
> > > > 
> > > > The big problem is that in 3.24 the "asource" column of entities has been
> > > > removed from the database schema. This is not a problem in itself, the problem
> > > > is that some migration operation like "add_attribute" will modify the entities
> > > > columns and past 3.24 will try to insert values without the asource column (as
> > > > expected) but ... if you are migrating an old DB, you'll still have this
> > > > asource column and this "add_attribute" (or other similar operations) will
> > > > fails because "asource" is a non-null column and "add_attribute" won't provide
> > > > a value for it anymore.
> > > 
> > > I don't understand how this would happen. In changeset 054a947b5415
> > > (linked below), there's a migration step doing 'ALTER TABLE entities
> > > DROP COLUMN asource'. So the column shouldn't be there anymore.
> > > 
> > > Or am I missing something?
> > 
> > Indeed, it's quite a mixed of different issues:
> > * indeed asource is droped in 3.24
> > * but, all the operation that works on the entities table are modified
> >   and will work *without* asource in ALL situations
> > 
> > By consequence, if you try to run a migration before 3.24 that works
> > on the entities table, CW code will generate SQL queries that don't
> > take the "asource" column into account (because this has been removed
> > from all the code without taking into account if the 3.24 migration
> > has been run or not) and since "asource" can't be null, sql
> > constrains fail and the migration can't run.
> > 
> > So it's really a story about desynchronisation between how CW
> > generates queries regarding the entities table after this changeset
> > and if the 3.24 migration has been run or not :/
> > 
> 
> I'm afraid I still don't understand the issue. Perhaps somebody else can
> take a look?
> 
> What do you mean by "if the 3.24 migration has been run or not"? Surely,
> if one does not run that migration, they might get into troubles
> 
> As far as I know, when one upgrades an instance, the cubicweb migration
> is first run (after bootstrap-migration stuffs) and then cubes'
> migrations are run. So by having cubicweb 3.24 migration run, "asource"
> column will get dropped and then all SQL queries from cubes' migrations
> should account for "entities" table change and be correct.
> 
> Do you have a concrete example of this issue?

Yes, let me make you a timeline, this should help:

- take the database of a very old instance (abreton on 3.14)
- restore it
- launch upgrade
- the file cubicweb/misc/migration/bootstrapmigration_repository.py
  will be executed
- this operation will be executed: add_attribute('CWAttribute', 'formula', commit=False)
  (here https://hg.logilab.org/master/cubicweb/file/tip/cubicweb/misc/migration/bootstrapmigration_repository.py#l381)
- when executing this operation, CW will modify the entities table
- the SQL generated to modify the entities table doesn't take into
  account the "asource" column because of the changeset in 3.24
- the "asource" column is required, therefor the SQL column failed
- migration 3.24 that needs to be reached to remove the "asource"
  column can't be reached because of the previous bug

I've tried removing the asource column, this allows the
add_attribute('CWAttribute', 'formula', commit=False) to run but fails
in migrations after that (before 3.24) that explicitely requires it.

To fix this issue, this patch makes CW takes into account the
"asource" column if it's present (in the specific situation where it
has broken this upgrade, not in all cases)

I hope this makes things a bit more understandable :x

-- 

Laurent Peuch -- Bram



More information about the cubicweb-devel mailing list