Migrate is broken with elixir 0.6.0

[guest]

I had migrate scripts that worked with Elixir 0.5.2 and they don' t anymore. I get this error:

File "urssus/versioning/versions/001_Add_initial_tables.py", line 42, in <module>

setup_all()

File "urssus/versioning/versions/init.py", line 145, in setup_all

File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/entity.py", line 815, in setup_entities File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/entity.py", line 154, in create_pk_cols File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/entity.py", line 432, in call_builders File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 371, in create_pk_cols File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 627, in create_keys File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 431, in inverse File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 424, in target File "urssus/versioning/versions/init.py", line 108, in resolve

Exception: %s resolves to several entities, you should use the full path (including the full module name) to that entity.

I attach the script that gives this error

[guest]

The migrate script that givs the error

[ged]

There is probably something fishy in your code somewhere... The code you have attached here is perfectly fine. What seems to happen is that you are somehow importing those entities twice. It's kinda strange as python normally prevents that to happen unless you unload the module manually or something like that.

[guest]

How can that be a problem with my code? That's all the code :-)

I mean, I can delete the rest of the project, and when I run the migrate script it crashes that way. Nothing from anywhere else in the app is imported, since I am running migrate manually.

Any other ideas?

[ged]

Which version of migrate are you using and how do you use it? I've never used migrate myself, but installing it (using easy_install) gave me version 0.4.5, and running your test case returns:

ged@plaf:~/devel/elixir/trunk/ > python 001_Add_initial_tables.py Traceback (most recent call last):

File "001_Add_initial_tables.py", line 6, in <module>

meta = MetaData(migrate.migrate_engine)

AttributeError: 'module' object has no attribute 'migrate_engine'

[guest]

Using migrate 0.4.5 and Elixir 0.6.0, here's how you reproduce the bug:

migrate create testrepo testrepo
migrate manage manage.py --repository=testrepo --url=sqlite:///testbase
python manage.py script "Add initial tables"
cp 001_Add_initial_tables.py testrepo/versions/
python manage.py version_control
python manage.py upgrade

You should see this:

0 -> 1...  done

And have a working sqlite database. And with these versions, it works:

migrate 0.4.5-1
elixir 0.5.2-1
sqlalchemy 0.4.6-1

Upgrading to elixir 0.6.0:

python manage.py upgrade
0 -> 1...                                           
Traceback (most recent call last):                  
  File "manage.py", line 4, in <module>             
    main(url='sqlite:///testbase',repository='testrepo')
  File "/usr/lib/python2.5/site-packages/migrate/versioning/shell.py", line 132, in main
    ret = command(**kwargs)                                                             
  File "/usr/lib/python2.5/site-packages/migrate/versioning/api.py", line 188, in upgrade
    return _migrate(url,repository,version,upgrade=True,err=err,**opts)                  
  File "/usr/lib/python2.5/site-packages/migrate/versioning/api.py", line 224, in _migrate
    schema.runchange(ver,change,changeset.step)
  File "/usr/lib/python2.5/site-packages/migrate/versioning/schema.py", line 159, in runchange
    change.run(self.engine,step)
  File "/usr/lib/python2.5/site-packages/migrate/versioning/script/py.py", line 84, in run
    func = self._func(funcname)
  File "/usr/lib/python2.5/site-packages/migrate/versioning/script/py.py", line 68, in _func
    fn = getattr(self.module, funcname, None)
  File "/usr/lib/python2.5/site-packages/migrate/versioning/script/py.py", line 62, in _get_module
    self._module = self.verify_module(self.path)
  File "/usr/lib/python2.5/site-packages/migrate/versioning/script/py.py", line 50, in verify_module
    module=import_path(path)
  File "/usr/lib/python2.5/site-packages/migrate/versioning/util/importpath.py", line 13, in import_path
    reload(module) # Might be out of date during tests
  File "/home/ralsina/elixirbug/001_Add_initial_tables.py", line 42, in <module>
    setup_all()
  File "testrepo/versions/__init__.py", line 145, in setup_all

  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/entity.py", line 815, in setup_entities
  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/entity.py", line 154, in create_pk_cols
  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/entity.py", line 432, in call_builders
  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 371, in create_pk_cols
  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 627, in create_keys
  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 431, in inverse
  File "/usr/lib/python2.5/site-packages/Elixir-0.6.0-py2.5.egg/elixir/relationships.py", line 424, in target
  File "testrepo/versions/__init__.py", line 108, in resolve

Exception: %s resolves to several entities, you should use the full path (including the full module name) to that entity.

[ged]

Sorry for the huge delay... I didn't see your answer until a few days ago. In the meantime I asked Daniel to enable trac's detailed timeline (ie including ticket updates) so this shouldn't happen anymore.

Anyway, after investigation, this is due to migrate's strange practice of forcing a reload of every module they import, in effect loading them all twice.

For reference, I'll attach an Elixir-only test case.

[ged]

And btw, thanks a lot for the detailed instructions to reproduce the problem. Very much appreciated.

[ged]

Model for the Elixir-only test case

[ged]

Elixir-only testcase

[ged]

The problematic line is at:

http://code.google.com/p/sqlalchemy-migrate/source/browse/trunk/migrate/versioning/util/importpath.py#13

[ged]

In fact, there is a very easy workaround, which is to add a "cleanup_all()" line in your module defining your entities, before you declare them.

[ged]

For future reference, the correct way to fix the reload problem is simply to not use a global entity list (hardwired into Elixir). In that case, the collection object itself will be recreated when the module is reloaded, hence the problem would disappear.

As a workaround (other than the simple one explained above), people could apply the following patch (applies to r467).

[guest]

Why is this ticket still open ? Will the patch be merged ?

[guest]

I have been facing this exact problem now (years later); would be great if there was some more official fix / hint / way of dealing with this.

(Different 'guest' this time BTW.)