Migrating Django projects with fixtures

When migrating a Django project it's often useful to work with the built-in dumpdata and loaddata management commands. However, this can easily lead to errors like django.db.utils.IntegrityError: Problem installing fixture Could not load foo.Bar(pk=ba): (1062, "Duplicate entry for key 'app_label'"). Such errors indictate that a similar object already exists in the database, and that you can't insert the duplicate from your fixture.

There are basically two approaches to dump data in a way that solves this problem:

  1. Specify exactly which models to dump
  2. Exclude the conflicting models from the dump

For the first approach you would use a command like:

./manage.py dumpdata --natural --indent=2 app1 app2 app3 auth.User > dump.json

I include the auth data as well in this example, because it's what I usually do. You want to include all third party apps that create their own database tables and store user generated content. You want to avoid apps that auto-populate the database, like thumbnail generators, search indexers, etc.

The second approach is to exclude models. You can do this like:

./manage.py dumpdata --natural --indent=2 --exclude=contenttypes --exclude=sessions.Session --exclude=south.Migrationhistory > dump.json

If you googled the error django.db.utils.IntegrityError: Problem installing fixture Could not load contenttypes.ContentType(pk=X): (1062, "Duplicate entry for key 'app_label'"). you probably want to exclude contenttypes. The example above also exlcudes live sessions and south migrationhistory.

Loading the fixtures

To load the fixtures you need to prepare the database on your new host. First:

./manage.py syncdb --noinput --no-initial-data

I use the --no-initial-data option by default. You might want to remove the option if some of your apps use fixtures, and you didn't include those apps in your data dump.

To finish, apply the migrations and load the data:

./manage.py migrate
./manage.py loaddata dump.json

If you still get errors, read them, and think about which data to include/exclude in the dump. Remeber to delete all objects in the database though before you try a new import. The sqlclear management command can help with that. Or you can simply DROP the database and CREATE it again.

4 comments

  1. avatar
    wrote this comment on
    hi, i am try your solution i exclude auth,content type,session,migration history but still it gives error "Integrity Error: problem installation fixture dump1" could not load pybb.Profile(pk=1): (1062,'Duplicate entry '1 for key user_id'') please help!!! Thanks in advanced!!!
  2. avatar
    wrote this comment on
    Well your DB already has an object with 1 as user_id. I think the --natural-foo options of dumpdata can help with that.
  3. avatar
    wrote this comment on
    Can you please help me with this http://stackoverflow.com/questions/35539241/getting-django-db-utils-integrityerror-when-migrating-data-from-sqlite-to-postgr
  4. avatar
    wrote this comment on
    Looks like your database is not empty when you import your data. Either remove the duplicates from the database or the fixture.

Reply

Cancel reply
Markdown. Syntax highlighting with <code lang="php"><?php echo "Hello, world!"; ?></code> etc.
DjangoPythonBitcoinTuxDebianHTML5 badgeSaltStackUpset confused bugMoneyHackerUpset confused bugX.OrggitFirefoxWindowMakerBashIs it worth the time?i3 window managerWagtailContainerIrssiNginxSilenceUse a maskWorldInternet securityPianoFontGnuPGThunderbirdJenkinshome-assistant-logo