Separate Laravel Migrations in Structure and actual Data changes

[Sairahcaz]

I feel like Laravels migration system could be improved. Especially for testing purposes. Wouldn't it make sense if we could separate migrations into schema changes and actual data changes?

This would be super helpful for testing! In testing you mostly want to work with an empty db and a migration which do data changes might not work or even break your migrations.

Sure you can do a schema:dump, but as soon as a team member writes the next migration that changes data or depends on related data, it will break again.

We could fix that if migrations would have 2 sub folders, one called "schema" the other "data" and for testing only the schema migrations would be executed.

Does this makes sense?

[alxndrmlr]

I don't think this is necessary.

Generally speaking I try to avoid data changes in migrations due potential timeouts in production that you miss in CI because its a smaller dataset.

Typically I'd write a migration to add a new column during a deployment, then outside of the deployment cycle we would populate the new column (run some SQL directly or it may be something that is eventually consistent via app-layer data changes/job). Then later we drop the old column.

I think the solution you're seeking is stemming from how you're managing your seeder data, correct me if I'm wrong but it seems like you're storing data in your schema.sql

You should pull your data generation out into Seeders so that they can be updated as your models change more easily. Also if you have a bunch of old data a simple way to get started with this is to do something like;

• Export Columns for Table to JSON from TablePlus
• Write a Seeder that reads the JSON in and uses a Factory to fill the remaining columns

<?php

namespace Database\Seeders;

use App\Models\Organization;
use Illuminate\Database\Console\Seeds\WithoutModelEvents;
use Illuminate\Database\Seeder;

class OrganizationSeeder extends Seeder
{
    use WithoutModelEvents;

    public function run(): void
    {
        // To update the organizations.json file, run the following sql in TablePlus copy as JSON
        // SELECT `name`, sector, longitude, latitude, phone, website_url, linkedin_url, twitter_url, facebook_url, address, city, state, country, timezone FROM organizations
        // Then export to organizations.json file
        $data = json_decode(file_get_contents(database_path('data/organizations.json')), true);

        foreach ($data as $datum) {
            Organization::factory()->createQuietly($datum);
        }
    }
}

[ghost]

Data changes can be done with seeding or via one time command.

[raftalks]

Here is an interesting package that does address the exact problem https://github.com/TimoKoerber/laravel-one-time-operations

[Sairahcaz]

This is pretty cool, thanks!
Honestly I think this should be built in to Laravel it makes CI/CD and especially testing so much easier...

[shaedrich]

I see your point. I sometimes work with codebases that have quite a few migration files. It would be nice to see at the first glance, where I can check the structure (the CREATE statement), because other migration files are usually renaming or deleting columns, setting indices and moving data from one column to another or even another table, but they don't show the whole picture. But, sure, this can be counteracted with schema:dump()

Another argument against having, let's say, two folders (one for creating and one for changing your table structure), is, that migration files should always be run in the same order and always all of them for consistency. Sure, Laravel could collect the files and sort them, but that would be an extra step.

In the end, it comes down to Laravel being built on the principle that migrations are simple, consistent and repeatable. The way, they are now, they keep their promise. So, chance is not likely.