Migrating from v7.0.1 to v7.3.2 fails with "SQLSTATE[42S21]: Column already exists"

[rfilmyer]

Lychee version

v7.3.2

Did you check the latest Lychee version?

Yes, I did

Which PHP version are you using?

PHP 8.5

Detailed description of the problem

Attempting to upgrade from v7.0.1 to v7.3.2, my lychee installation fails to migrate and spits out the error on photos..com/migrate

HttpException
SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'duration' (Connection: mysql, Host: lychee_db, Port: 3306, Database: lychee, SQL: alter table `photos` add `duration` varchar(255) null after `focal`)

This lines up with the corresponding errors.log entry

[2026-02-03 00:40:49] production.ERROR: PDOStatement::execute:584 SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'duration'
App\Console\Commands\Laravel\Migrate::handle:55 SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'duration'; caused by
Illuminate\Database\Connection::runQueryCallback:838 SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'duration' (Connection: mysql, Host: lychee_db, Port: 3306, Database: lychee, SQL: alter table `photos` add `duration` varchar(255) null after `focal`); caused by
App\Console\Commands\Laravel\Migrate::handle:55 SQLSTATE[42S21]: Column already exists: 1060 Duplicate column name 'duration' (Connection: mysql, Host: lychee_db, Port: 3306, Database: lychee, SQL: alter table `photos` add `duration` varchar(255) null after `focal`); caused by

This is the half-migrated schema of the photos table:

DROP TABLE IF EXISTS `photos`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8mb4 */;
CREATE TABLE `photos` (
  `id` char(24) NOT NULL,
  `created_at` datetime(6) NOT NULL,
  `updated_at` datetime(6) NOT NULL,
  `owner_id` int(10) unsigned NOT NULL DEFAULT 0,
  `old_album_id` char(24) DEFAULT NULL,
  `title` varchar(300) DEFAULT NULL,
  `description` text DEFAULT NULL,
  `license` varchar(20) NOT NULL DEFAULT 'none',
  `is_starred` tinyint(1) NOT NULL DEFAULT 0,
  `rating_avg` decimal(5,4) DEFAULT NULL,
  `iso` varchar(255) DEFAULT NULL,
  `make` varchar(255) DEFAULT NULL,
  `model` varchar(255) DEFAULT NULL,
  `lens` varchar(255) DEFAULT NULL,
  `aperture` varchar(255) DEFAULT NULL,
  `shutter` varchar(255) DEFAULT NULL,
  `focal` varchar(255) DEFAULT NULL,
  `duration` varchar(255) DEFAULT NULL,
  `fps` varchar(255) DEFAULT NULL,
  `latitude` decimal(10,8) DEFAULT NULL,
  `longitude` decimal(11,8) DEFAULT NULL,
  `altitude` decimal(10,4) DEFAULT NULL,
  `img_direction` decimal(10,4) DEFAULT NULL,
  `location` varchar(255) DEFAULT NULL,
  `taken_at` datetime(6) DEFAULT NULL COMMENT 'relative to UTC',
  `taken_at_orig_tz` varchar(31) DEFAULT NULL COMMENT 'the timezone at which the photo has originally been taken',
  `initial_taken_at` datetime DEFAULT NULL COMMENT 'backup of the original taken_at value',
  `initial_taken_at_orig_tz` varchar(31) DEFAULT NULL COMMENT 'backup of the timezone at which the photo has originally been taken',
  `type` varchar(30) NOT NULL,
  `filesize` bigint(20) unsigned NOT NULL DEFAULT 0,
  `checksum` varchar(40) NOT NULL,
  `original_checksum` varchar(40) NOT NULL,
  `live_photo_short_path` varchar(255) DEFAULT NULL,
  `live_photo_content_id` varchar(255) DEFAULT NULL,
  `live_photo_checksum` varchar(40) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `photos_owner_id_foreign` (`owner_id`),
  KEY `photos_checksum_index` (`checksum`),
  KEY `photos_original_checksum_index` (`original_checksum`),
  KEY `photos_live_photo_content_id_index` (`live_photo_content_id`),
  KEY `photos_live_photo_checksum_index` (`live_photo_checksum`),
  KEY `photos_album_id_taken_at_index` (`old_album_id`,`taken_at`),
  KEY `photos_album_id_created_at_index` (`old_album_id`,`created_at`),
  KEY `photos_album_id_is_starred_index` (`old_album_id`,`is_starred`),
  KEY `photos_album_id_type_index` (`old_album_id`,`type`),
  KEY `photos_album_id_is_starred_created_at_index` (`old_album_id`,`is_starred`,`created_at`),
  KEY `photos_album_id_is_starred_taken_at_index` (`old_album_id`,`is_starred`,`taken_at`),
  KEY `photos_album_id_is_starred_type_index` (`old_album_id`,`is_starred`,`type`),
  KEY `photos_album_id_is_starred_description(128)_index` (`old_album_id`,`is_starred`,`description`(128)),
  KEY `photos_is_starred_title_taken_created_index` (`is_starred`,`title`,`taken_at`,`created_at`),
  KEY `photos_rating_avg_index` (`rating_avg`),
  CONSTRAINT `photos_owner_id_foreign` FOREIGN KEY (`owner_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC;
/*!40101 SET character_set_client = @saved_cs_client */;

Conveniently, I have two lychee deployments and didn't upgrade the other one, this is what the photos table looks like on the one still on v7.0.1:

DROP TABLE IF EXISTS `photos`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8mb4 */;
CREATE TABLE `photos` (
  `id` char(24) NOT NULL,
  `created_at` datetime(6) NOT NULL,
  `updated_at` datetime(6) NOT NULL,
  `owner_id` int(10) unsigned NOT NULL DEFAULT 0,
  `old_album_id` char(24) DEFAULT NULL,
  `title` varchar(100) NOT NULL,
  `description` text DEFAULT NULL,
  `license` varchar(20) NOT NULL DEFAULT 'none',
  `is_starred` tinyint(1) NOT NULL DEFAULT 0,
  `iso` varchar(255) DEFAULT NULL,
  `make` varchar(255) DEFAULT NULL,
  `model` varchar(255) DEFAULT NULL,
  `lens` varchar(255) DEFAULT NULL,
  `aperture` varchar(255) DEFAULT NULL,
  `shutter` varchar(255) DEFAULT NULL,
  `focal` varchar(255) DEFAULT NULL,
  `latitude` decimal(10,8) DEFAULT NULL,
  `longitude` decimal(11,8) DEFAULT NULL,
  `altitude` decimal(10,4) DEFAULT NULL,
  `img_direction` decimal(10,4) DEFAULT NULL,
  `location` varchar(255) DEFAULT NULL,
  `taken_at` datetime(6) DEFAULT NULL COMMENT 'relative to UTC',
  `taken_at_orig_tz` varchar(31) DEFAULT NULL COMMENT 'the timezone at which the photo has originally been taken',
  `initial_taken_at` datetime DEFAULT NULL COMMENT 'backup of the original taken_at value',
  `initial_taken_at_orig_tz` varchar(31) DEFAULT NULL COMMENT 'backup of the timezone at which the photo has originally been taken',
  `type` varchar(30) NOT NULL,
  `filesize` bigint(20) unsigned NOT NULL DEFAULT 0,
  `checksum` varchar(40) NOT NULL,
  `original_checksum` varchar(40) NOT NULL,
  `live_photo_short_path` varchar(255) DEFAULT NULL,
  `live_photo_content_id` varchar(255) DEFAULT NULL,
  `live_photo_checksum` varchar(40) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `photos_owner_id_foreign` (`owner_id`),
  KEY `photos_checksum_index` (`checksum`),
  KEY `photos_original_checksum_index` (`original_checksum`),
  KEY `photos_live_photo_content_id_index` (`live_photo_content_id`),
  KEY `photos_live_photo_checksum_index` (`live_photo_checksum`),
  KEY `photos_album_id_taken_at_index` (`old_album_id`,`taken_at`),
  KEY `photos_album_id_created_at_index` (`old_album_id`,`created_at`),
  KEY `photos_album_id_is_starred_index` (`old_album_id`,`is_starred`),
  KEY `photos_album_id_type_index` (`old_album_id`,`type`),
  KEY `photos_album_id_is_starred_created_at_index` (`old_album_id`,`is_starred`,`created_at`),
  KEY `photos_album_id_is_starred_taken_at_index` (`old_album_id`,`is_starred`,`taken_at`),
  KEY `photos_album_id_is_starred_type_index` (`old_album_id`,`is_starred`,`type`),
  KEY `photos_album_id_is_starred_title_index` (`old_album_id`,`is_starred`,`title`),
  KEY `photos_album_id_is_starred_description(128)_index` (`old_album_id`,`is_starred`,`description`(128)),
  CONSTRAINT `photos_owner_id_foreign` FOREIGN KEY (`owner_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC;

Both lychee installations are several years old, dating back to mid-2022.

Steps to reproduce the issue

1. Set up Lychee on v7.0.1 with Docker
2. Upgrade the setup to v7.3.2
3. Start the migration

Diagnostics [REQUIRED]

�[36mSmart Diagnostics�[39m
�[36m-----------------�[39m
�[32mInfo: �[39mHash: 312792fb4db3b834—dc0bb07f48a6135e
         2581 files and 14015 vendor files
�[33mWarning: �[39m/*********c/uploads/ is owned by group www-data, but should be owned by one out of root
�[33mWarning: �[39m/*********c/uploads/ has permissions 0755, but should have 0775
�[33mWarning: �[39m/a********ge/tmp/jobs/ is owned by group www-data, but should be owned by one out of root
�[33mWarning: �[39m/a********ge/tmp/jobs/ has permissions 0750, but should have 0775
�[33mWarning: �[39m/a********ge/tmp/uploads/ is owned by group www-data, but should be owned by one out of root
�[33mWarning: �[39m/a********ge/tmp/uploads/ has permissions 0750, but should have 0775
�[32mInfo: �[39mFull directory permission check is disabled
�[31mError: �[39mDatabase is behind file version. Please apply the migrations.
�[32mInfo: �[39mFound 9878 placeholders that could be generated.
         You can use `php artisan lychee:generate_thumbs placeholder 9878` to generate them.
�[32mInfo: �[39mHave you considered supporting Lychee? :)

�[36mSystem Information�[39m
�[36m------------------�[39m
Lychee Version (release):                  7.3.2
DB Version:                                7.1.2

Docker:                                    lycheeorg-frankenphp
composer install:                          --no-dev
APP_ENV:                                   production
APP_DEBUG:                                 false
APP_URL:                                   set
APP_DIR:                                   default
LOG_VIEWER_ENABLED:                        false

System:                                    Linux
PHP Version:                               8.5.1
PHP User agent:                            Lychee/6 (https://lycheeorg.dev/)
Timezone:                                  America/New_York
Max uploaded file size:                    128M
Max post size:                             128M
Chunk size:                                102.40 MB
Max execution time:                        0
MySQL Version:                             10.11.15-MariaDB-ubu2204

exec() Available:                          yes
Imagick Available:                         1
Imagick Enabled:                           yes
Imagick Version:                           1809
GD Version:                                bundled (2.1.0 compatible)
Number of foreign key:                     33 found.

�[36mConfig Information�[39m
�[36m------------------�[39m
version:                                   070102
check_for_updates:                         0
sorting_photos_col:                        taken_at
sorting_photos_order:                      ASC
sorting_albums_col:                        max_taken_at
sorting_albums_order:                      ASC
imagick:                                   1
skip_duplicates:                           1
small_max_width:                           0
small_max_height:                          360
medium_max_width:                          1920
medium_max_height:                         1080
lang:                                      en
image_overlay_type:                        desc
default_license:                           CC-BY-4.0
compression_quality:                       90
grants_full_photo_access:                  1
delete_imported:                           1
mod_frame_enabled:                         1
mod_frame_refresh:                         30
thumb_2x:                                  1
small_2x:                                  1
medium_2x:                                 1
landing_page_enable:                       0
search_public:                             1
hide_version_number:                       0
recent_age:                                1
grants_download:                           0
photos_wraparound:                         1
map_display:                               1
zip64:                                     1
map_display_public:                        1
map_provider:                              Wikimedia
map_include_subalbums:                     0
update_check_every_days:                   3
has_exiftool:                              1
share_button_visible:                      0
import_via_symlink:                        0
has_ffmpeg:                                1
location_decoding:                         1
location_decoding_timeout:                 30
location_show:                             1
location_show_public:                      0
rss_enable:                                0
rss_recent_days:                           7
rss_max_items:                             100
prefer_available_xmp_metadata:             0
editor_enabled:                            1
lossless_optimization:                     0
local_takestamp_video_formats:             .avi|.mov
log_max_num_line:                          1000
unlock_password_photos_with_url_param:     0
nsfw_visible:                              1
nsfw_blur:                                 0
nsfw_warning:                              0
nsfw_warning_admin:                        0
nsfw_banner_override:                      
map_display_direction:                     1
album_subtitle_type:                       oldstyle
upload_processing_limit:                   4
new_photos_notification:                   0
zip_deflate_level:                         6
default_album_protection:                  1
album_decoration:                          layers
album_decoration_orientation:              row
allow_username_change:                     1
auto_fix_orientation:                      1
random_album_id:                           starred
use_last_modified_date_when_no_exif_date:  0
layout:                                    justified
date_format_photo_thumb:                   M j, Y, g:i:s A e
date_format_photo_overlay:                 M j, Y, g:i:s A e
date_format_sidebar_uploaded:              M j, Y, g:i:s A e
date_format_sidebar_taken_at:              M j, Y, g:i:s A e
date_format_hero_min_max:                  F Y
date_format_hero_created_at:               M j, Y, g:i:s A T
date_format_album_thumb:                   M Y
upload_chunk_size:                         0
nsfw_banner_blur_backdrop:                 0
search_pagination_limit:                   300
search_minimum_length_required:            4
photo_layout_justified_row_height:         320
photo_layout_masonry_column_width:         300
photo_layout_grid_column_width:            250
photo_layout_square_column_width:          200
photo_layout_gap:                          12
display_thumb_album_overlay:               always
display_thumb_photo_overlay:               hover
default_album_thumb_aspect_ratio:          1/1
use_album_compact_header:                  0
back_button_enabled:                       0
back_button_text:                          Return to Home
SA_random_thumbs:                          0
login_required:                            0
enable_unsorted:                           1
enable_starred:                            1
enable_recent:                             1
enable_on_this_day:                        1
login_required_root_only:                  1
dark_mode_enabled:                         1
show_keybinding_help_popup:                1
show_keybinding_help_button:               1
disable_se_call_for_actions:               0
enable_se_preview:                         0
slideshow_timeout:                         5
keep_original_untouched:                   1
low_quality_image_placeholder:             1
timeline_photos_enabled:                   1
timeline_photos_public:                    0
timeline_photos_granularity:               day
timeline_photos_order:                     taken_at
timeline_photos_layout:                    square
timeline_photos_pagination_limit:          200
timeline_albums_enabled:                   1
timeline_albums_public:                    0
timeline_albums_granularity:               year
timeline_left_border_enabled:              1
timeline_photo_date_format_year:           Y
timeline_photo_date_format_month:          M Y
timeline_photo_date_format_day:            j M Y
timeline_photo_date_format_hour:           g:i
timeline_album_date_format_year:           Y
timeline_album_date_format_month:          M Y
timeline_album_date_format_day:            j M
hide_nsfw_in_smart_albums:                 1
hide_nsfw_in_frame:                        1
hide_nsfw_in_map:                          1
hide_nsfw_in_search:                       1
search_photos_layout:                      square
hide_nsfw_in_rss:                          1
hide_nsfw_in_timeline:                     1
number_albums_per_row_mobile:              3
cache_enabled:                             1
cache_ttl:                                 86400
low_number_of_shoots_per_day:              10
medium_number_of_shoots_per_day:           50
high_number_of_shoots_per_day:             100
maintenance_processing_limit:              5000
disable_recursive_permission_check:        1
exif_disabled_for_all:                     0
file_name_hidden:                          0
client_side_favourite_enabled:             0
default_old_settings:                      0
default_expert_settings:                   0
default_all_settings:                      0
disable_thumb_download:                    1
disable_thumb2x_download:                  1
disable_small_download:                    0
disable_small2x_download:                  0
disable_medium_download:                   0
disable_medium2x_download:                 0
temporary_image_link_enabled:              0
temporary_image_link_when_logged_in:       0
temporary_image_link_when_admin:           0
temporary_image_link_life_in_seconds:      86400
secure_image_link_enabled:                 0
SA_override_visibility:                    0
TA_override_visibility:                    0
enable_colour_extractions:                 0
colour_extraction_driver:                  farzai
home_page_default:                         gallery
flow_enabled:                              1
flow_public:                               0
flow_base:                                 
flow_max_items:                            10
flow_strategy:                             auto
flow_include_sub_albums:                   0
flow_include_photos_from_children:         0
flow_open_album_on_click:                  0
flow_display_open_album_button:            0
flow_highlight_first_picture:              1
flow_min_max_enabled:                      1
flow_display_statistics:                   1
flow_compact_mode_enabled:                 1
flow_image_header_enabled:                 1
flow_image_header_cover:                   cover
flow_image_header_height:                  24
flow_carousel_enabled:                     1
flow_carousel_height:                      6
date_format_flow_published:                M j, Y, g:i:s A e
date_format_flow_min_max:                  F Y
flow_blur_nsfw_enabled:                    1
hide_nsfw_in_flow:                         1
skip_duplicates_early:                     1
sync_delete_missing_photos:                0
sync_delete_missing_albums:                0
sync_dry_run:                              1
gallery_header_enabled:                    0
gallery_header_logged_in_enabled:          0
gallery_header:                            
gallery_header_bar_transparent:            1
gallery_header_bar_gradient:               1
import_via_url_forbidden_localhost:        1
import_via_url_forbidden_local_ip:         1
import_via_url_require_https:              1
import_via_url_forbidden_ports:            1
is_scroll_to_navigate_photos_enabled:      1
is_swipe_vertically_to_go_back_enabled:    1
details_links_enabled:                     0
details_links_public:                      0
photo_thumb_tags_enabled:                  0
hide_nsfw_in_tag_albums:                   1
hide_nsfw_in_tag_listing:                  1
sorting_pinned_albums_col:                 created_at
sorting_pinned_albums_order:               DESC
copyright_text:                            
timeline_page_enabled:                     1
timeline_quick_access_date_format_year:    Y
timeline_quick_access_date_format_month:   M
timeline_quick_access_date_format_day:     j M
timeline_quick_access_date_format_hour:    h M, g:i
watermark_enabled:                         0
watermark_photo_id:                        
watermark_public:                          1
watermark_logged_in_users_enabled:         0
renamer_enabled:                           0
renamer_enforced:                          0
renamer_enforced_before:                   0
renamer_enforced_after:                    0
renamer_photo_title_enabled:               1
renamer_album_title_enabled:               1
extract_zip_on_upload:                     1
webshop_currency:                          EUR
webshop_default_description:               No description provided.
webshop_allow_guest_checkout:              1
webshop_terms_url:                         
webshop_privacy_url:                       
webshop_default_price_cents:               0
webshop_default_license:                   personal
webshop_default_size:                      medium
enable_untagged:                           1
photos_pagination_limit:                   500
deduplicate_pinned_albums:                 0
desktop_dock_full_transparency_enabled:    0
mobile_dock_full_transparency_enabled:     0
import_via_url_block_redirect:             1
enable_smart_album_per_owner:              0
rating_enabled:                            1
rating_public:                             0
rating_show_only_when_user_rated:          0
rating_show_avg_in_details:                1
rating_photo_view_mode:                    hover
rating_show_avg_in_photo_view:             0
rating_album_view_mode:                    hover
rating_show_avg_in_album_view:             1
album_layout:                              grid
albums_per_page:                           30
albums_pagination_ui_mode:                 infinite_scroll
albums_infinite_scroll_threshold:          10
photos_per_page:                           100
photos_pagination_ui_mode:                 infinite_scroll
photos_infinite_scroll_threshold:          10
shared_albums_visibility_default:          show
enable_unrated:                            0
enable_1_star:                             0
enable_2_stars:                            0
enable_3_stars:                            0
enable_4_stars:                            1
enable_5_stars:                            1
enable_best_pictures:                      1
best_pictures_count:                       100

Browser & System [REQUIRED]

Lychee v7.3.2 with MariaDB 10.11.15 in a Docker-Compose setup on Ubuntu 24.04.3

Please confirm (incomplete submissions will not be addressed)

• I have provided easy and step-by-step instructions to reproduce the bug.
• I understand my bug report will be removed if I haven't met the criteria above.
• I understand that if I am requested to provide more information, I must do so within 14 days or the bug report will be closed.

[ildyria]

So my guess is that it failed the first time for some reason and when you tried to rerun it it obviously crashed.
I will make the migration more robust.

[ildyria]

Though since there were permissions errors reading the logs, I wouldn't have been able to trace how much of the migration had been done before this fix.

That's fine, once the PR is merged, you can do a pull on edge tag and that will re-run the migration, giving us more info on what failed.

[rfilmyer]

I'll hold off on trying to fix it manually then.

[ildyria]

edge is available. Assuming you are using the our docker image and not the one from LinuxServer.

[rfilmyer]

Switched to edge and the migration completed successfully.

Now trying to switch back to v7.3.2...

[rfilmyer]

edge -> v7.3.2 downgrade was successful. Thanks for the fix! Will test the other installation later as well.