WhatsApp images have an invalid creation date #9116
Replies: 22 comments 14 replies
-
|
Yeah, this is a bit weird but I don't think we have any reason to support photos from the year 10,000 or beyond. |
Beta Was this translation helpful? Give feedback.
-
|
I guess thinking further we should try to understand why the year on that photo was so wrong, and maybe handle that better? Was the image uploaded from the app? |
Beta Was this translation helpful? Give feedback.
-
|
Maybe #3547? |
Beta Was this translation helpful? Give feedback.
-
Most are forwarded photos and videos. No, It doesn't upload from the app. I used bulk upload docker CLI. |
Beta Was this translation helpful? Give feedback.
-
|
Whatsapp pictures management is shit. In this case date is indicated on filename. For instance |
Beta Was this translation helpful? Give feedback.
-
|
Whatsapp seems to be scrampling the EXIF on the file, so yes, metadata should be ignored if there is any. |
Beta Was this translation helpful? Give feedback.
-
|
Would anyone be kind enough to show the full output of exiftool on a WhatsApp image? |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
The same happens with photos that are shared via Signal: all exif data is gone. |
Beta Was this translation helpful? Give feedback.
-
|
Just for information. Librephotos did a hacky solution for pictures with no metadata https://docs.librephotos.com/docs/user-guide/settings/date-rules It has a list of defined rules to extract dates from different places (standar exif tag, alternate tags, filename regex, etc). User may add some of that rules from a list and may assign different priority to each rule.
|
Beta Was this translation helpful? Give feedback.
-
|
The issue still there, the parsing not works properly or at all. |
Beta Was this translation helpful? Give feedback.
-
|
if I edit an image manualy or using a script, to modify and fix the date for whatsapp images, how can I make immich take the new changes on considiration, I even moved the image to the correct folder structure in my immich library, but it still shows as the old date. |
Beta Was this translation helpful? Give feedback.
-
|
I have just discovered this issue after uploading around 50,000 images to a library. Maybe 20% of them are Whatsapp images that have the dates all wrong! Not I need to somehow bulk delete them from my immich library if anyone has any ideas. |
Beta Was this translation helpful? Give feedback.
-
|
Moving to feature request |
Beta Was this translation helpful? Give feedback.
-
|
I have made this script to perform the extraction and setting the date to |
Beta Was this translation helpful? Give feedback.
-
|
do you think it can be:
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
A batch-edit feature for file dates would solve something like 99% of the issues people describe with bad date metadata. I just looked at the file names of my local signal exports and they follow the pattern signal-YYYY-MM-DDHHmmSS.png. I could export them from signal, move them to my computer, batch fix them with exiftool and reimport them, but fixing it in the app would be a lot simpler. Immich knows how it wants the dates stored, and it already has support for editing dates and for setting a date for multiple files at once. Letting people shoot themselves in the foot using regexps (or something simplified) would be pretty simple. I expect to get some more computer time when my newborn gets a bit older, but I am not counting on it. When I do I can have a look at implementing it if there is any interest in having it mainstream. Even having a guesser would be simple enough. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @bjoli, I'd like to resolve this for you and maybe others. |
Beta Was this translation helpful? Give feedback.
-
|
Signal nukes the metadata. What I am asking for would be a simple batch-edit-date utility that could match against a file name and set the metadata. Say my file is called signal-2023-09-14120005.jpg i could write a match string like "signal-[YYYY]-[MM]-[DD][HH][mm][SS].jpg and have immich batch-set-the-date. Or maybe declare named matches and set them as metadata fields. I can probably write a proof of concept for the web UI, but that will happen first in a couple of months. |
Beta Was this translation helpful? Give feedback.
-
|
At least as I'm seeing it, a given WhatsApp-downloaded image has a proscribed place in my iPhone's timeline. The challenge is, when I use the Immich app to synchronize the image, that place is lost. More, the behavior is undefined: On the Immich side, I can delete the picture and then re-synchronize it with the Immich app, and it will appear in a completely new arbitrary place (sometimes never to be found again). So I suppose the ask for the feature request for me would be, "in the absence of EXIF data, use the date from the device's timeline for the Immich timeline." That way when I take pictures at a birthday party, and my wife takes pictures too and sends them to me by WhatsApp, they'll all be predictably together, like they are on my phone, to be the same in the Immich timeline. In the meantime I found another app that synchronizes them to my server by SFTP with the timeline-based mtime preserved, which also solved the problem in a somewhat messy way. |
Beta Was this translation helpful? Give feedback.
-
|
I have the same problem. |
Beta Was this translation helpful? Give feedback.
-
This was one of the reasons I procured another app, to see if iPhone apps themselves have access to the iPhone Photos timeline dates. Personally I used PhotoSync over SFTP with the "Preserve File Dates" option set to true, and the answer is yes, they do. |
Beta Was this translation helpful? Give feedback.
-
|
This means that it is technically possible to resolve this "bug". |
Beta Was this translation helpful? Give feedback.
-
|
Bless you. I know how to troubleshoot but haven't written code in decades (and wasn't good at it then). |
Beta Was this translation helpful? Give feedback.
-
|
Are Whatsapp borked dates causing disappearing timelines? |
Beta Was this translation helpful? Give feedback.
-
|
Given that this is a generic problem for WhatsApp images, and we don't want to fix the problem at the client (such as suggested by #18243) perhaps we add an optional feature that lets the import handle WhatApp files (identified by filename) both from client, web and import as they are added into the database |
Beta Was this translation helpful? Give feedback.
-
|
I'll have to look again but I believe this is a generic issue with any image that does not contain valid EXIF, including those that might be downloaded from web browsers. Separately, on iOS, WhatsApp apparently does not use the datestamp-filename convention, so anything relying on that alone will fail in a large number of circumstances as well. |
Beta Was this translation helpful? Give feedback.
-
|
On iOS, WhatsApp seems to use a UUID as filename. To retrieve a date for media files without EXIF data on iOS, the native Photos app uses the date the media was saved to the gallery. This date is then used to display the item in the app’s timeline. If the media wasn’t saved on the same day it was taken, the displayed date may be inaccurate. However, the app allows users to manually edit this date to better match their memories. This method works regardless of which app the file was saved from into the Photos app, for both videos and photos. The saved date is accessible through the same API normally used to retrieve photos and videos from the library. Unfortunately, the method isn't perfect, as I don't know if Android implements the same system. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
I'm currently using a script that uses the immich API to get all information about files with WhatsApp filenames and then compares immich's date with the filename. If they differ it overwrites dateTimeOriginal. This worked fine one the first files I tested it with. My code is not very nice and it's Powershell but if anyone is interested I could post it to get the general idea. |
Beta Was this translation helpful? Give feedback.
-
|
I'd be keen to see this ! 🙋♂️ |
Beta Was this translation helpful? Give feedback.
-
|
Yeah same, I'd be interested! |
Beta Was this translation helpful? Give feedback.
-
|
So this is just a quick and dirty script but it should help you get the idea. No parameters from CLI, just change the server address and api key at the beginning of the script. Comment the $changeDateRequest line to do a dry run. Edit: Changed the source for $orgDate from fileCreatedAt to localDateTime, should fix the date difference for timezones other than +0. # WhatsApp_Metadata.ps1
# Changes Immich Metadata for WhatsApp images
#
# Known Bugs:
# - Immich only returns 250 assets
# - Due to no timezone handling in the script some pictures may say "different date", even after changing
# User settings, change these for your instance
$immichpath = "https://immich.example.com"
$apiKey = "apikeyapikey"
# Counters and stuff
$numAssets = 0
$countSameDate = 0
$countDifferentDate = 0
$regexFilenameDate = "(?<year>\d{4})(?<month>\d{2})(?<day>\d{2})"
$regexExifDate = "(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})T(?<hour>\d{2}):(?<minute>\d{2}):(?<second>\d{2})"
# Get Number of Picures/Videos with -WA in name.
# Immich Api seems to be limited to 250 results, maybe also search by year/month when there are houndreds/thousands of files
# Search parameters at https://immich.app/docs/api/search-assets
$requestBody = @"
{
"originalFileName": "-WA"
}
"@
$headers = @{
'Accept' = 'application/json'
'Content-Type' = 'application/json'
'x-api-key' = $apiKey
}
$assetsJSON = Invoke-WebRequest -Uri $immichpath/api/search/metadata -Method Post -Body $requestBody -Headers $headers
$assets = ConvertFrom-JSON $assetsJSON
$numAssets = $assets.assets.count
If ($numAssets -gt 0)
{
Write-Host "Got $numAssets results,"
$assets.assets.items | ForEach-Object {
$assetID = $_.id
$orgFilename = $_.originalFileName
$orgDate = $_.localDateTime
if($orgFilename -match $regexFilenameDate)
{
$fileYear = $Matches.year
$fileMonth = $Matches.month
$fileDay = $Matches.day
$fileDate = $fileYear+$fileMonth+$fileDay
}
if($orgDate -match $regexExifDate)
{
$exifYear = $Matches.year
$exifMonth = $Matches.month
$exifDay = $Matches.day
$exifHour = $Matches.hour
$exifMinute = $Matches.minute # Also get time to write it again with the filename date
$exifSecond = $Matches.second
$exifDate = $exifYear + $exifMonth + $exifDay
}
# Compare dates
if($fileDate -eq $exifDate)
{
# Dates identical, nothing to do, no console logging
$countSameDate += 1
} else {
# Dates different, print details to console
$countMaxFiles += 1
Write-Host $assetId
Write-Host "Filename:`t$orgFilename"
Write-Host "Filename date`t$fileDate"
Write-Host "Exif date`t$exifDate"
Write-Host "Dates different, change Immich metadata...`n"
$countDifferentDate += 1
# Building PUT Request
$date = $fileYear + "-" + $fileMonth + "-" + $fileDay
$time = "T" + $exifHour + ":" + $exifMinute + ":" + $exifSecond + ".000Z"
$dateTime = $date + $time
$changeDateBody = @"
{
"dateTimeOriginal": "$dateTime"
}
"@
$changeDateRequest = Invoke-WebRequest -Uri $immichpath/api/assets/$assetId -Method Put -Body $changeDateBody -Headers $headers
}
}
} else {
Write-Host "No images found."
}
# Statistics
Write-Host "Results:"
Write-Host "$numAssets`tImages found"
Write-Host "$countSameDate`t already had the same date"
Write-Host "$countDifferentDate`thad a differet date" |
Beta Was this translation helpful? Give feedback.
-
|
I've taken the liberty to adjust the above script and added pagination, manual mode, and mismatch is above 1 day. # Changes Immich Metadata for WhatsApp images
#
# Known Bugs:
# - Immich only returns 250 results per page
Add-Type -AssemblyName System.Windows.Forms
# === Configuration ===
$baseUrl = "https://example.immich.com" # Enter Immich URL
$apiKey = "<< API KEY >>" # Enter Immich API key
$pageSize = 200 # Batch size per API call
# === Headers for API requests ===
$headers = @{
"Accept" = "application/json"
"x-api-key" = $apiKey
}
# === Initialize variables ===
$allAssets = @()
$page = 1
Write-Host "`nStarting asset search for filenames containing '-WA'..."
Write-Host "Base URL: $baseUrl"
Write-Host "Page size: $pageSize`n"
do {
Write-Host "Requesting page $page..."
$body = @{
"originalFileName" = "-WA"
"page" = $page
"size" = $pageSize
} | ConvertTo-Json -Depth 3
try {
$response = Invoke-RestMethod -Uri "$baseUrl/api/search/metadata" -Method POST -Headers $headers -Body $body -ContentType "application/json"
} catch {
Write-Error "API call failed on page ${page}: $_"
break
}
$assets = $response.assets.items
$count = $assets.Count
Write-Host " -> Retrieved $count assets on page $page.`n"
if ($count -eq 0) {
Write-Host "No more assets to retrieve. Ending search.`n"
break
}
$allAssets += $assets
$page++
} while ($true)
Write-Host "`nFinished fetching assets."
Write-Host "Total matching assets: $($allAssets.Count)`n"
# === Date Comparison Logic ===
Write-Host "Comparing filename dates with fileCreatedAt (ignoring ≤ 1 day diff)..."
$mismatchedAssets = @()
foreach ($asset in $allAssets) {
$filename = $asset.originalFileName
$createdAt = $asset.fileCreatedAt
if ($filename -match '\b(20\d{2})(\d{2})(\d{2})\b') {
$year = $matches[1]
$month = $matches[2]
$day = $matches[3]
try {
$filenameDate = Get-Date "$year-$month-$day"
$actualDate = Get-Date $createdAt
$dateDiff = [math]::Abs(($filenameDate - $actualDate).TotalDays)
if ($dateDiff -gt 1) {
$mismatchedAssets += [PSCustomObject]@{
id = $asset.id
originalFileName = $filename
fileCreatedAt = $actualDate
filenameDate = $filenameDate
DifferenceInDays = [math]::Round($dateDiff, 2)
}
}
} catch {
Write-Warning "Failed to parse dates for asset: $filename"
}
}
}
Write-Host "`nMismatched assets found (with >1 day diff): $($mismatchedAssets.Count)`n"
$mismatchedAssets | Format-Table -AutoSize
# === Manual or Bulk Fix Options ===
if ($mismatchedAssets.Count -gt 0) {
$answer = Read-Host "Would you like to fix assets? Enter 'one' to fix one manually or 'bulk' to fix all"
if ($answer -eq "one") {
$idToFix = Read-Host "Enter the asset ID to fix"
$asset = $mismatchedAssets | Where-Object { $_.id -eq $idToFix }
if ($asset) {
$newDate = $asset.filenameDate.ToUniversalTime()
$patchUrl = "$baseUrl/api/assets/$($asset.id)"
Write-Host "`n[DEBUG] Preparing to PATCH asset"
Write-Host "[DEBUG] Asset ID : $($asset.id)"
Write-Host "[DEBUG] PATCH URL : $patchUrl"
Write-Host "[DEBUG] New Timestamp: $($newDate.ToString("yyyy-MM-ddTHH:mm:ss.fffZ"))"
$patchBody = @{
dateTimeOriginal = $newDate.ToString("yyyy-MM-ddTHH:mm:ss.fffZ")
} | ConvertTo-Json -Depth 2
Write-Host "[DEBUG] PATCH Body : $patchBody"
try {
$result = Invoke-RestMethod -Uri $patchUrl -Method Put -Headers $headers -Body $patchBody -ContentType "application/json"
Write-Host "Asset $($asset.id) updated to $($newDate.ToShortDateString())"
Write-Host "$result"
} catch {
Write-Warning "`n Failed to update asset:"
Write-Warning $_.Exception.Message
}
} else {
Write-Warning "Asset ID not found."
}
} elseif ($answer -eq "bulk") {
Write-Host "`nStarting bulk update of $($mismatchedAssets.Count) assets..."
foreach ($asset in $mismatchedAssets) {
$newDate = $asset.filenameDate.ToUniversalTime()
$patchUrl = "$baseUrl/api/assets/$($asset.id)"
$patchBody = @{
dateTimeOriginal = $newDate.ToString("yyyy-MM-ddTHH:mm:ss.fffZ")
} | ConvertTo-Json -Depth 2
try {
$result = Invoke-RestMethod -Uri $patchUrl -Method Put -Headers $headers -Body $patchBody -ContentType "application/json"
Write-Host "Updated asset $($asset.id) with date $($newDate.ToString("yyyy-MM-dd"))"
} catch {
Write-Warning "Failed to update asset $($asset.id): $($_.Exception.Message)"
}
}
Write-Host "`n >>>> Bulk update completed."
}
}
|
Beta Was this translation helpful? Give feedback.
-
|
Very much inspired by @Turmoil1112 and with a little help from @copilot, here's a bash script that does a very similar thing for those who prefer bash: #!/bin/env bash
# Requires: bash >= 4.0, jq >= 1.5, curl >= 7.58.0
#
# This script uses the immich API to find images that have WhatsApp-like filenames,
# then extracts dates from filenames and timestamps, compares them, and updates the API
# if mismatches are found. It assumes that the filename is correct enough for your needs,
# and that the timestamp is not.
#
# --- Usage ---
# To run this script from the command line, ensure that it is executable and that you
# have the required environment variables set.
# API_KEY: The key used for authenticating API requests.
# BASE_URL: The base URL of the API.
# > API_KEY=XXXX BASE_URL=http://localhost:2283 ./whatsapp.sh
set -euo pipefail
# Ensure API_KEY and BASE_URL are set
if [[ -z "${API_KEY:-}" ]]; then
echo "Error: API_KEY is not set." >&2
exit 1
fi
if [[ -z "${BASE_URL:-}" ]]; then
echo "Error: BASE_URL is not set." >&2
exit 1
fi
# --- Functions ---
# icurl: A wrapper around curl for making API requests with JSON headers and
# authentication.
icurl() {
local path="$1"
shift
local response
response=$(/usr/bin/curl "$@" -s -w "\n%{http_code}" --header 'Content-Type: application/json' --header 'Accept: application/json' --header "X-Api-Key: ${API_KEY}" "${BASE_URL}/api${path}")
local http_code
http_code=$(echo "$response" | tail -n1)
local body
body=$(echo "$response" | head -n-1)
if [[ $http_code -ge 200 && $http_code -le 299 ]]; then
echo "$body" | /usr/bin/jq
else
echo -e "Error: API request failed with status code $http_code\n$body" >&2
exit 1
fi
}
# fetch_metadata: Fetches metadata from the API for a given page.
fetch_metadata() {
local page="$1"
icurl /search/metadata -d '{"page":'"$page"',"withExif":true,"isVisible":true,"language":"en-GB","originalFileName":"-WA"}'
}
# Initialize counters
inspected_count=0
changed_count=0
# process_asset: Processes an asset by comparing and updating date information if
# necessary.
process_asset() {
local asset_id="$1"
local file_name="$2"
local timestamp="$3"
# Increment inspected count
inspected_count=$((inspected_count + 1))
# Extract date from timestamp
local date_from_timestamp
date_from_timestamp=$(echo "$timestamp" | cut -c 1-10 | tr -d '-')
# Extract date from filename
local date_from_filename=""
if [[ "$file_name" =~ ([0-9]{8})-WA ]]; then
date_from_filename="${BASH_REMATCH[1]}"
fi
# Compare and update if necessary
if [[ -n "$date_from_filename" && "$date_from_timestamp" != "$date_from_filename" ]]; then
local time_part
time_part=$(echo "$timestamp" | cut -d'T' -f2-)
local year=${date_from_filename:0:4}
local month=${date_from_filename:4:2}
local day=${date_from_filename:6:2}
local new_date_part="${year}-${month}-${day}"
local new_timestamp="${new_date_part}T${time_part}"
local new_date_time_original
new_date_time_original=$(icurl /assets/"${asset_id}" -X PUT -d '{"dateTimeOriginal":"'"${new_timestamp}"'"}' | jq -r '.exifInfo.dateTimeOriginal')
echo "Asset: $asset_id, $file_name $timestamp -> $new_date_time_original"
# Increment changed count
changed_count=$((changed_count + 1))
fi
}
# --- Pagination ---
# PAGE: Tracks the current page of results.
# CONTINUE: Controls whether to continue fetching pages.
PAGE=1
CONTINUE=true
while [[ "$CONTINUE" == true ]]; do
# Fetch metadata from the API.
RESULT=$(fetch_metadata "$PAGE")
if [[ $(echo "$RESULT" | jq '.assets.items | length') -eq 0 ]]; then
CONTINUE=false
else
while IFS=$'\t' read -r asset_id file_name timestamp; do
process_asset "$asset_id" "$file_name" "$timestamp"
done <<< "$(echo "$RESULT" | jq -r '.assets.items[] | [.id, .originalFileName, .localDateTime] | @tsv')"
PAGE=$((PAGE + 1))
fi
done
# Print summary
echo "Summary: $inspected_count assets inspected, $changed_count dates changed."example: $ API_KEY=MY_API_KEY BASE_URL=http://localhost:2283 ./whatsapp.sh
Asset: 4eea028f-4a70-406e-9c15-7d98dfed4081, IMG-20211203-WA0007.jpg 2018-12-03T11:05:00.000Z -> 2021-12-03T11:05:00+00:00
Asset: 8d01cd61-eee9-4a30-8400-b515c6745e1a, IMG-20170615-WA0007.jpg 2017-06-12T20:39:00.000Z -> 2017-06-15T20:39:00+00:00
Summary: 1025 assets inspected, 2 dates changed. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
The bug
[BUG] I got some forwarded photos and videos in Whatsapp and I see that the year of these media are more than 4 digits. So It'll not appear in the timeline. I attached photos here.
The OS that Immich Server is running on
Debian
Version of Immich Server
v1.70
Version of Immich Mobile App
v1.71
Platform with the issue
Your docker-compose.yml content
Same as v1.70 docker-compose.ymlYour .env content
Reproduction steps
1. try to upload photos with year more than 4 digits in that like 123456Additional information
No response
Beta Was this translation helpful? Give feedback.
All reactions