Conversions job runs into League\Flysystem\FileNotFoundException in a few cases

[LMCom]

I am finding some failed jobs in failed_jobs that tell me about a League\Flysystem\FileNotFoundException . This happens in less than 1 % of cases, I estimate. Whenever I try to reproduce this bug, there is no Exception.

The code to generate a file and make the conversions:

return $this->addMediaFromString($content)
            ->usingName('logo')
            ->usingFileName('logo.png')
            ->withCustomProperties($customProperties)
            ->toMediaCollection('logo');

While the conversions are done by a queued job, my understanding is that the following part should always happen sequentially:

• The original file is created
• The conversion jobs are pushed to the queue

So the file should always have been created successfully before the conversion starts.

The last method of laravel-medialibrary being called is Spatie\MediaLibrary\MediaCollections\Filesystem->getStream(), then the code of
League\Flysystem takes over and notices that the file does not exist.

public function getStream(Media $media)
    {
        $sourceFile = $this->getMediaDirectory($media).'/'.$media->file_name;

        // $sourceFile does not exist on disk
        return $this->filesystem->disk($media->disk)->readStream($sourceFile);
    }

Based on these observations, my best guess is that the creation of logo.png sometimes fails and laravel-medialibrary fails to notice that and proceeds. That is quite surprising, because \Spatie\MediaLibrary\MediaCollections\FileAdder->toMediaCollection() checks for the existence of the file:

if (! is_file($this->pathToFile)) {
            throw FileDoesNotExist::create($this->pathToFile);
}

However, the Exception FileDoesNotExist is never thrown and the code runs into League\Flysystem\FileNotFoundException when trying to process the file of the Media model.

Edit: If that rare error occurs, the path to the file logo.png is created but the file itself is missing. I find this remarkable.

My code is not manipulating the files in any way or interfering with the process. It's all happening inside the black box of laravel-medialibrary, and I have not been able to ultimately track down the root cause.

Here is the relevant part of the log from failed_jobs:

League\Flysystem\FileNotFoundException: File not found at path: JobLogo/logo/3c163a1feffd7fb-20-0-60-140/logo.png in /var/www/vhosts/examplehost.com/code/vendor/league/flysystem/src/Filesystem.php:390
Stack trace:
#0 /var/www/vhosts/examplehost.com/code/vendor/league/flysystem/src/Filesystem.php(195): League\Flysystem\Filesystem->assertPresent()
#1 /var/www/vhosts/examplehost.com/code/vendor/laravel/framework/src/Illuminate/Filesystem/FilesystemAdapter.php(474): League\Flysystem\Filesystem->readStream()
#2 /var/www/vhosts/examplehost.com/code/vendor/spatie/laravel-medialibrary/src/MediaCollections/Filesystem.php(182): Illuminate\Filesystem\FilesystemAdapter->readStream()
#3 /var/www/vhosts/examplehost.com/code/vendor/spatie/laravel-medialibrary/src/MediaCollections/Filesystem.php(187): Spatie\MediaLibrary\MediaCollections\Filesystem->getStream()
#4 /var/www/vhosts/examplehost.com/code/vendor/spatie/laravel-medialibrary/src/Conversions/FileManipulator.php(54): Spatie\MediaLibrary\MediaCollections\Filesystem->copyFromMediaLibrary()
#5 /var/www/vhosts/examplehost.com/code/vendor/spatie/laravel-medialibrary/src/Conversions/Jobs/PerformConversionsJob.php(38): Spatie\MediaLibrary\Conversions\FileManipulator->performConversions()
#6 /var/www/vhosts/examplehost.com/code/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Spatie\MediaLibrary\Conversions\Jobs\PerformConversionsJob->handle()

[AlirezaManiac]

I'm having the same problem . I use MinIO as my object storage and the file uploads works perfectly fine but confersion jobs always fails because of FileNotFoundException . and I check my storage dashboard and see that the file exists and retry the job but again it cannot find the file although my bucket is set to public ...
can anyone help me with that?

[LMCom]

With remote file systems, there are some specific error conditions. And they are more likely. I am writing about problems on the local file system where the cause seems to be a bug in media library itself. I would rather not confuse issues of remote file systems with my issue.

[francoism90]

@LMCom Did you tried this? https://laravel.com/docs/10.x/queues#jobs-and-database-transactions

Are you sure it fails because the file does not exist (yet) or that the database isn't synced yet?

[LMCom]

Thank you for caring. No, the problem does not seem to be a transaction, I checked that already. Also, that should lead to an \Illuminate\Database\Eloquent\ModelNotFoundException by the Laravel code responsible for unserializing the models.

Also note that I added some new info to my post: "If that rare error occurs, the path to the file logo.png is created but the file itself is missing."

[francoism90]

@LMCom Ok, another thing, what kind of queue driver are you using? I solved my issues by using 'block_for' => null, as redis setting, as for some reason the queue would restart them when still being processed.

Are you sure you're not getting double jobs?