Skip to content

Commit 6e0a8b4

Browse files
artembilancppwfs
authored andcommitted
GH-10372: Fix the ignoring logic for tmp files (#10373)
Fixes: #10372 The `AbstractInboundFileSynchronizingMessageSource` uses a `CompositeFileListFilter` for a combination of the user-provided filter and then the one to ignore temporary files. However, the `CompositeFileListFilter` logic is `OR`, which means the temporary file might be accepted by the user-provided. * Fix the `AbstractInboundFileSynchronizingMessageSource.buildFilter()` to use a `ChainFileListFilter` instead (with an `AND` logic), and put `ignoreTemporaryFiles` filter as the first one. * Fix (S)FTP docs for an actual logic regarding local files filtering. **Cherry-pick to `6.5.x`**
1 parent 50d17d4 commit 6e0a8b4

File tree

3 files changed

+11
-15
lines changed

3 files changed

+11
-15
lines changed

spring-integration-file/src/main/java/org/springframework/integration/file/remote/synchronizer/AbstractInboundFileSynchronizingMessageSource.java

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@
2020
import java.io.FileNotFoundException;
2121
import java.io.IOException;
2222
import java.net.URI;
23-
import java.util.Arrays;
2423
import java.util.Comparator;
24+
import java.util.List;
2525
import java.util.regex.Pattern;
2626

2727
import org.springframework.beans.factory.BeanFactory;
@@ -31,7 +31,7 @@
3131
import org.springframework.integration.file.DirectoryScanner;
3232
import org.springframework.integration.file.FileHeaders;
3333
import org.springframework.integration.file.FileReadingMessageSource;
34-
import org.springframework.integration.file.filters.CompositeFileListFilter;
34+
import org.springframework.integration.file.filters.ChainFileListFilter;
3535
import org.springframework.integration.file.filters.FileListFilter;
3636
import org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter;
3737
import org.springframework.integration.file.filters.RegexPatternFileListFilter;
@@ -49,9 +49,7 @@
4949
* from an 'inbound' adapter).
5050
* <p>
5151
* The base class supports configuration of whether the remote file system and
52-
* local file system's directories should be created on start (what 'creating a
53-
* directory' means to the specific adapter is of course implementation
54-
* specific).
52+
* local file system's directories should be created on start.
5553
* <p>
5654
* This class is to be used as a pair with an implementation of
5755
* {@link AbstractInboundFileSynchronizer}. The synchronizer must
@@ -193,9 +191,7 @@ protected void onInit() {
193191
this.fileSource.setDirectory(this.localDirectory);
194192
initFiltersAndScanner();
195193
BeanFactory beanFactory = getBeanFactory();
196-
if (beanFactory != null) {
197-
this.fileSource.setBeanFactory(beanFactory);
198-
}
194+
this.fileSource.setBeanFactory(beanFactory);
199195
this.fileSource.afterPropertiesSet();
200196
this.synchronizer.afterPropertiesSet();
201197
}
@@ -281,9 +277,9 @@ public final AbstractIntegrationMessageBuilder<File> doReceive(int maxFetchSize)
281277
}
282278

283279
private FileListFilter<File> buildFilter() {
284-
Pattern completePattern = Pattern.compile("^.*(?<!" + this.synchronizer.getTemporaryFileSuffix() + ")$");
285-
return new CompositeFileListFilter<>(
286-
Arrays.asList(this.localFileListFilter, new RegexPatternFileListFilter(completePattern)));
280+
var ignoreTemporaryFilesPattern = Pattern.compile("^.*(?<!" + this.synchronizer.getTemporaryFileSuffix() + ")$");
281+
return new ChainFileListFilter<>(
282+
List.of(new RegexPatternFileListFilter(ignoreTemporaryFilesPattern), this.localFileListFilter));
287283
}
288284

289285
/**

src/reference/antora/modules/ROOT/pages/ftp/inbound.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ metadata store on every update (if the store implements `Flushable`).
8787

8888
IMPORTANT: Further, if you use a distributed `MetadataStore` (such as xref:redis.adoc#redis-metadata-store[Redis]), you can have multiple instances of the same adapter or application and be sure that each file is processed only once.
8989

90-
The actual local filter is a `CompositeFileListFilter` that contains the supplied filter and a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`).
90+
The actual local filter is a `ChainFileListFilter` that contains a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`) and the supplied filter.
9191
Files are downloaded with this suffix (the default is `.writing`), and the file is renamed to its final name when the transfer is complete, making it 'visible' to the filter.
9292

9393
The `remote-file-separator` attribute lets you configure a file separator character to use if the default '/' is not applicable for your particular environment.

src/reference/antora/modules/ROOT/pages/sftp/inbound.adoc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -69,8 +69,8 @@ You can handle any other use-cases by using `CompositeFileListFilter` (or `Chain
6969

7070
The above discussion refers to filtering the files before retrieving them.
7171
Once the files have been retrieved, an additional filter is applied to the files on the file system.
72-
By default, this is an`AcceptOnceFileListFilter`, which, as discussed in this section, retains state in memory and does not consider the file's modified time.
73-
Unless your application removes files after processing, the adapter re-processes the files on disk by default after an application restart.
72+
By default, this is an `AcceptOnceFileListFilter`, which, as discussed in this section, retains state in memory and does not consider the file's modified time.
73+
Unless your application removes files after processing, the adapter re-processes the files on disk by default after an application restarts.
7474

7575
Also, if you configure the `filter` to use a `SftpPersistentAcceptOnceFileListFilter` and the remote file timestamp changes (causing it to be re-fetched), the default local filter does not allow this new file to be processed.
7676

@@ -86,7 +86,7 @@ metadata store on every update (if the store implements `Flushable`).
8686

8787
IMPORTANT: Further, if you use a distributed `MetadataStore` (such as xref:redis.adoc#redis-metadata-store[Redis Metadata Store]), you can have multiple instances of the same adapter or application and be sure that one and only one instance processes a file.
8888

89-
The actual local filter is a `CompositeFileListFilter` that contains the supplied filter and a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`).
89+
The actual local filter is a `ChainFileListFilter` that contains a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`) and the supplied filter.
9090
Files are downloaded with this suffix (the default is `.writing`), and the files are renamed to their final names when the transfer is complete, making them 'visible' to the filter.
9191

9292
See the https://github.com/spring-projects/spring-integration/tree/main/spring-integration-core/src/main/resources/org/springframework/integration/config[schema] for more detail on these attributes.

0 commit comments

Comments
 (0)