Skip to content

Commit ce6e398

Browse files
authored
GH-10372: Fix the ignoring logic for tmp files (#10373)
Fixes: #10372 The `AbstractInboundFileSynchronizingMessageSource` uses a `CompositeFileListFilter` for a combination of the user-provided filter and then the one to ignore temporary files. However, the `CompositeFileListFilter` logic is `OR`, which means the temporary file might be accepted by the user-provided. * Fix the `AbstractInboundFileSynchronizingMessageSource.buildFilter()` to use a `ChainFileListFilter` instead (with an `AND` logic), and put `ignoreTemporaryFiles` filter as the first one. * Fix (S)FTP docs for an actual logic regarding local files filtering. **Cherry-pick to `6.5.x`**
1 parent b4ba5df commit ce6e398

File tree

3 files changed

+13
-16
lines changed

3 files changed

+13
-16
lines changed

spring-integration-file/src/main/java/org/springframework/integration/file/remote/synchronizer/AbstractInboundFileSynchronizingMessageSource.java

Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@
2020
import java.io.FileNotFoundException;
2121
import java.io.IOException;
2222
import java.net.URI;
23-
import java.util.Arrays;
2423
import java.util.Comparator;
24+
import java.util.List;
2525
import java.util.regex.Pattern;
2626

2727
import org.jspecify.annotations.Nullable;
@@ -33,7 +33,7 @@
3333
import org.springframework.integration.file.DirectoryScanner;
3434
import org.springframework.integration.file.FileHeaders;
3535
import org.springframework.integration.file.FileReadingMessageSource;
36-
import org.springframework.integration.file.filters.CompositeFileListFilter;
36+
import org.springframework.integration.file.filters.ChainFileListFilter;
3737
import org.springframework.integration.file.filters.FileListFilter;
3838
import org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter;
3939
import org.springframework.integration.file.filters.RegexPatternFileListFilter;
@@ -51,9 +51,7 @@
5151
* from an 'inbound' adapter).
5252
* <p>
5353
* The base class supports configuration of whether the remote file system and
54-
* local file system's directories should be created on start (what 'creating a
55-
* directory' means to the specific adapter is of course implementation
56-
* specific).
54+
* local file system's directories should be created on start.
5755
* <p>
5856
* This class is to be used as a pair with an implementation of
5957
* {@link AbstractInboundFileSynchronizer}. The synchronizer must
@@ -94,7 +92,8 @@ public abstract class AbstractInboundFileSynchronizingMessageSource<F>
9492
@SuppressWarnings("NullAway.Init")
9593
private File localDirectory;
9694

97-
private @Nullable FileListFilter<File> localFileListFilter;
95+
@SuppressWarnings("NullAway.Init")
96+
private FileListFilter<File> localFileListFilter;
9897

9998
/**
10099
* Whether the {@link DirectoryScanner} was explicitly set.
@@ -196,9 +195,7 @@ protected void onInit() {
196195
this.fileSource.setDirectory(this.localDirectory);
197196
initFiltersAndScanner();
198197
BeanFactory beanFactory = getBeanFactory();
199-
if (beanFactory != null) {
200-
this.fileSource.setBeanFactory(beanFactory);
201-
}
198+
this.fileSource.setBeanFactory(beanFactory);
202199
this.fileSource.afterPropertiesSet();
203200
this.synchronizer.afterPropertiesSet();
204201
}
@@ -263,7 +260,7 @@ public boolean isRunning() {
263260
* @param maxFetchSize the maximum files to fetch.
264261
*/
265262
@Override
266-
public @Nullable final AbstractIntegrationMessageBuilder<File> doReceive(int maxFetchSize) {
263+
public final @Nullable AbstractIntegrationMessageBuilder<File> doReceive(int maxFetchSize) {
267264
AbstractIntegrationMessageBuilder<File> messageBuilder = this.fileSource.doReceive();
268265
if (messageBuilder == null) {
269266
this.synchronizer.synchronizeToLocalDirectory(this.localDirectory, maxFetchSize);
@@ -284,9 +281,9 @@ public boolean isRunning() {
284281
}
285282

286283
private FileListFilter<File> buildFilter() {
287-
Pattern completePattern = Pattern.compile("^.*(?<!" + this.synchronizer.getTemporaryFileSuffix() + ")$");
288-
return new CompositeFileListFilter<>(
289-
Arrays.asList(this.localFileListFilter, new RegexPatternFileListFilter(completePattern)));
284+
var ignoreTemporaryFilesPattern = Pattern.compile("^.*(?<!" + this.synchronizer.getTemporaryFileSuffix() + ")$");
285+
return new ChainFileListFilter<>(
286+
List.of(new RegexPatternFileListFilter(ignoreTemporaryFilesPattern), this.localFileListFilter));
290287
}
291288

292289
/**

src/reference/antora/modules/ROOT/pages/ftp/inbound.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ metadata store on every update (if the store implements `Flushable`).
8787

8888
IMPORTANT: Further, if you use a distributed `MetadataStore` (such as xref:redis.adoc#redis-metadata-store[Redis]), you can have multiple instances of the same adapter or application and be sure that each file is processed only once.
8989

90-
The actual local filter is a `CompositeFileListFilter` that contains the supplied filter and a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`).
90+
The actual local filter is a `ChainFileListFilter` that contains a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`) and the supplied filter.
9191
Files are downloaded with this suffix (the default is `.writing`), and the file is renamed to its final name when the transfer is complete, making it 'visible' to the filter.
9292

9393
The `remote-file-separator` attribute lets you configure a file separator character to use if the default '/' is not applicable for your particular environment.

src/reference/antora/modules/ROOT/pages/sftp/inbound.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ You can handle any other use-cases by using `CompositeFileListFilter` (or `Chain
6969

7070
The above discussion refers to filtering the files before retrieving them.
7171
Once the files have been retrieved, an additional filter is applied to the files on the file system.
72-
By default, this is an`AcceptOnceFileListFilter`, which, as discussed in this section, retains state in memory and does not consider the file's modified time.
72+
By default, this is an `AcceptOnceFileListFilter`, which, as discussed in this section, retains state in memory and does not consider the file's modified time.
7373
Unless your application removes files after processing, the adapter re-processes the files on disk by default after an application restarts.
7474

7575
Also, if you configure the `filter` to use a `SftpPersistentAcceptOnceFileListFilter` and the remote file timestamp changes (causing it to be re-fetched), the default local filter does not allow this new file to be processed.
@@ -86,7 +86,7 @@ metadata store on every update (if the store implements `Flushable`).
8686

8787
IMPORTANT: Further, if you use a distributed `MetadataStore` (such as xref:redis.adoc#redis-metadata-store[Redis Metadata Store]), you can have multiple instances of the same adapter or application and be sure that one and only one instance processes a file.
8888

89-
The actual local filter is a `CompositeFileListFilter` that contains the supplied filter and a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`).
89+
The actual local filter is a `ChainFileListFilter` that contains a pattern filter that prevents processing files that are in the process of being downloaded (based on the `temporary-file-suffix`) and the supplied filter.
9090
Files are downloaded with this suffix (the default is `.writing`), and the files are renamed to their final names when the transfer is complete, making them 'visible' to the filter.
9191

9292
See the https://github.com/spring-projects/spring-integration/tree/main/spring-integration-core/src/main/resources/org/springframework/integration/config[schema] for more detail on these attributes.

0 commit comments

Comments
 (0)