Internal_log_rate_limit=true

[Manniac07]

Please,any help would be appreciated:
So I am running vector on a test server pulling data metrics from a csv file that gets loaded with a different file name every minute. When the file arrives, we transform it and Prometheus remote write it to Victoria metrics. (Note: we have 4 different tomls watching 4 different directories)(also given that it is a test server there will be instances that I will receive new files with different file names but exact same file values)
We set the remove_after_secs to 10, so the directories don't get full. This process will run for about an hour, then will error out with the following error:

2023-09-29T23:47:52.067020Z ERROR source{component_kind="source" component_ id=id component_type=file component_name=name}:file_ser ver:
vector::internal events::file: :sourc
e: Failed in deleting file. file=/ home/user name/directory path/file.csv error=Operation not permitted (os error 1)
error code="deletion failed' error type="command failed" stage="receiving" internal_log_rate_limit=true

[jszwedko]

Hi @Manniac07 ,

It sounds like the vector process doesn't have access to the necessary permissions to delete the file in question.

[Manniac07]

@jszwedko I thought that at 1st as well but then why does it delete every file until it hits that 50 minute mark. Oh and when it starts erroring out if I restart vector it will proceed to delete all files for about 50 more minutes.