You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When reading files from SFTP Server, failure in component execution do lead to retry
Steps to reproduce
Upload 10+ files of 50MB to SFTP folder
Build an integration flow with a single SFTP trigger reading from the folder
Launch it
Expected result
Flow suspended, however number of files in .elasticio_processed directory equal to the number of successfully generated messages (seen as green in log)
Actual result
Flow suspended however .elasticio_processed folder has more files than successful messages in the log
Reasoning behind
SFTP component first moves the file to processed directory and after that reads the file, this behavior is beneficial from concurrency perspective when multiple pollers poll the same folder, however in case of container failures (e.g. OOM) file is not rep-processed / re-uploaded again.
The text was updated successfully, but these errors were encountered:
When reading files from SFTP Server, failure in component execution do lead to retry
Steps to reproduce
Expected result
.elasticio_processed
directory equal to the number of successfully generated messages (seen as green in log)Actual result
.elasticio_processed
folder has more files than successful messages in the logReasoning behind
SFTP component first moves the file to processed directory and after that reads the file, this behavior is beneficial from concurrency perspective when multiple pollers poll the same folder, however in case of container failures (e.g. OOM) file is not rep-processed / re-uploaded again.
The text was updated successfully, but these errors were encountered: