CLAIMATE

Technical blog

SFTP adapter - how to handle errors within processing a file

created by DALL-E

Jiri Fridrich

Sep 4, 2024

Integration

There are some errors while saving file to SFTP server, which happen after the SFTP connection has been establish. The in-built retry function is not applied, as it is specified in the SAP Help documentation.

'Maximum Reconnect Attempts' setting is only relevant for establishing the initial connection to the server. If the server connection is interrupted during message processing, the connection will not be recovered. A retry attempt of the interrupted message processing can only be achieved by explicitly modeling this functionality via integration flow. 

For instance, the error I needed to handle was: 
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot store file

In such case the integration processing fails. We can handle the error in exception subprocess, but if we want to apply a retry mechanism, we can't really do that within exception subprocess.  

I suggest following solution: we decouple main process from the process of saving the file via SFTP. Let's say we receive a file via HTTPS and we want to save it to SFTP.  The simplified main iflow will look as follows:

  1. We set up a Looping process call, which will be our retry mechanism. 

  2. We base the loop on a header parameter 'file_saved_ok'. As long as the value is 'false', the loop continues. 
     

    jirifridrich_2-1725294185260.png


    After 5 attempts, an exception is raised and is handled in Exception subprocess, which can be for example an alert email.

  3. Local integration process contains only Process direct call to our proprietary iflow, which actually saves the file to SFTP server.

And this is our new integration flow, which manages saving the file to SFTP server.

  1. Iflow is called by the Process direct, which we set in our main calling iflow.

  2. We set the file name and waiting time (see step 4).

  3. We attempt to save the file. If everything went well, the process continues to Content modifier step, where we set header parameter 'file_saved_ok' to 'true'. It is important to set it as header and not property, as property lives only within one iflow.
    Processing is then returned to calling iflow, where the header parameter causes the loop to end and the processing finishes successfully.

  4. If an error occurs, the iflow jumps into Exception subprocess, where header parameter 'file_saved_ok' is set to 'false'. Then we've got a groovy script, which triggers a sleep for a specified waiting time (see step 2). The processing returns to calling iflow, where the header parameter causes the loop to continue.

This repeats till the file is saved successfully or the number of attempts is depleated. In that case we receive an alert email, stating that even after 5 attempts the file was not saved.

Even though simplified, the above example demonstrates the principles by which we handled an SFTP error and implemented a custom retry mechanism.

It was also published here:

https://community.sap.com/t5/technology-blogs-by-members/sftp-adapter-how-to-handle-errors-within-processing-a-file/ba-p/13851788

Navigation

© 2023 CLAIMATE

Navigation

© 2023 CLAIMATE