Tuesday, May 27, 2014

Process all files, even when single file is corrupt


Usually in ETL, we get requirements to process all files from a particular folder and load them to database. If one of the file is corrupt, we will be asked to move the file to Reject Folder and send an email. But continue to process other files from the source folder.




Default SSIS behavior is such that, even if a single error occurs, the package execution is stopped.
So, to continue even if a single file is failed to process, below steps needs to be followed:
Go to Event Handler Section for that Data Flow Task
Go to Variables Section and select system variables. Select the variable Propogate and set is FALSE
Exception:
Some times for excel file, we may have to do some additional settings along with the above Propogate
Either of the below approach will work fine
·         Set the property, validate metadata at the source to false
·         Set the MaxErrorCount in ForEachLoop to 1000

No comments:

Post a Comment