AnsweredAssumed Answered

How to trigger a transformation only if table has been updated?

Question asked by Alina Lot on Jun 20, 2018
Latest reply on Jun 21, 2018 by Joao Figueiredo


This might be a simple question so apologies in advance.

I have a job I only want to run if the data in the source table has been updated I thought about checking the latest update date in the source file or comparing counts but I am not sure which combination of actions to us to get this to work.

My ideal flow will be:

1. select max('update_date) from table (This i can achieve by a sql query with Table Input)

2. Check id now() == max('update_date) -

if Yes run the Job, if not Do nothing

So i guess my main question is how do I perform this data matching condition?

If I can then only add the new data to the existing table instead of pulling all the available data that will be great as well.


Any advice here will be greatly appreciated!