MySQL: how to keep track of changes in a view

In my MySQL database, I have a massive view built from a very complex query (+5 million rows, 118 columns), which changes part of its content daily. Every day, I have to query this view and insert its whole content into an even bigger table, this operation can take from 3.5 to 5 hours.

I tried to export the data to CSV files in chunks and then use LOAD DATA INFILE to load the records in a faster fashion, but the export stage is painfully slow.

Is there a way to detect what changed in the view and only update those records in the bigger table? Any recommendations to deal with that amount of records stored in a view?

Thanks

Continue reading...