duplicate records destination Snowflake

  • 41Views
  • Last Post 30 March 2022
0
votes
Gabriel Longarai posted this 28 March 2022

Can you check why duplicate records are being loaded? Before when the destination was PostgreSQL this did not happen, but in Snowflake this problem happens. I tried with the "Direct Id Check" option and this problem also occurred. 

Run Id 86406121

https://app.skyvia.com/#/76069/packages/156225

Attached Files

Order By: Standard | Newest | Votes
0
votes
Mariia Zaharova posted this 29 March 2022

Hello Gabriel,

 

Thank you for contacting Skyvia Support.

 

According to the logs of this package, it was changed and the LastSyncTime parameter was reset on March 28. 

This led to performing full replication and creating duplicates. Please note, if you make any changes to the package, you will be asked to reset or not reset the LastSyncTime parameter on saving it. We recommend you look at this topic in our documentation.

 

In order to resolve the issue with duplicates, we recommend performing a full replication once again but with the Drop Tables checkbox selected in addition to Create Tables and Incremental Updates. In this case, target tables will be recreated and the data will be recopied without duplicates.

 

Best regards,

Mariia

0
votes
Gabriel Longarai posted this 29 March 2022

Is there any safe way to not duplicate records when resetting lastSync?

0
votes
Mariia Zaharova posted this 30 March 2022

Hello Gabriel,

The recommendation, in this case, is to use the Drop Tables and Create Tables checkboxes. In this case, target tables will be recreated and the data will be recopied without duplicates. Otherway, duplicates may be created.

 

Best regards,

Mariia

Close