Import package extremely slow

  • 67Views
  • Last Post 10 December 2021
0
votes
Peter Blonde posted this 25 November 2021

We have 2 scheduled packages in Skyvia. Both packages contain the same sort data/rows/numbers, but 1 import takes 30 minutes and the other one 5 hours. Can it be explained how this is possible?

Order By: Standard | Newest | Votes
0
votes
Nataliia Nikulina posted this 26 November 2021

Hello Peter,

 

Thank you for contacting Skyvia Support team. 

 

Please send us the links to both packages so we continue to investigate the issue.

We look forward to hearing from you.

 

Best regards,

Nataliia,

Customer Support Engineer

0
votes
Peter Blonde posted this 26 November 2021

Hi Nataliia,

 

the links are:

https://app.skyvia.com/#/34138/packages/61663

https://app.skyvia.com/#/34138/packages/61639

 

if you need more information, please let me know.

 

regards Peter Blondé

 

0
votes
Nataliia Nikulina posted this 26 November 2021

Hello Peter,

 

Thank you for contacting Skyvia Support team.

 

Please select a 'Use new runtime' checkbox on the tab bar of both packages and save the changes.

After that you can launch both packages once again.

This will help us to monitor the package performance results better. 

 

Kindly update us about the results. We look forward to hearing from you.

 

Best regards,

Nataliia,

Customer Support Engineer

0
votes
Peter Blonde posted this 02 December 2021

Hello Nataliia,

 

I've lauched the packages again with the "Use new runtime". Result: de duration is far more slower. I hope you can analyse the results to see what the problem is.

 

Regards Peter Blondé

0
votes
Nataliia Nikulina posted this 06 December 2021

Hello Peter,

Thank you for the update.

Please specify what are Ids of the runs that were launched on the New runtime as having checked the logs of 

several runs, I can see that both packages were launched on the Old runtime. 

Currently, New runtime is also not used in the packages.

Note that logs on the New runtime are more detailed so this can help us to better define the causes of a bad package performance.

We look forward to hearing from you.

 

Best regards,

Nataliia,

Customer Support Engineer

 

0
votes
Peter Blonde posted this 08 December 2021

Hi Nataliia,

 

the correct links to the packages are:

https://app.skyvia.com/#/34138/packages/148391

https://app.skyvia.com/#/34138/packages/148395

 

both are launched on new runtime. I hope you can define the bad performance.

 

best regards, Peter

 

0
votes
Nataliia Nikulina posted this 10 December 2021

Hello Peter, 

Thank you for the update. 

We looked at the package's settings and the execution time is rather expectable.

First of all, there are a lot of records in the package. Second of all, the UPSERT operation in the package is quite resource-demanding and may take a certain time to perform. 

When the cached lookup is used, the necessary fields from all rows of the lookup object are queried and cached on Skyvia, and lookup is performed against this cache. Depending on the number of rows in the lookup object and the number of imported rows, this may provide a performance gain.

Please enable this option for all the lookups, run the package and inform us about the result. 

We look forward to your reply. 

 

Best regards,

Nataliia,

Customer Support Engineer 

Close