Greetings:
We need a detailed explanation of what happens when a specific execution of a replication package fails?
We need to understand what kind of fault tolerance / recoverability / etc. mechanism exists in the event of a failure?
This is a critical need so a timely response greatly appreciated.
Respectfully,
Darryll Petrancuri
If possible, please specify these details:
- specify numbers of the packages with which this behavior occurred for the last time;
- approximate time period when this behaviour occurred.
Also, we can enable ID logging of all processed records in some of your packages (where this behaviour occur more frequently). In this case, we will be able to get all IDs and compare the results with the database copy. Note: this may increase number of used API calls. If it is not a problem for you, we will do this. Just let us know.
Looking forward to your reply.
Best regards,
Mariia
Mariia:
Thank you for your reply. Neither of these circumstances apply. The change was not processed by the next package run and we are replicating the entire object. As you can imagine, we are very concerned with such.
At this time it is very difficult for us to provide more details but candidly, this lack of proper replication has lead my client to begin aggressively looking for an alternative to SkyVia.
I wish we could provide more details, but at this time we cannot. We 'assume' that our replication solution simply will not allow such to occur and as a result are not actively auditing and comparing the replication target to the replication source. It's only when we have data anomalies that reveal themselves in reporting scenarios that we then are able to find records where it can be clearly established target (which is static and has no data modifications performed other than by SkyVia replication) record(s) do(es) not match the source.
We would be grateful for any further insight / assistance you might be able to provide. Absent such, it is likely my client will need to move to a different solution.
Respectfully,
Darryll Petrancuri
However, we have seen cases where a record has been updated (LastModifiedDate) has been changed, yet the record has not been replicated.
This behavior may occur in the following cases:
1. Changes in the source table were made at the time when the package was run. LastModifiedDate was changed, but the record was not processed. It will be processed by the next package run.
2. You are replicating not the whole object (not all fields are included). Or you have added some fields to your Salesforce object and did not update the task. The field that is not selected in the package was changed, the LastModifiedDate value changes, update is performing, but the line remains the same. A really changed value is not involved in the package.
We would be grateful if you provide us with some additional details that you might have noticed regarding this behavior.
Best regards,
Mariia
Mariia:
Thank you for your response. I am not confident at present in your fault tolerance. I appreciate you providing the response regarding the conditions under which the LastSyncTime parameter is updated. However, we have seen cases where a record has been updated (LastModifiedDate) has been changed, yet the record has not been replicated.
I'm not quite sure how to go about proving such, but I can state unequivocally wee have seen this behavior.
Please advise.
Respectfully,
Darryll Petrancuri
Sorry for providing incomplete information for your question.
When Replication package fails, the LastSyncTime parameter is not updated and, thus, all the records (records that were processed in the failed run and records that were not processed) will be processed again with the next package run. The LastSyncTime parameter is updated only in case if package succeeds. Each next package run processes all the records that were changed since the last package run (LastSyncTime value).
Please note, we have updated Skyvia yesterday. Now, if replication package fails, successfully processed records are not counted.
As for response time, we are doing our best to process requests as soon as possible.
Thank you for your patience and thank you for using Skyvia!
Best regards,
Mariia
Mariia:
I must express my extreme disappointment with the lack of thought given to providing COMPLETE answers to the questions posed. I must also express my frustration with the fact that responses are hours in coming.
The lack of throughness and timeless are quite disturbing and serve to reinforce an already gravely undermined trust in DevArt's SkyVia components.
Disheartened,
Darryll Petrancuri
Mariia:
Thank you for your response, albeit an incomplete one. What happens to the the records that were not processed? You'll forgive me, but it truly seems like fault tolerance was not given due consideration and that in it's current state it's difficult to trust the 'data integrity' in target environments.
Comments from your software engineering team would be most appreciated.
Respectfully,
Darryll Petrancuri
Currently, if replication package fails, all successfully processed records are counted and are not rolled back. However, we are working on this behaviour so that these records are not counted. We will inform you when any results regarding this are available.
If you have any further questions, feel free to contact us.
Best regards,
Mariia
