Replication - Deleted Records in Salesforce Not Being Deleted in Target System

  • 280Views
  • Last Post 22 December 2017
1
votes
API User Reporting posted this 29 November 2017

Hello:

We are using SkyVia to replication from Salesforce to Azure SQL Database.

We have seen repeatedly a problem where deletions done in SalesForce are not reflected in replication.

My suspicion is that delete records are going to the recycle bin and

That the recycle bin is being purged before scheduled replication (incremental) and the delete is 'lost' 

OR

The LastModifiedDate is used to determine whether a record has been updated / deleted, but perhaps LastModifiedDate is not modified when a record is deleted so it is not included in replication.

The SystemModstamp is not replicated in the tasks that make up the package.

The replication package in question is only being run once a day.

This is a critical defect.

Please advise if this has been fixed, and if so, when?

If it has not been fixed, please advise how to properly work around.

Thank you for your attention and consideration.

Respectfully,

Darryll Petrancuri

  • Supported by
  • Jacob Widmer
Order By: Standard | Newest | Votes
0
votes
Mariia Zaharova posted this 30 November 2017

Unfortunately, we cannot reproduce this issue.

 

That the recycle bin is being purged before scheduled replication (incremental) and the delete is 'lost' 

We have checked this case. All deleted records are deleted from target tables even if Recycle Bin was purged before package run.

 

The LastModifiedDate is used to determine whether a record has been updated / deleted, but perhaps LastModifiedDate is not modified when a record is deleted so it is not included in replication.

For tracking deleted records IsDeleted column is used. If table doesn't have this column it is impossible to use Incremental Updates option.

 

Please tell us:

- how much time passes between deletion of records in Salesforce and running the package?

- try deleting one records from any table manually and run the package in a few minutes; what is the result? 

- whether this behaviour occurs only with some concrete table or with different tables?

- whether this behaviour occurs only with scheduled runs?

 

Looking forward to your reply.

0
votes
API User Reporting posted this 01 December 2017

 

I have confirmed there are records in the Recycle Bin that were placed there prior to the latest execution of a replication package that should have deleted the records from the target Azure SQL Database. The records are from a table having an IsDeleted column and the IsDeleted column is included in the replication.

I have also confirmed that there are no enforced foreign key constraints present in the target Azure SQL Database that would cause a delete to fail.

Is there a detailed replication log available from SkyVia that shows the individual records to be replicated. 

The package runs once a day at 1130P ET. 

This is a fatal / critical problem in the use of SkyVia for replication that will cause my client to have to abandon the use of SkyVia if we cannot quickly get acknowledgement, acceptance and resolution of the problem.

Please advise.

0
votes
Mariia Zaharova posted this 04 December 2017

 Thank you for the reply. Please tell us the following details: 

- the package number with which this issue occurs;

- how much time passes between deletion of records in Salesforce and running the package?

- try deleting one records from any table manually and run the package in a few minutes; what is the result? 

- whether this behaviour occurs only with some concrete table or with different tables?

- whether this behaviour occurs only with scheduled runs?

 

Additional note: Please refer to: https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/polling_for_changes.htm#topic-title

It says, that "We recommend polling no more frequently than every five minutes. There are built in controls to prevent errant applications from invoking the data replication API calls too frequently".

This behaviour can be reproduced if the package runs directly after the deletion from Salesforce. In this case, deleted records may be omitted, however, the LastSyncTime parameter will be refreshed and next package run will not treat these records as deleted because LastSyncTime parameter will be more than the time of records deletion.

 

Looking forward to your reply.

0
votes
API User Reporting posted this 04 December 2017

Respectfully, please stop replying with the same reply you gave to others on the same problem.

The package runs once a day at 1130p ET.

We only run scheduled replication.

It is not our job to test these other scenarios.

The bug is real. 

I do not know what you mean by package number. There is no such identifier on the user interface. The package name is: Replicate - Zuora SFDC Objects.

In the specific example I am citing, the record is being deleted from a table in Zuora. Zuora uses a service called Zuora 360 to keep Zuora and Salesforce sync'd. I have confirmed the record deleted from Zuora is placed into the Salesforce Recycle Bin (IsDeleted = True). The object does have a Created / LastModified Date columns and these columns are being properly maintained.

I am confirm the incorrect queries by using querying from the Anonymous Execution window of the Developer Console and including the ALL ROWS modified to the query I am executing to make sure the Recycle Bin is being considered / evaluated.

Furthermore, I can see that there are days where SkyVia is reporting no rows for delete replication for the tasks of a specific package yet I can find rows to be deleted for the time frame in question in the Recycle Bin for the object in question

and

there are other days where SkyVia is reporting n rows for delete replication for the tasks of a specific package and I can find the matching number of rows to be deleted for the time frame in question in the Recycle Bin for the object in question

For what my client is paying per month for Skyvia Replication and the mission critical nature of such for their operations, we require a timely and interactive response that does not seek to place burden of testing on us. The information I provided should be sufficient. I can identify a record that should have been deleted from the target table that was not deleted from the target table.

Please advise.

0
votes
Mariia Zaharova posted this 05 December 2017

We will enable extended logging for this package today and will watch it for a week.

We would be grateful if you'll track the records, which are not deleted while replication within some period of time, and send us the IDs of these records and the approximate time interval when these rows have been deleted in Salesforce.

 

Thank you for using Skyvia and for your assistance.

0
votes
API User Reporting posted this 05 December 2017

Mariia:

 

Thank you for your reply. I will do my best to give you some list of records that should have been deleted from Azure SQL Database and should have been included in the count of records delete as shown on the SkyVia replication history logs.

Run # 9530498 demonstrates this problem as it reports 0 records for replication deletion on any of the tables, yet I can confirm with Apex queries from the Execute Anonymous Code window there are records which are in the Recycle Bin for Zuora__SubscriptionProductCharge__c and requiring deletion that are not being 'detected' by SkyVia replication.

That being said, I'm concerned this is all I received from you in a response. Is there no preliminary feedback from your development team on what could be causing the deleted records sitting in the Recycle Bin to not be included in replication.

I have also confirmed using the Salesforce Data Replication API I am able to get the same records that I do querying using ALL ROWS looking for those records deleted between a particular time period, so I'm at even more of a loss as to how there are SkyVia replication package executions where a task is reporting no rows for delete replication yet the Salesforce Data Replication API clearly establishes there are rows for the corresponding object that have been deleted uring the time period in question.

It would really be useful to know what SkyVia's mechanism is for find records requiring replication.

I don't believe it too much to ask for more feedback at this time. This issue has been reported by other users of SkyVia dating back several months and has gone unaddressed / unresolved. Furthermore, I reported it now about a week ago and your organization has done little to work with me to establish an understanding of what's happening and provide any meaningful feedback. It seems to me it should be reasonably straightforward to determine what possible conditions might be causing the problem to happen.

I await your reply.

Respectfully,

Darryll Petrancuri

 

 

0
votes
API User Reporting posted this 06 December 2017

Mariia:

Please respond.

Respectfully,

Darryll

0
votes
API User Reporting posted this 07 December 2017

Mariia:

I'm at a loss as to how this is acceptable support.

We are a paying customer with a legitimate concern about a defect in SkyVia replication.

If this is not the appropriate forum for support in a timely manner, please tell me what means we should use for such.

Respectfully.

Darrylll

0
votes
Mariia Zaharova posted this 08 December 2017

Thank you for all provided information. We are sorry to find out that you are not satisfied with our product. We are doing our best to find the reason of this issue.

 

As we have said above, we have enabled extended logging for this package and we need some time to watch it.

Unfortunately, we cannot tell you any exact reason or solution for this case for now. Most likely, the issue is related to incorrect processing some records being in the Recycle Bin. However, as we can see from the history of your package, some runs delete records and some not. We need to determine whether Skyvia does not receive these records or Salesforce does not provide these records for deletion and in what exactly cases this behaviour occurs.

 

According to the last package run #9587287, Salesforce returned IDs of deleted records for all the tables except Zuora__CustomerAccount__c.

Is this correct behaviour? Or this table also had some records for deletion in the last package run?

 

0
votes
API User Reporting posted this 11 December 2017

Why is it that you won't address what has already been reported? I've given you table names and package names and run #s. 

I told you it's intermittent behavior and identified a package run # that is incorrect. You still have not provide one measure of meaningful feedback or help.

Perhaps a robust campaign of informing the community via social media, and contacting all parties who have previously given favorable reviews to your product will motivate you to provide some meaningful support and feedback since you have yet to offer anything tangible.

From above:

Run # 9530498 demonstrates this problem as it reports 0 records for replication deletion on any of the tables, yet I can confirm with Apex queries from the Execute Anonymous Code window there are records which are in the Recycle Bin for Zuora__SubscriptionProductCharge__c and requiring deletion that are not being 'detected' by SkyVia replication.

That being said, I'm concerned this is all I received from you in a response. Is there no preliminary feedback from your development team on what could be causing the deleted records sitting in the Recycle Bin to not be included in replication.

I have also confirmed using the Salesforce Data Replication API I am able to get the same records that I do querying using ALL ROWS looking for those records deleted between a particular time period, so I'm at even more of a loss as to how there are SkyVia replication package executions where a task is reporting no rows for delete replication yet the Salesforce Data Replication API clearly establishes there are rows for the corresponding object that have been deleted uring the time period in question.

0
votes
Simon Bubnov posted this 12 December 2017

Hi Darryll,

First of all, I'd like to sincerely apologize that solution of the issue took much more time than you had probably expected. Thank you very much for your patience. 

And thank you for the provided information. Due to the information you provided us with, we made a deep research of your package with extended logging during one week. We succeeded to localize and fix the issue. We will update Skyvia within the next few days and let you know here as soon as the new version with the fix is released.

0
votes
API User Reporting posted this 12 December 2017

In order to restore confidence and allow us to make an informed decision as to whether or not to continue using Devart's products, including SkyVia, we need to know what the problem is and how it's been addressed.

I look forward to your response.

Respectfully,

Darryll Petrancuri

0
votes
Simon Bubnov posted this 13 December 2017

We have updated the "Replicate - Zuora SFDC Objects" package, and the issue should not be reproduced again in this package. We will also update Skyvia soon, and the issue won't be reproduced in any package of any user.

 

> we need to know what the problem is and how it's been addressed.

 

It was a complex issue that is hard to describe. It can be reproduced in a rare specific case. When we enabled extended logging and studied your package with logging, we have found this rare bug, and fixed it.

We are very sorry for the inconveniences and hope that you will have only positive experience with Skyvia in future.

0
votes
API User Reporting posted this 13 December 2017

Simon:

Respectfully, we don't have confidence it was only occuring in the package stated. It is the one we were most easily able to identify and diagnose the problem in.

Whatever the fix, we need to know that it has solved the problem in any SkyVia replication package where deleted records are a possiblity.

With respect to it being a complex issue, only reproducable in rare specific cases, I am having a difficult time believing such as this problem was reported during the summer by another organization and they gave up and moved on from your product, as they were not as persistent or able to provide the proof that I was.

I hope you'll consider that I'm a member of a highly skilled software engineering team and that we have the ability to understand complex problems and bugs. We have to make an informed decision as to whether or not to proceed with your products, and your decision not to share the source of the problem with us will be a key factor in the decision.

Respectfully,

Darryll Petrancuri

 

0
votes
Simon Bubnov posted this 13 December 2017

We've been looking for the issue's reason for quite a long time. We just couldn't replicate it in order to localize the bug. Your specific case helped us to localize the bug. As soon as we localized it, it took us just several hours to solve the issue. 

The issue occurred, when the package run at the same time every day and the package's processing took less than 1 minute. Then the interval for choosing deleted records was calculated wrong and it was shorter than needed. That is why a part of deleted records couldn't get there.

 

Daryll, Skyvia team and I would like to apology that this bug disappointed you. We'd also like to thank you for persistence. At the end of the day, you helped us to finally find the bug, fix it and therefore improve Skyvia. Thank you so much for such an invaluable contribution. In gratitude for that, we'd like to kindly prolong your Professional Data Integration account for an extra week free of charge. 

If you have any ideas with regards to new features you would like to see in the product, please kindly share those ideas with us. We’re flexible and always trying to improve Skyvia so you could have the best user experience possible.

0
votes
API User Reporting posted this 13 December 2017

Simon:

Thank you for your effort to be transparent.

It makes sense the logic for the date range during which to search for deleted records was compromised. However, might I suggest that if you are using the Replication API from Salesforce you would not have these issues because you do not need to search for deleted records seperately. If your product is not built using such, I would strongly encourage you to consider switching to such because it is the only method guaranteed by Salesforce to capture all records requiring replication since it goes through their API and does not rely on querying with ALL ROWS with a SOQL where clause that you must construct as part of your code.

I welcome the opportunity to be of assistance. Furthermore, I think a more significant good faith gesture on your organizations part is in order to help us rationalize our continued use of your product, as these improperly handled deleted records have cost our organization significant time, development resources and credibility to our management team because of the problems they created in our reporting solution which is built upon replicated records in Azure SQL Database,

Perhaps this conversation needs to be taken 'off-line' / private due to the nature of the consideration / gesture you may be motivated to make given the nature of the impact to our team.

Thank you for your further consideration.

Respectfully,

Darryll Petrancuri

dpetrancuri@quenchonline.com

 

0
votes
Simon Bubnov posted this 14 December 2017

Thank you for your consideration and your wish to make our product better. We very much appreciate it. 

Currently we do not use Salesforce Replication API in Skyvia Replication so I sent your idea to our development team. They have already added a task of investigating the possibility to use Salesforce Replication API in Skyvia to our roadmap.

 

We have also updated Skyvia. Now the issue won't reproduce in any replication package.

 

As for one FOC week - have I got you right that you fully support this compensation?

0
votes
API User Reporting posted this 14 December 2017

Simon:

I am not the final decision maker. This will be a group decision. That being said, I don't believe that your offer of a free week is 'adequate'  considering the severity of the problem and it's impact over several months here. That is to say this has been a problem for several months. The root cause what not understood until I got involved and did the analysis, research, etc.

Respectfully,

Darryll Petrancuri

0
votes
Simon Bubnov posted this 15 December 2017

Hello Darryll,

We deeply appreciate your help in finding the reason of the error, and we are grateful to you. As you contacted us two weeks ago, we can offer you to prolong your Professional Data Integration subscription for the same term (two weeks). At the moment we can offer you extending for 2 weeks and we hope that you and your colleagues, who make decisions, find this offer fair.

0
votes
API User Reporting posted this 15 December 2017

Simon:

I will pass the offer along.

Please consider that just because that's when we reported it to you doesn't mean that's the extent of our exposure. You can certainly see a history of packages going back further.

Additonally, I believe you should also consider that others reported this problem as early as the summer, but because they did not have the skill, persistence, perseverance or time to dedicate to proving it was a problem with SkyVia, it went unaddressed, when clearly that should not have been the case.

Respectfully,

Darryll Petrancuri

Show More Posts
Close