Batch Size = 1. Bulk API limits overran.

  • 45Views
  • Last Post 16 September 2019
0
votes
Claudia NEUMUELLER posted this 09 September 2019

Dear Support,

we got the problem today, that we ran over bulk API limits (10.000) on PROD environment after starting the skvia package https://app.skyvia.com/#/packages/78591

I checked the logs on environment and realizes that the job has produced thousends of batches because the batch size of each was 1 or 2 records.

What could be the reason for that?

It's very critical as we blocked the PROD bulk PI completelly with this actions.

Please advice.

 

Attached Files

Order By: Standard | Newest | Votes
0
votes
Claudia NEUMUELLER posted this 09 September 2019

one more attachment

0
votes
Dmitriy Muzurov posted this 10 September 2019

Hi Claudia,

Thank you for submitting a ticket with us.

There are 9 packages in your account that use the same Salesforce connection.

We suppose you reached API limit not with the package ID 78591 but with the total requests made to your Salesforce org by all other packages.

More details about Salesforce API calls usage can be found here.

Please consider decreasing the number of packages.

We are looking forward to hearing from you, should you have any further questions.

0
votes
Claudia NEUMUELLER posted this 10 September 2019

Hi Dmitriy,

yes, a lot of packages are using the same connection.

But not all af them are runnign in parallel. The number of packages is big, but we will need to keep them for business purposes.

The package I mentioned was fired with batch size = 1 or 2.

That was critical and causes the out-of-limit problem.

Please let me know why the batch size is set so small on that package.

 

kind rgds

Claudia

 

0
votes
Claudia NEUMUELLER posted this 11 September 2019

Hi Dimitriy, Team,

we are still waiting for your reply here.

Thnx!

Claudia

0
votes
Dmitriy Muzurov posted this 12 September 2019

Hi Claudia,

Thank you for getting back to us.

We asked our development team to look into the package.

Once we have any details, we will update this ticket.

0
votes
Dmitriy Muzurov posted this 16 September 2019

Hi Claudia,

Our development team looked into the package.

The limit is reached with a number of packages and this particular package should not be considered as a root cause.

The batch that contains 1 or 2 records is a normal behaviour. Skyvia determines the number of records in the batch automatically based on several points: the parallel tasks running, complexity of the package, overal load of the system etc.

We would recommend decreasing the number of packages running in parallel and setting up some of them to run on different days as Salesforce counts all API calls for 24 hours made from an IP address regardless of the batch size.

Should you have any questions, do not hesitate to contact us back.

Close