[Task]: Time out issue taking time to complete job

[chiragvats18]

What needs to happen?

Can anyone help as i was trying to make upsert job by pipeline by taking table input and doing upsert the data is about 2lakh something so it was taking time to get data as we have setup the timeout so it is timeout connection so is there any other way to build the job easily and fast without any error.

Issue Priority

Priority: 2

Issue Component

Component: Pipelines

[bamaer]

I don't know if I understand this correctly. If your database connection times out, there usually are JDBC properties to control timeouts etc. If your upsert is slow: check your indexes.

[dmainou]

Further, if the target table is large you may want to implement logic that reads, merges and inserts new rows followed by logic that reads merges and updates existing rows.

If the target table is humongous then you may want to avoid the upsert transform and use logic that puts the onus on the database. E.g load your rows to a table and execute a script that merges and updates inside the db.

[chiragvats18]

We are not putting this in table we are taking data from table and data is bulk as an input and doing upsert to the Salesforce i have also tried by increase number of rows but buffer input and buffer output becomes fill either we have to increase buffer size but unable to find the location how to change it or if any other solution like bulk input from the table then which component should we can use?

[dmainou]

So break it down into smaller pieces.
output your results to a table.
Then use a different job to process on line at a time.
It sounds like Salesforce is getting overwhelmed so send less data in one go.

[dmainou]

200k rows is very very little so there is some setting you need to change or Salesforce is throttling you.

[chiragvats18]

Dear Sir,

Is there any option to use Bulk API(like table bulk input / salesforce bulk upsert/salesforce bulk api setting on for salesforce upsert) because apache hop is currently using i think so soap api but we are unable to see bulk api setting on for this if we have to use any plugin or like that
as it has run as i have reduced batched size and increased number of copies.

[chiragvats18]

Dear Sir, Is there any option to use Bulk API(like table bulk input / salesforce bulk upsert/salesforce bulk api setting on for salesforce upsert) because apache hop is currently using i think so soap api but we are unable to see bulk api setting on for this if we have to use any plugin or like that as it has run as i have reduced batched size and increased number of copies. CHIRAG VATS 8076590813

…

On Tue, Mar 24, 2026 at 4:38 AM Diego Mainou ***@***.***> wrote: So break it down into smaller pieces. output your results to a table. Then use a different job to process on line at a time. It sounds like Salesforce is getting overwhelmed so send less data in one go. — Reply to this email directly, view it on GitHub <#6841?email_source=notifications&email_token=BWJJNT3V7ICRXBC5CO4GI2T4SG7VNA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNRSHAZTOMBSUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-16283702>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BWJJNT6BEF44OST33OD5S2D4SG7VNAVCNFSM6AAAAACW3IAITSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMMRYGM3TAMQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[hansva]

Bulk API for Salesforce is not (yet) available.