What are common reasons that would cause a background task to timeout?

jared gibb

I am running a task using pagination and streamining to loop over 166,000 records and modify a field. The task is timing out after around 4 hours and then continues to run consume DB resources for some period of time. I am making 0 external API calls so I cant guess what I'm timing out.

Is the total time a task can run dependent upon the plan a user has? Is there some memory being consumed/eaten up by error logs that is causing my timeout perhaps?

Has anyone else experienced these odd task timeouts?

Michael Udinski

Hey @jared gibb I received clarification from the engineers about the 4-hour mark on background tasks: When a task reaches the 4-hour mark it will say that it is "timed-out", however, the task will in fact continue to process and update the status once it is completed... (so it's not actually timed-out even though it will say it until the task completes).

I apologize for the confusion as this is not obvious, we're going to update our documentation to clarify this... then, we are going to update the messaging in background tasks to make this more apparent. And finally, once we improve upon the background task experience, we'll address any grey area.

Michael Udinski

Running out of memory could definitely cause a background task to timeout.

I'm curious though, you have pagination + streaming on the same query? Streaming is designed to perform pagination for you automatically and make it so the For Each Loop is more efficient with memory. So typically, you would not need to use pagination if you are using the stream return type.

jared gibb

Yes. i thought it would be helpful per conversations with Chris

many records have pages worth of text attached to each. so paging on the stream seemed to, theoretically, be in place to avoid running out of memory