Tasks sometimes don't run or take very long and fail silently

Unknown

I have 5 background tasks. They search for games in my "games" table that are about to start, end, or transition to the next phase of the game, and run some logic accordingly.

A few issues I've noticed:

1. I have all my tasks set to run every minute. Sometimes I notice in the history of the task that the tasks just skips some chunks of time (e.g. the gap between 7:44PM and 7:48pm).
[Screen Shot 2021-12-13 at 7.55.23 PM.png]
2. Most of the time, my tasks don't find any games that are about to start/end/move to the next phase, so they "do nothing" - but even these "do nothing" tasks take 15+ seconds. Should this be happening?

3. When there ARE games that are about to start/end/change phase, and I need to process them in my tasks, sometimes the tasks take 60+ seconds - even for a single game. I can't figure out whether this is because the functions I'm calling are inherently slow (and could be sped up with better code, indexes, etc.) or because they're something slow about the task itself. I've also noticed that if a task takes more than 60 seconds to run, and the task itself runs even 60 seconds ... sometimes the task just silently fails midway. The history might show a gap like in point #1 after a "big" task is started and fails to complete, without leaving any errors to help me debug what happened.

Would appreciate some insight into this. Some of these tasks really need to terminate in a timely fashion and I'm worried that if processing one game in a task can take 60+ seconds and might also fail silently 10% of the time, that's a huge problem, since we want to eventually be processing potentially many games every minute.

Michael Udinski

Is there a chance you did any editing of your task/function during that gap? You say sometimes you notice a gap? Is it often can you let us know when it happens...

That will depend, on all the logic it needs to run through, the timings look consistent so I would assume the run time is accurate. If you want, you can add a conditional that only runs the rest of the logic if something is found. Tasks are designed to be able to handle processes of longer run times. 

It's definitely possible that the logic you have set up in fact takes this long to run. Once again, tasks are designed to handle processes of longer run times. You can debug running the task manually in Run and Debug - it'd be helpful if you can trigger an error this way. If you have many length functions set up, you may need to examine them individually as well. Especially as lots of business logic has a tendency to overload the debugger with data.

I'd recommend doing some digging first and attending an upcoming office hours with direction so we can help take a look. It's difficult to determine anything without details or specifics.

Michael Udinski

 - we now have support to view the timings of your functions in task history. Open task history, then select an item from the history and you can view a breakdown for the runtime of each function in your logic. This should help start to zero in on where things might be taking longer in your logic

Unknown

 That's awesome - I'll diagnose this in depth before posting again. Thanks!

Unknown

[Screen Shot 2021-12-20 at 1.16.53 AM.png]
This is what the first function in most of my tasks looks like - it's a query that tries to fetch all records in the games table that match a certain criteria. Most of the variables it's looking for are booleans, except for a timestamp field (gameStartTime) which has to fall within a minute-long window of when the task was started.

This query takes 5-10 seconds to run each time. I tried adding indices to the table:
[Screen Shot 2021-12-20 at 3.11.54 PM.png]
but that didn't seem to make a difference.

5-10 seconds seems really long - my games table has <10 records. Any suggestions?

Michael Udinski

 I would not recommend adding those indexes to your table in the specific use-case.

In the query all records function. Is what you have in the screen shot the only customizations you made to that function - no joins, no addons, no paging, no sort, return type list, no field customization, etc.? It's just what you have in the custom query section?

Unknown

 There is one addon - I am fetching one field from another table - but I can get rid of it. There isn't any other customization to the Query All Record function.

Unknown

[Screen Shot 2021-12-22 at 11.25.11 AM.png]

Michael Udinski

 see if that makes a difference. But I might have you sign up for an upcoming office hours if that still doesn't help for a closer look. 

Unknown

 It doesn't - I only added the addon yesterday, it was taking a similar amount of time before the addon too.

Unknown

 Would you mind directing me to the office hours schedule?

Unknown

Nvm found it.