Improvements to timeout and retry behavior in Packs

Making Packs
1

Summary: Fetcher requests will timeout and retry more reliably, and with more visibility in the logs.

As many developers know, dealing with network requests can be messy. Most of the time everything just works, but occasionally you run into a latency spike or hit a rate limit. If you stop and think about how complex the internet is, it’s amazing it works at all!

The Fetcher service included in the Packs SDK does a lot of work behind the scenes to smooth over these bumps, but it hasn’t been as robust or transparent as we’d like. We recently made some updates that should improve the experience for Pack makers.

Smart timeouts

Previously requests would have a 50 second timeout, to stay under the 60 second execution time limit. However if you made multiple requests in sequence that static limit would break down.

Now the timeouts are dynamic, set just under the time remaining in the execution, so that the request will fail with an error before the whole execution does.

Transparent retries

When you hit an API rate limit and get a 429 response, Coda will automatically retry the request after a few seconds. However those retries were hidden from sight, and made it hard to understand the behavior of the Pack.

We still perform automatic retries, but now each failed request and subsequent retry are visible in the logs shown in the Pack maker tools.

Better rate limit errors

Even with retries, sometimes you just run out of time to wait. Previously when this happened the Pack execution would fail with an obscure error message, which couldn’t be caught.

Now we detect if there is enough time left for a retry, and if not fail the request with the last 429 response that the server returned. This allows you to catch it in your code and show an error to the user.

All of these changes should be backwards-compatible with the code you have today, but if you notice any issues let us know. And add a reply if you have other changes you’d like to see in the Fetcher service.

13 Likes
2

Ho awesome, thank you for the update @Eric_Koleda and all the team involved :+1:
I just have a quick question, I have a pack where I wrote custom code to catch 429 errors and auto adjust some parameters before retrying to avoid rate limits. Does that mean this will never get executed or does coda only handle retries automatically if I re throw the error ?

1 Like
3

@Martin_Portevin - Good question. I don’t think that code would have worked previously, as we would auto-retry the request until the execution timed out. With the new behavior your code will eventually get the 429 response, but only at the last second right before the execution times out.

We chatted about having an option to turn off the auto-retry behavior, so developers could implement their own retry logic. Is that what you’d want to do in this case?

4

I think that having a predictible and uniform behaviour is better and I’d opt for Coda handling that part, rather than doing some guess work within the code.

1 Like
5

My bad ! I just checked my code again and the API I was talking about actually always return a 200 status code but with a custom error message in the returned JSON. That’s what I was handling, not a 429.
So, at least in my case, I don’t think that would be necessary.

2 Likes
6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.