Open AI took too long to respond...why don´t wait a bit more?

[Bill_French]

@Sam_Barton

I’d love a fix for this.

Unfortunately, if a single inference is unable to complete in the allotted API call interval, you will see timeouts. I believe most inferences created using Coda’s AI is a single request, so you’re really at the mercy of the LLM.

I grew tired of AI systems that were sluggish and switched to Google’s PaLM 2. It seems to be about 3X faster responding and I have seen only a few times when it was busy enough to result in a failed request.

Since it also requires no paid account, I tend to break my AI processes into multiple stages. This speeds up each request while also giving me the latitude to perform long-running automations. I have a few agents that will run for 10 minutes and not time out. [more on that topic here]

Stop Building the Thing

I agree. It’s good enough to help someone who needs a little bit of guidance. Often, this guidance needn’t be perfect or pristine. But it still supports my belief that a universal law of productivity that we rarely benefit from is attainable through AI. Let’s examine a few simple goals that AI has been fairly successful with. To demonstrate, imagine you want to build a “thing”, and that thing is a plan to go see the Cybertruck in Los Angeles. Having never been to the Petersen Museum (where Cyb…

While OpenAI is the leader in LLM capabilities, I have learned that many of the open-source LLMs are powerful enough in many use cases. Ideally, Coda will make it’s AI features LLM-agnostic allowing us to choose from the dozens (soon to be hundreds) of inferencing options avialable.