About today's Coda outage

Today Coda was down. I could only access it offline. There are many things you can’t do when you work offline. You can’t upload pictures or do cross-doc for example
This is a very rare event. Coda is usually very reliable.
I usually check status.coda.io to know if it’s me or Coda’s issue.
The thing is if you check today’s outage, it says it only lasted 8 min but for me it lasted way longer. I can’t tell exactly how long but it took at least one hour to get back up.
Was it my internet or status.coda.io that is not showing correctly?

Hello @Breno_Nunes,

Thanks for asking about this!

We did have a partial outage today due to a bad code change in our daily push which we unfortunately didn’t catch in our staging environment.

When our primary oncall engineer attempted to update our status page with a more descriptive message while we continued debugging the issue, we discovered a login issue which prevented him from making manual updates. Our automated system for updating the status page thankfully did work properly.

The nature of this outage could have affected some customers more than others, and you were probably one of the unlucky ones :frowning: This would have been made more clear in our manual message if we had been able to set it.

Both the root cause and the login issue have since been resolved.

We follow a fairly standard post mortem process to capture details of the incident, identify root causes, and ensure they cannot recur. If there’s interest, I’ll look at publishing the internal Coda template we use for this.

4 Likes

I am still having issue with my Doc where the “Calculating” shown up all the time and I can’t use my Doc at all.

The problem started yesterday suddenly and the Debug calculator did not show me the cause. Is it related to the incidents you have mentioned?

Anyway, I have contacted support and still waiting for reply.

Not saying it’s impossible, but it’s likely it’s a different issue.

Contacting support was a good route to go as we can check stats more easily there.