Some Performance Announcements

In our case, we’d love a pagination feature in tables as well.

We have a table with an icebox where we only need to see a few items in one view. Pagination will allow us to avoid overwhelming the user with too many entries.

The same happens with our backlog.

In my opinion using controls for this is too cumbersome.

So I’m a clear +1 to adding pagination controls to tables : )

2 Likes

The main use-case for pagination is what it always has been… user experience. Nobody wants to see a list of 400 items. Especially if this table is supposed to just be an element in a longer form page/report.

I really struggle to see why you guys don’t just implement this. Its almost impossible to find a grid / table system out in the wild that doesn’t have native paging, yet somehow you keep trying to justify not having it?

Hey @Richard_Browbek1 and @Raul_San_N.H

Just to make the distinction, the topic of pagination should have nothing to do with performance anymore, since we’re working on making big tables scroll faster regardless of pagination.

Totally understand that there are valid scenarios for pagination that are not possible yet. There are a few workarounds like using filtered views, canvas controls or the detail layout with bottom navigation, which is paginated with one item at a time. I completely agree that it will be useful to be able to paginate through a small number of rows at a time. It’s not a question of justifying not doing pagination but just a matter of prioritizing it against other improvements on our list.

Really appreciate hearing the feedback and scenarios and will definitely pass them on to the team to prioritize appropriately. However, would suggest we move the discussion about pagination to a different thread like this one since it is unrelated to performance.

Angad

1 Like

Edit: Ooops just seen this was already in the examples :upside_down_face:

Another possible tip, regarding CountIf.

I have this 500+ rows table with time tracking info. Date/Task/Start/End then some calcs for Duration etc.

I had previously given up on this doc as it lagged as hell. Some of the views make heavy use of grouping, so I thought it could be that, but was never able to fix it. Now with the performance tool I went back to check what was going on.

There was basically just one column taking tens of seconds to calculate. It counted the number of times a task was executed in a day:

thisTable.CountIf(Task=thisRow.Task AND Date=thisRow.Date)

I rewrote the formula as

thisTable.Filter(Task=thisRow.Task AND Date=thisRow.Date).Count()

Now it takes ~100ms.

2 Likes

Hey @Dalmo_Mendonca, sorry about the slow response here.
Yup CountIf() is one of the formulas we haven’t optimized yet.
Glad you were able to rewrite it and make it much faster!

Angad

Hey Codans (@Jason_Tamulonis, @Angad et al),

Can you please tell more about how updates are propagated through lookup-linked tables? Including tables that reference other rows from the same table.

I want to know performance implications of this vs that approach for an ever-growing data set.

What triggers a full table recalculation? And what doesn’t? How does Coda resolve which rows and columns to recalculate, or does it recalculate everything? Is there internally a sort of dependency graph, a digestion pass, or something else? How about if tables are linked through manually set references vs a formula.

As a software engineer, I’m interested in as many technical details as possible. And when clients ask me about possible slowdowns, I’d like to have some answers.

4 Likes

I have cloned a doc which works seamessly. I populated the clone with new data. It is a base table with several lookups. It is “calculating…” once and again

Very good point, @Paul_Danyliuk.
I think that having an idea of the - hopefully improving - breaking point is a rough metric we need to take into consideration.
Optimisation hints according with the under-the-hood implementations could let the community to be more aware of this.

@Angad I believe at one point I asked the support team why the load times and performance varied so much per device in comparison to other mobile apps, but don’t remember what they said. When I upgraded my phone, performance jumped incredibly. Unfortunately though, most users on my team don’t have that same advantage. It seems iOS devices have more of an issue with performance versus Android. Your thoughts on this?

Also, love the debugging tool. When @Mallikar (I believe) tipped me off to that, my life was changed :smiley: That’s also when I realized the performance differences between filters and lookups.

Thank you for your continual dedication to making Coda the best it can be!

Hi @Paul_Danyliuk,

Thanks for the question. I should first point out we are constantly improving our optimizations and thus things are always changing, so what holds today might not tomorrow. :slight_smile:

At a high level Coda does understand the dependencies between each piece of data in your document. The references in a formula and the rows selected from a lookup help build up our understanding of dependencies within the document.

When static data changes we end up looking through the dependencies to figure out which formulas need to be recalculated, those calculations can also trigger more recalculations if they themselves have dependencies.

Now there is a fair bit of complexity that goes into deciding if we can recalculate a single cell in a column verse the entire column. I could probably write a longer blog post on this, but I’ll try to outline an example where this comes up. Imagine a scenario where you have Products which have a Name, a lookup column against a Category table, and Cost. In a Categories table you have want to sum the cost of all products in that category. You could do this a number of ways but lets consider two options:

A) TotalCost= Products.Filter(Category=thisRow).Cost.Sum()
B) Items = Products.Filter(Category=thisRow)
TotalCost = Items.Cost.Sum()

Now let us consider what happens when we update the Cost of one product in both scenarios.

In option A we know the TotalCosts column depends on the Category and Cost column in the Products table. For every category we need to extract the Cost of every product associated with them and sum it. The problem here is without remembering which products belong to a given category we have to recompute the filter for every row in order to determine which category is associated with the product whose Cost changed. This is far from ideal.

You might be able to see why option B performs better now. When the cost is updated we know that only the rows, where the given product is present in Items need to be updated. There is one giant caveat here that sufficiently complicated setups can throw off optimizations and fall back to performance similar to option A.

We do have some work planned on our backlog to automatically detect and optimize option A into option B, but until that happens option B should always perform better.

14 Likes

I love Coda. The concept is a complete game-changer, and I have been able to build a workable database for our firm in less than a week. However, now that we have real data loaded into it, the performance has been dismal. After optimizing formulas, it is still taking 10s+ to load, and will continue to recalculate throughout the day. I am shocked at its inability to process this many lines, as a google spreadsheet can easily handle 3-5x as many rows without any slowdown.

I know that performance is on the minds of the developers, but this will be a complete show-stopper for anyone who is thinking about using Coda for a serious purpose. Right now it’s limited to small projects with very limited data.

What is the roadmap for bringing performance up to par with your main competitors, Airtable, Google Sheets, etc?

2 Likes

I agree with Gabriel. In my experience, as supported by Coda’s support docs too, anything above 3,000 rows will make your doc pretty much useless (give or take a few thousands depending on complexity, but then again lots of people use Coda specifically for complex purposes as you can’t do these things elsewhere). This happens either through long calculation times, or because cross-docs and Zapier will stop working with those docs (due to API limits).

The Coda doc I set up was for HR ops (timesheet, overtime, holidays, custom holiday and timing rules, self-service portal, etc) with at least a hundred columns, after lots and lots of optimizations. Then I hit several major roadblocks - the first was the slow loading, then came the long calculations, then cross-doc stopped working, then my Zapier connection with external tools stopped working. I could split the doc further and regularly archive rows into another software to keep it going, but it’s not worth the hassle. More importantly, it’s not reliable. One day it will be working, and the next day it won’t. This is a deal-breaker for me, as I’ll never be able to get my internal teammates to use this reliably. I had even planned to create a doc for our external clients, but after seeing how unreliable the bases were for our HR purposes, I’ve decided not to risk our company’s reputation by using this. So I’ve reverted to using Airtable for our HR operations after devoting over a month to setting up Coda, even though Airtable has far less features than Coda.

Hate to say it, but the answers I’ve seen so far from Coda around scalability remind me of certain threads in Airtable’s Community forums (regarding cross-base support and advanced user permissions) - the official answers over there from Airtable initially seemed very hopeful, with responses around the lines of “there’s no concrete timeframe yet, we’re looking into it, there’s lots of different use-cases to consider, lots of complexity, we want it to be solid, please keep sending us your use cases, etc”. All those answers are perfectly fine actually, but here’s the thing - those issues in Airtable have still not been addressed four years later, and counting. You can’t really blame the users for getting turned off by the repeated non-answers from Airtable now, because as it turns out that’s what they ultimately were: non-answers.

I hope Coda doesn’t go in the same direction, but to me personally, it already has. Simply put, I’ve lost faith that this will get solved in Coda in the near future, like in this year or the next, because I know this is a difficult problem to solve, and because they probably have other issues that may be getting more priority now. I’ll still keep an eye out though, in case it does get solved, but I’ve already decided to move on.

Why do I still use Airtable over Coda though? Because it can still hold data without slowing down that Coda can’t even begin to think of, in terms of number of rows, columns, formulas, and attachments.

4 Likes

Unfortunately the case, indeed. Coda is not a great tool for building a replacement for CRM or ERP or anything near it. It serves a purpose for prototyping and some not heavy loading docs. I personally build docs quickly enough to validate ideas, but then the product needs to go to something more reliable. Unfortunately because who wouldn’t want to have a scalable and easy to update document which works like a platform (very far from the app idea). Coda will be within our team for the next at least 2 years and hopefully the product will evolve to become something more reliable and really help us do business, instead of prototyping ideas.

1 Like

@Stefan_Stoyanov

What do you mean by this?

In terms of formulas and functionality (except some restrictions), I managed to create a pretty decent platform last year (for validating the idea) of a platform for managing restaurants - from menus, to ordering and cooking. However, it’s practically useless on a mobile device unless I duplicate the entire functionality with mobile in mind, which works for small things only. However, Coda was good enough only for testing the idea and quickly updating functionality until the client was happy with the product plan. After this, even a hit of one button took about over night to execute which logically meant an end of the prototyping in Coda. “Doc as powerful as app” is far from reality for Coda but I still love the product for small things and definitely nice for quickly validating ideas. There are 4 major things since Beta version of Coda that really are needed to turn into great product for long-term use and I see these has been slowly upgraded. For some projects too slowly, but we have faith for the future :slight_smile:

3 Likes

It’s also worth stating that this platform, with no changes except for the performance issues worked out, would be worth thousands of dollars per year to my firm. We are currently evaluating other platforms that would need to be built out custom, including a Salesforce-based application. None of them can beat, and few can match what Coda can do. But because of the poor performance these other platforms are not only in the running, they are likely our final destination. I would much prefer Coda, where we can create anything we want in real-time, but we have hit a wall and cannot continue to build out from here since the performance decline would be too great.

Adnan, I feel your pain. Airtable has a complementary set of issues: great performance but lack of permissions. If either one would solve the problems they keep talking about solving, they’d have my backing 100%.

1 Like

And that describes the tradeoff that is a constant battle. Why can’t the fastest cars also offroad in Moab? This is where it’s important to choose the right tool for the job and understand how to make use of the benefits each tool offers.

While Coda is not a data repository for never ending streams of data that grow well beyond hundreds of thousands of rows, it is a fantastic place for the results of these datasets and this is where the information can become actionable. Rest assured, Coda is a young company and still building out features and improving performance along the way.

If you find yourself having over 100 tables and views, over 100 columns in a single table, or over 100 sections, there is more than likely a place where information can be consolidated or offloaded. I’d encourage everyone to look at the benefits they like most in Coda, then step back and look at your project and ask yourself what data and information would benefit most by being in a Coda doc. That’s the point you build from.

I have a doc with almost 10,000 rows running as quick as any other docs I have. It’s also linked with others by Cross-doc, both sending and receiving. I generally only need a few formulas to run on a few items from this doc, so I don’t run those in that particular doc. I pull in what I need to another doc and calculate there. And I only pull in what I need, not everything every single time.

A heck of a lot can be done in Coda, it’s not small projects only, but I recommend you think through your schema and identify what you need in order to accomplish your goals.

We’ll continue to work on all facets of the product, both features and performance, and we’ll stay tuned in to all feedback we receive.

3 Likes

Let’s try to keep our expectations realistic, keep up the positive tone to stimulate the future development and be the inspiration for the Codans to flourish.

Let’s be the MAKERS and create an orchestra of tools in Coda that set the tone for the future :musical_score::musical_score::musical_score:

2 Likes

@BenLee – Don’t get me wrong. Coda is one of the most brilliant platforms I’ve seen. Ever. My point is that until the performance issues are sorted out, Coda has a hard ceiling on it for use cases. You’re welcome to jump on our database and tell me where it can be leaner, but I think we’ve built an extremely efficient app to run our firm. And as I said before, there’s just one significant issue that keeps it from being the unparalleled solution.

I am advocating for a prioritization of this issue because I believe in the insane potential that this platform has. It’s not unreasonable to propose given the performance of other platforms and the way in which the performance issues spring up: i.e., formula calculations on cross-table datasets, which are a core part of Google Sheets and Airtable. I am sure the team at Coda is painfully aware of the issue too, but I want to make sure you know that this deeply affects your everyday users and the people who would pay a significant amount of money for a platform that can already deliver in areas that your main competitors can not.

I’m behind you all, and want to give my two cents for what would make the biggest difference to us.

5 Likes