Is +-5000 rows table limit a rendering issue, a data model issue or what?

[Juan_Luis_Chulilla]

Here and there I have been reading about how it is not possible to work with a Coda table in a fluid way if it exceeds a limit close to, say, 5000 rows.

In my own experience I have found a limit of rendering: if the table exceeds a few tens of rows, it will render dynamically as you scroll down towards its end.

Now, my doubt is if this dynamic rendering is the culprit of slowing down the performance of the tables or if there is another more important cause for not being able to work smoothly with tables of more than 2000 rows.

[GJ_Roelofs]

I feel like I need to rewrite my comments in the other thread to be more nuanced because, to be frank, there is no real limit (to my knowledge), and at least the numbers I mentioned were for cases where I was doing quite specific and advanced things.

Having said all of that!
I generally recommend creating a custom pager for tables of that size.
I.e. create a button / dropdown / field at the top that allows you to say “Show me the first 100 rows” and pages to cycle through the table.

Is there really a usecase for wanting to show 2000 rows at once? Personally I found you’ll want to do a bit of filtering or paginating to drill down to what you want. And once you reduce the number, performance is no longer an issue.

EDIT: An example of both simple and filtered pagination:

[Juan_Luis_Chulilla]

thanks @GJ_Roelofs . As I suspected, is a rendering issue.

And you are right. There is no need of rendering, say, to render 8705 rows of a table of mine in google spreadsheet. Indeed, fine-grain controls are a far better interface than scrolling up and down.

Indeed this is a great case of why is important to restart our basic interface rutines, exchanging scrolling for filtering. A nice illustration of spreadsheet to coda migration

Anyhow, I wonder about the reason of this performance rendering issue

[William_Porter]

@GJ_Roelofs

I generally recommend creating a custom pager for tables of that size.
I.e. create a button / dropdown / field at the top that allows you to say “Show me the first 100 rows” and pages to cycle through the table.

Good grief–this is possible? This is exactly what I do in FileMaker and I’ve been grumbling about not being able to do it in Coda.

Is this done with a filter? What do you filter on? I can’t find a function like RowNumber() or GetRowNumbers(). And to be really useful, this is going to have to be a second-level filter. That is, first I want to filter for all the people whose state=Texas. Say that returns 900 rows. Now I’d like to show rows 1-20, then 21-40, of that previously filtered record group.

Can anybody please throw me a hint about how to do this (a hint or a link)? I’ll buy you the best barbecue lunch in North Texas the next time you’re in Dallas.

Will

p.s. Are bribes allowed in this forum?

[BenLee]

Hi @William_Porter,

I don’t have the full answer for you here, but something that might help. This is sort of a hack, but has worked for a long while now…

thisTable.Filter(Column1="Whatever").Find(thisRow)

This is working off of a hack that will show the row order number and adding in the filter you’ll be applying to your table. The base formula for this sort of row count is…

thisTable.Find(thisRow)

I used this in the “Options” section as a Priority column in all my options table for this Project Manager and Meeting Template:

[William_Porter]

Thanks very much, @BenLee, for the quick response and for sharing that doc. I’m not sure this gets me where I need to go, but you’ve introduced me to some new ideas and that’s always valuable.

.

I think I understand the formula for the row-number column:

thisTable.Find(thisRow)

Tried that out myself and it did insert a unique number into every row. I named that column ThisRowNum, and was able to write a filter that looks like this:

ThisRowNum > 39 and ThisRowNum < 61

which filtered the rows to display rows 40-60. Great, so far.

.

But this wasn’t exactly what I was thinking of, not what I thought (perhaps mistakenly) @GJ_Roelofs was thinking of. The problem is that the formula column (“ThisRowNum”) is populated with a value that does not change when the rows are sorted, or when a new record is added. So when I created this row-number column using the formula you suggested, I had my list of people in creation order and the first rows looked like this:

1 Adam Smith
2 Allen Morrison
3 Arlene Martin
4 Arthur Prendergast

(sorted by first name)

At that point there were 358 rows in the table. Then I added a new row for Aaron Klein and (keeping records sorted) I got this:

359 Aaron Klein
1 Adam Smith
2 Allen Morrison
3 Arlene Martin
4 Arthur Prendergast

And when I filtered for the first 10 rows, of course, Aaron Klein disappeared from view because his value is 359.

.

Is there a better way to do this? The best thing I can think of at the moment is to create a column that can be used just for filtering, say, getting first letter of family name, turning that to a number (A=1 etc) and then filtering on that column. That would produce filtered lists that vary greatly in length (more Ms than Qs) but that’s not a problem. The greater problem is, there could be several hundred Ms. And then I’m back to the display and performance problem that I’m trying to solve.

Alternatively, is there a way to limit the number of rows that Coda displays in table view, say, by somehow just telling it to show 25 rows at a time (like Gmail can do)? As far as I can tell, the only way to do this is to use a Detail viewer, since that object limits the number of rows it displays by referencing the user’s window height. Not possible to do that with normal table view?

Thanks again,

William

[BenLee]

I definitely understand the pagination need.

I think the solution here is that any sort and filter will likely need to be in the “ThisRowNum” formula to have the values assigned properly. thisTable.Find(thisRow) is not enough.

@BenLee

thisTable.Filter(Column1="Whatever").Find(thisRow)

or even

thisTable.Filter(Column1="Whatever").Sort( , ).Find(thisRow)

might be required.

[William_Porter]

HOLY COW, I think we have a winner! Many thanks, @BenLee.

I read your suggestion multiple times and couldn’t make sense of it. Was writing to ask you to explain further but thought I’d play with the suggestions myself first. Decided to leave the original column in place so I could compare the result. Then I created this new column (“Column 3” in my screenshot) with this formula:

thisTable.Sort( true,thisTable.[Name Full] ).Find(thisRow)

And looky what I got back:

As you can see, ThisRowNum (the column I created earlier) shows a value that apparently corresponds to the creation order of the records. But column 3 returns the value of the rows as sorted on the [Name Full] column. So the 229th record I created (“Aaron…”) was showing a 1 in Column 3, until I created a new record (“AAA Pest Control”). And then Aaron became #2 and AAA became #1.

And then I was able to filter this view using this filter:

Column 3 > 0 and Column 3 < 21

to show twenty records at a time.

Awesome. NOT VERY OBVIOUS. But awesome.

You sir have won a delicious Texas barbecue dinner with all the trimmings, next time you are in Dallas. Heck, if you make it to Fort Worth, I’ll drive over to meet you.

William

[GJ_Roelofs]

@BenLee to the rescue!

Sorry, I was away. For others arriving on this, an example of both filtered and simple pagination:

[William_Porter]

Okay, I’m going to follow up here and explain how I finished the job.

@BenLee showed me how to create a column that is populated with an index number representing each row’s position in the table, when the rows are sorted on some column. In my case, the rows are being sorted on the Name Full field. See my preceding message for how I implemented Ben’s suggestion.

Having done that, it’s possible to filter the records manually by writing a filter like

NameIndexNumber > 40 and NameIndexNumber < 61

and that would show rows 41-60.

The problem is, I wanted to make it easy to page through the records with clicks, so users don’t have to keep writing filter formulas. Here’s how I’ve done it for the moment.

.

Create a view in another section to do the paging/filtering

Need to have an instance of the table that is not filtered, so I can use the Count() function to get the total number of records in that table. I renamed the original table “Contacts All”, removed any filters that I’d been playing with and above this table, I placed a big red warning: “DO NOT FILTER THIS TABLE.”

Then I duplicated this section, renamed the new section “Contacts 20 at a time.” And renamed the instance of the table view there the same way (“Contacts 20 at a time”). Everything from this point on refers to this second section, which becomes the UI view, as it were.

.

Create a slider

On this second section, above the table view, I placed a slider. Named it “displayRowStarting”. Configured it so that the minimum value = 1 and the maximum value is the count of records in the Contacts All table instance. If the table instance Contacts All does get filtered, it will affect the max value of the slider and can even produce an error.

.

Create the filter that references the slider

All that remained was to filter the view by reference to the value set by the slider. The formula was simple:

NameIndexNumber > (displayRowStarting - 1 ) and
NameIndexNumber < (displayRowStarting + 20)

Et voilà!

User drags the slider and sees 20 rows at a time, based on the value the slider has set.

.

There are some things I’m concerned about. For one thing, I wonder if the formula @BenLee helped me come up with for the NameIndexNumber column is going to cause performance problems when there are thousands of records in this table, rather than the scant 230 I have in it right now. For another thing, while a slider is nifty, it doesn’t achieve the paging idea I thought I wanted initially. I used the slider because I am not yet good enough with the other Coda controls to do it any other way.

Plan to work on it some more. But this is a terrific start.

Thanks again to Ben. And I hope I haven’t wrecked this thread. I just realized I hijacked the OP’s question about technical limits. My apologies.

William

[William_Porter]

@GJ_Roelofs

Sorry, I was away. For others arriving on this, an example of both filtered and simple pagination:

YOUR TIMING IS IMPECCABLE. Seriously. I’m grateful you didn’t post this earlier, because then I would have looked at what you did first. Instead I wrestled with @BenLee’s tips and managed to find a solution on my own. Which puts me in a much better position to appreciate what looks like a better solution in your example.

Off to copy your doc. Thanks again for the original suggestion about paging and for sharing this example.

This is a great forum.

William

[GJ_Roelofs]

Looking at your solution, it seems to use the same basis!
But I agree, it’s always better to first come to a basis yourself - then at least you gain a good understanding you can use for the next issue.

Missed out on the BBQ though!

[BenLee]

@GJ_Roelofs

Missed out on the BBQ though!

I think you had the winning solution @GJ_Roelofs!

Complete with the actual pagination part.

[William_Porter]

@GJ_Roelofs

Missed out on the BBQ though!

Nothing would tickle me more than to be able to take both of you out for BBQ. Let me know if you’re ever in my neck of the woods.

William

[GJ_Roelofs]

@BenLee I think this all does show a need for a “normal” index on row objects.

[Mathieu_Brun-Picard]

Hi guys and thanks for your ideas.
I happen to have a slow doc issue : I’ve got one table with roughly 3000 rows.

Unfortunately, this table is referenced several times (c. 15) across my doc through filtered views that have 10/20 rows in average.

So in my case, I am not rendering a lot of data but it is really slow to process or to even add a row to this table.

Just for fun, I tried the index hack above and it took 15 seconds to perform the calculation

Did you ever have experience with a big table referenced across a doc ?

[Jean_Pierre_Traets]

Dear @Mathieu_Brun-Picard

Without actually having access to your document, it’s hard to give any meaningful feedback.

I assume you checked out the guidelines in this post:

Some Performance Announcements

Hey everyone! We’ve been hearing from some users about performance issues in big Coda docs. We’ve designed Coda to be a simple tool to get started but it is also packed with capabilities that let you build very complex docs. In many cases, Coda scales well to the needs of teams of hundreds of people working in the same document. However, as docs keep growing, some of our most loyal users run into performance issues. To that end, we are currently working on a number of fixes to the performance o…

If this doesn’t work out, I recommend you to get in contact with the Codans at support@coda.io

[Mathieu_Brun-Picard]

Hi @Jean_Pierre_Traets,

Thanks for your answer.
I already went through these guidelines which helped a lot when I first saw them. I’ll have another look to see if there is some quick wins to be found.

I’ll try contacting support but to be honest I don’t know if I am reaching the limits of the docs or not.