Xano Pack | Coda with no row limits

:raised_hands: A Xano Pack for Coda: Store Millions of rows :raised_hands:

tl;dr: A 2 way sync pack for Xano allowing you to interact with millions of records in Coda. Grow and scale your workflows without limits with the new Xano pack for Coda.

What does this pack do? :thinking:

The Xano pack for Coda allows you to interact with your data in Coda and leverage all of Codas formulas, buttons, dashboards, and collaborative features without fear of your data ever growing too large or your workflow getting too big.

The Xano pack for Coda provides a 2 way sync to your databases in Xano so all the changes you make in Coda effortlessly flow back into Xano

And any changes you make in Xano of course appear in Coda for a never stale source of truth.

Tell me more, how does it work :triangular_ruler:

Xano is a scalable no-code database with no limits on the amount of records/rows that can be held

The Xano pack for Coda allows you to pull in rows from your databases into Coda with a powerful 2 way sync. Additionally, the pack allows you to create new rows and records directly from Coda with intuitively designed actions.

All your data is stored in Xano and then viewed, interacted with, and edited on Coda as the “front-end”

Why might I need this?

To be very honest you may not. There are many workflows where having support for 10,000 - 50,000 rows in a Coda table is sufficient. But when and where those numbers are not sufficient, this pack shines.

As your business grows and scales, so often will the size of your data. If you are noticing that your data sets are getting too large for Coda or your docs are staring to slow down, this pack may be for you.

Why did we build this?

At Simpladocs we have seen Coda transform how organizations are able to work, collaborate, and streamline operations and data.

Unfortunately though, we have also seen teams steer away from the use of Coda due to row/record limits. Especially enterprises that deal with larger amounts of data! We are excited to help continue to bring Coda to the Enterprise by allowing Codas unique building blocks to be leveraged even when data sizes far outgrow what Coda is able to handle.

Questions / Support

If you have any questions you can always find us at info@simpladocs.com

:framed_picture: Some sweet photos :framed_picture:

Oooohhh the dashboards you can build with your data


Look at that amazing Two way sync
Edit your Coda tables directly, and all the information is instantly synced back to Xano for a never stale source of truth


Easily add new records to your databases
Use our extremely easy to use and intuitive button to add new records to your databases! The button already knows the columns existing in your database, so you just need to add the inputs/values for the columns!

10 Likes

Hi lovely community. Scott and I have had a great time working on this pack together, and we’re really proud of the outcome. It’s been a while since I’ve launched a non-private pack, and it’s incredible to see how far the platform has matured.

The XANO pack takes full advantage of all the newest features the packs team have launched, and I think it’s turned out to be a perfect example of how those features can come together to make packs a total powerhouse feature of Coda as a platform.

We’ve obviously got 2-way sync, and with dynamic sync tables your content just streams in as a perfect 1:1 schema match to however your tables are setup within XANO. The new Coda.ValueType.Select field type combined with the ability to provide options asynchronously via a fetch means that ENUM columns are particularly delightful to use when editing. For example, if you have a STATUS column setup with a set of available options, you don’t have to manually replicate those options within your doc - the pack will do that for you. Couldn’t be more intuitive!

We’ve also got support for complex table filters (called “search” within XANO) so if you only need a subset of your rows synced into Coda, there are advanced ways to do just that. We think this is perfect for ensuring good doc performance if you’re working with massive tables. But in our testing, this turned out to be a great way to restrict access to sensitive information too. You don’t need to apply a filter to your table in Coda and hope other users don’t figure out how to “unhide rows”. Rather setup the sync table to only retrieve rows that suit your needs, and have other rows never come into Coda at all.

There’s a ton more, such a Scott’s lovely solution to adding new rows when we obviously can’t predict the sync table’s schema. He used it for DocuMint and I basically lifted it from there. It’s a total boss solution!

I’ve zhooshed up the launch doc with a ton of examples and super clear documentation, so if you’re a XANO user or even just need a not-yet-available solution for handling access control, check it out.

Here’s the updated link:

3 Likes

So excited to check this out! This is coming JUST as our company is moving a big system of ours to Xano (while still doing project management in Coda)

We also have a few others Docs that have “archive” Docs where we slowly move rows so that our main Doc never gets too big (mainly so the API continues to function)

This pack looks like it will allow us to use Xano as our main storehouse and just bring in current items into our Coda Doc — thank you for building this!

Hello @Jono_Bouwmeester and @Scott_Collier-Weir ,

This sounds very promising for a lot of projects. I hate to admit I didn’t try it yet, but I do have two questions after reading the above and the documentation about filtering the data you pull from Xano, in particular hiding confidential data or otherwise filtered data:

  1. say, you want to retrieve (like for a dashboard) the sales data (from a Xano table) for the last 100 days. I understand how you would setup the filter. Next week, the filter would skip the first 7 days from the previous batch and add the last 7 days. So far, so good. But the first 7 days ars still in the Coda synced table. Obviously you can set a filter on the synced table in Coda, but the data is still there. Eventually, you are going to have data from many years in your Coda table, getting to a point where your doc slows down to a halt.

  2. Say you filter the sync data based on the logged in user’s name. When another user logs in, the sync data is filtered based on the other user’s name. At that point, even if you filter your Coda table based on the same criteria, both user’s will have access to the data of the other person (with search or by any of the other means discussed previously in this community).

Both of these scenarios are about distinctly different situation, but come down to the lack of being able to protect any confidential data or excess data once it reaches Coda. How would you suggest to address these issues?
Am I overlooking something obvious, or is this something your pack is not trying to address at this time?

Greetings, Joost

2 Likes

Hi @Michael_Warren,
You’ve highlighted some perfect use-cases. Good luck with the integration, and feel free to hit us up if you need a hand with anything. Excited to hear how it goes! :slight_smile:

Yeah for sure Michael!

We think it will be a Godsend to many different people. If you need any help setting it up or simply have more questions about whether or not this could be a solution for your workflows, drop us a line at: info@simpladocs.com

This might be a dumb question but does the performance suffer less if you have a synced table with 100k rows rather than a normal table of the same size? Or if we’re only syncing a small subset, what’s the difference if the 100k rows are in another doc instead of Xano?

As far as I know, you just can’t sync more than 10K rows. No pack can bypass this limit.

@Jono_Bouwmeester ,
@Scott_Collier-Weir ,

Is it possible to come back with a substantial response to the questions of @joost_mineur. I believe it is important.

Looking forward reading your responses.

Cheers, Christiaan

1 Like

Here! Jono and I collaborated to get you all the answers you need below :point_down:

2 Likes

Hi @joost_mineur and @Christiaan_Huizer

Joost, thanks for the question and apologies for a late response. I was clearly a little vague in my original verbiage so wanted to take the time to be more precise in my clarification.

To be clear, the pack pre-filtering table rows before they hit Coda is definitely not a universally applicable solution for securely restricting content.

Lets address both of your questions first Joost:

QUESTION 1

Say, you want to retrieve (like for a dashboard) the sales data (from a Xano table) for the last 100 days. I understand how you would setup the filter. Next week, the filter would skip the first 7 days from the previous batch and add the last 7 days. So far, so good. But the first 7 days ars still in the Coda synced table. Obviously you can set a filter on the synced table in Coda, but the data is still there. Eventually, you are going to have data from many years in your Coda table, getting to a point where your doc slows down to a halt.

Sync tables, aka packs, aka the integration we have built for Xano always follows the same dynamic filtering principles. So if you say to Xano: “Hey, can you give me just the most recent records from the last 7 days”, Those records will continue to change as each day progresses.

You will always get the most recent 7 days based on the current days date. So there’s no fear of, as you put it, “You are going to have data from many years in your Coda table”

The brilliance of the Coda<>Xano pairing is that you can always maintain data from many many years, growing into the millions of records. But In Coda, via your sync table’s filters, you can choose which relevant rows you want to see/interact with at any given point

QUESTION 2

Say you filter the sync data based on the logged in user’s name. When another user logs in, the sync data is filtered based on the other user’s name. At that point, even if you filter your Coda table based on the same criteria, both user’s will have access to the data of the other person (with search or by any of the other means discussed previously in this community).

First off, let me state clearly that we did not create this pack to deal with or address issues of security, access, and permissions. To deal faithfully with large datasets around security, permissions, etc its honestly best not to use Coda at all. Coda wasn’t developed to be a true webApp development platform where the data is filtered at the API level.

We built this pack to enable users to continue using Coda as the collaborative platform of choice even when datasizes grow well beyond Codas limits.

Sync tables can never sync in different data based on the logged in user. Personal controls don’t work as valid inputs to a sync table filter.

Therefore, the same data that is loaded from Xano for JOHN DOE will always be the same data available for JANE DOE.

So then what did we build this pack for? What’s its ideal use?

Example 1: Social Media Company

Say a larger influencer marketing company has 50 clients. And for each client, they are keeping track of high level metrics on the success of a social media campaign they are running for that client.

Lets say for each client, there are 30,000 relevant records in relation to their campaign metrics. That’s a total of 1.5 million records. These sizes are too larger for Coda.

What this enables for the client is that they are able to have one unified source of truth database for all their campaigns in Xano, and then they can use our Xano pack to create unique portals (separate Coda docs) for their clients where they sync in only the clients related records / views.

Example 2: Here’s another cool, and slightly different example

Let’s say my global e-commerce business stores all its data in Xano, and I need to provide a subset of this data to a subcontractor who manages distribution and support for a specific region without sharing sensitive info about global operations. Here we can do that safely. Rows for orders or customers from other regions just never make their way into the doc.

What’s also cool is that you can setup Xano API tokens with fine-grained permissions for who can do what. Combine this with private accounts for pack writes, and we have a doc that only syncs in rows we predetermine (and never anything else) and also constrain individual users of the doc to only perform specific write operations within their job function. Distribution can only modify the orders table, customer support can only reset passwords on the users table, and the accounts team can only modify payment information. And each user class can perform these intuitively directly on the sync table, without us having to build fragile workflows to try control their behavior. I’m sure we’ve all launched docs with a “users” table that maps users to specific roles, then implement those role permissions with a mess of Switch() statements, complex layouts, “read-only” formula columns. Those docs are a nightmare to inherit, and i think our pack solves this difficult problem really easily.

Example 3: Customer Research / Reports

Lets say there is a large organization that operates / sells 20 different products (We actually have a client just like this!) - and for each product they do extensive customer research, cataloging things such as:

  • Customer reviews
  • Bug reports
  • Feature requests
  • Customer interviews, etc

For each product i there are 50,000 records over a couple years, thats around 2 million records. Once again, too large for Coda.

But maybe the teams workflow requires that during a certain day, they are only looking at bug reports for product x. Or during another day they are only looking at open features requests in the last year for product y.

They can declare those filters and sync the relevant data in from Xano and then utilize all the wonderful building blocks of Coda that we know all too well!

I hope that answers the questions raised, and better clarifies our goals for this pack build. Please let me know if anything’s still unclear and if you give the pack a test, let me know how you get along! :slight_smile:

1 Like

Hey @Pablo_DV. Thanks for the questions. As @Breno_Nunes has already pointed out, sync tables from packs are hard-limited to a maximum of 10,000 rows. That’s the highest table limit you can set from the Advanced tab in the table options.

To answer your question regarding performance, a native table will be the most performant option, but even native tables with that many rows will probably cause your doc to become sluggish.

What sync table filters tries to achieve is the best of both worlds - your primary data store (XANO) can have millions of rows without any performance issues, and because you’re only syncing the rows into Coda that you need for your specific purpose, you’re able to keep your doc performant too.

3 Likes

hi @Jono_Bouwmeester ,

thanks for the detailed response, much appreciated.

cheers, Christiaan

2 Likes

Let’s say you have a register of all sales transactions on Xano and you want want a multi-year dashboard of sales by month.

Naturally, you can’t import millions of rows and summarize in Coda.

So, can you import a transformed, summarized table of totals/averages/etc. by month?

Or can you only import data as-is?

Awesome pack! I will consider using Xano for projects with large datasets. Just exploring the possibilities.

Hello @Jono_Bouwmeester,

Sorry about the impatience - it kind of felt like you guys were skipping my question, which turns out to be a wrong assumption.

Thank you for your very elaborate answer. I am glad to hear that the pack will limit and update the sync to only show the filtered records and that it will (magically) get rid of records that don’t fit the filter any longer. That allows indeed for a master table in Xano and work with a limited data set in Coda.

About your agency example: I can see some value there, but I have to find a way to update my ‘agency’ docs in an automated way. I don’t mean the data, but the actual layout of tables and pages. It seems to me like quite a burden and error prone to update 50 or more docs everytime you make a change to the master (or template) doc. We have solved this in the past by exporting our Coda data to firebase and have a secure app with proper logins for our users retrieve the right dataset for these individual users. But then bottleneck is the somewhat limited data capacity of Coda, so a better backend like Xano seems the way to go. In our solution we only have to update one (1) app, rather than a document for each user: I hope to find a solution to do the same with Coda, but I guess we are not there yet. My personal choice would be to have our users access data in a (shared) coda doc, which is so much more flexible (and modifialble) than a ‘normal’ Appsstore or Playstore app.

I am writing this in the hope it inspires the both of you to perhaps work on some other great packs.

Thanks again,
Greetings, Joost

2 Likes

Hi @Alberto_Delgado

Thanks for the query. XANO does support something called Direct Database Queries, where you effectively run a MySQL query against your tables. It’s only available to paid users (“Launch” plan or higher). Direct Database Query - Xano Documentation

This would allow you to write a statement which outputs your data in a more meaningful way for that particular use case. Something like:

SELECT MONTH(transaction_date) as Month,SUM(total_sales) as Sales FROM orders GROUP BY MONTH(transaction_date)

That’d output a list of each month with the total sales for that month summed together. So you’d get a sync table with 12*Years rows, each with the Month and total Sales for that particular month. That’d be SUPER efficient in terms of the goal you’ve presented.

Direct Database Queries are not something we currently support in the pack, but if there’s enough interest of course we’re keen to add additional functionality. Good pack design seems to be finding the right combination of “what do Coda users need” and “how do I keep this simple”.

Something you’d be keen to investigate further, @Alberto_Delgado ?

Hi @joost_mineur

I hear ya on the “one shared client doc” dream - I’ve actually spent a significant amount of time working on a solution for that dilemma in the past, and just haven’t been able to get it right without some real hacky stuff.

I do feel like at least having a central database somewhere and being able to intuitively work with it in Coda is a step in the right direction. Hopefully this pack is helpful for that purpose, and moves us all a little closer to a good solution.

This is something we’re actively working on, so please feel free to post your feedback and feature requests - we’ll review everything! There’s a form in the launch doc. Here’s that link again:

3 Likes

Awesome!

Thanks for the detailed explanation. I’m glad it’s possible to query aggregated data, because such queries would make this pack useful for larger enterprise use-cases.

With that said, I don’t have a use-case for it myself. Just curious.

3 Likes

Great work getting this stood up. I assume you have done similar integrations with Airtable?

Definitely seen your post there! Have done similar things with Airtable in the past but never specifically with Xano.

Why do you ask?