Xano Pack | Coda with no row limits

:raised_hands: A Xano Pack for Coda: Store Millions of rows :raised_hands:

tl;dr: A 2 way sync pack for Xano allowing you to interact with millions of records in Coda. Grow and scale your workflows without limits with the new Xano pack for Coda.

What does this pack do? :thinking:

The Xano pack for Coda allows you to interact with your data in Coda and leverage all of Codas formulas, buttons, dashboards, and collaborative features without fear of your data ever growing too large or your workflow getting too big.

The Xano pack for Coda provides a 2 way sync to your databases in Xano so all the changes you make in Coda effortlessly flow back into Xano

And any changes you make in Xano of course appear in Coda for a never stale source of truth.

Tell me more, how does it work :triangular_ruler:

Xano is a scalable no-code database with no limits on the amount of records/rows that can be held

The Xano pack for Coda allows you to pull in rows from your databases into Coda with a powerful 2 way sync. Additionally, the pack allows you to create new rows and records directly from Coda with intuitively designed actions.

All your data is stored in Xano and then viewed, interacted with, and edited on Coda as the ā€œfront-endā€

Why might I need this?

To be very honest you may not. There are many workflows where having support for 10,000 - 50,000 rows in a Coda table is sufficient. But when and where those numbers are not sufficient, this pack shines.

As your business grows and scales, so often will the size of your data. If you are noticing that your data sets are getting too large for Coda or your docs are staring to slow down, this pack may be for you.

Why did we build this?

At Simpladocs we have seen Coda transform how organizations are able to work, collaborate, and streamline operations and data.

Unfortunately though, we have also seen teams steer away from the use of Coda due to row/record limits. Especially enterprises that deal with larger amounts of data! We are excited to help continue to bring Coda to the Enterprise by allowing Codas unique building blocks to be leveraged even when data sizes far outgrow what Coda is able to handle.

Questions / Support

If you have any questions you can always find us at info@simpladocs.com

:framed_picture: Some sweet photos :framed_picture:

Oooohhh the dashboards you can build with your data


Look at that amazing Two way sync
Edit your Coda tables directly, and all the information is instantly synced back to Xano for a never stale source of truth


Easily add new records to your databases
Use our extremely easy to use and intuitive button to add new records to your databases! The button already knows the columns existing in your database, so you just need to add the inputs/values for the columns!

12 Likes

Hi lovely community. Scott and I have had a great time working on this pack together, and weā€™re really proud of the outcome. Itā€™s been a while since Iā€™ve launched a non-private pack, and itā€™s incredible to see how far the platform has matured.

The XANO pack takes full advantage of all the newest features the packs team have launched, and I think itā€™s turned out to be a perfect example of how those features can come together to make packs a total powerhouse feature of Coda as a platform.

Weā€™ve obviously got 2-way sync, and with dynamic sync tables your content just streams in as a perfect 1:1 schema match to however your tables are setup within XANO. The new Coda.ValueType.Select field type combined with the ability to provide options asynchronously via a fetch means that ENUM columns are particularly delightful to use when editing. For example, if you have a STATUS column setup with a set of available options, you donā€™t have to manually replicate those options within your doc - the pack will do that for you. Couldnā€™t be more intuitive!

Weā€™ve also got support for complex table filters (called ā€œsearchā€ within XANO) so if you only need a subset of your rows synced into Coda, there are advanced ways to do just that. We think this is perfect for ensuring good doc performance if youā€™re working with massive tables. But in our testing, this turned out to be a great way to restrict access to sensitive information too. You donā€™t need to apply a filter to your table in Coda and hope other users donā€™t figure out how to ā€œunhide rowsā€. Rather setup the sync table to only retrieve rows that suit your needs, and have other rows never come into Coda at all.

Thereā€™s a ton more, such a Scottā€™s lovely solution to adding new rows when we obviously canā€™t predict the sync tableā€™s schema. He used it for DocuMint and I basically lifted it from there. Itā€™s a total boss solution!

Iā€™ve zhooshed up the launch doc with a ton of examples and super clear documentation, so if youā€™re a XANO user or even just need a not-yet-available solution for handling access control, check it out.

Hereā€™s the updated link:

4 Likes

So excited to check this out! This is coming JUST as our company is moving a big system of ours to Xano (while still doing project management in Coda)

We also have a few others Docs that have ā€œarchiveā€ Docs where we slowly move rows so that our main Doc never gets too big (mainly so the API continues to function)

This pack looks like it will allow us to use Xano as our main storehouse and just bring in current items into our Coda Doc ā€” thank you for building this!

1 Like

Hello @Jono_Bouwmeester and @Scott_Collier-Weir ,

This sounds very promising for a lot of projects. I hate to admit I didnā€™t try it yet, but I do have two questions after reading the above and the documentation about filtering the data you pull from Xano, in particular hiding confidential data or otherwise filtered data:

  1. say, you want to retrieve (like for a dashboard) the sales data (from a Xano table) for the last 100 days. I understand how you would setup the filter. Next week, the filter would skip the first 7 days from the previous batch and add the last 7 days. So far, so good. But the first 7 days ars still in the Coda synced table. Obviously you can set a filter on the synced table in Coda, but the data is still there. Eventually, you are going to have data from many years in your Coda table, getting to a point where your doc slows down to a halt.

  2. Say you filter the sync data based on the logged in userā€™s name. When another user logs in, the sync data is filtered based on the other userā€™s name. At that point, even if you filter your Coda table based on the same criteria, both userā€™s will have access to the data of the other person (with search or by any of the other means discussed previously in this community).

Both of these scenarios are about distinctly different situation, but come down to the lack of being able to protect any confidential data or excess data once it reaches Coda. How would you suggest to address these issues?
Am I overlooking something obvious, or is this something your pack is not trying to address at this time?

Greetings, Joost

3 Likes

Hi @Michael_Warren,
Youā€™ve highlighted some perfect use-cases. Good luck with the integration, and feel free to hit us up if you need a hand with anything. Excited to hear how it goes! :slight_smile:

1 Like

Yeah for sure Michael!

We think it will be a Godsend to many different people. If you need any help setting it up or simply have more questions about whether or not this could be a solution for your workflows, drop us a line at: info@simpladocs.com

1 Like

This might be a dumb question but does the performance suffer less if you have a synced table with 100k rows rather than a normal table of the same size? Or if weā€™re only syncing a small subset, whatā€™s the difference if the 100k rows are in another doc instead of Xano?

1 Like

As far as I know, you just canā€™t sync more than 10K rows. No pack can bypass this limit.

1 Like

@Jono_Bouwmeester ,
@Scott_Collier-Weir ,

Is it possible to come back with a substantial response to the questions of @joost_mineur. I believe it is important.

Looking forward reading your responses.

Cheers, Christiaan

2 Likes

Here! Jono and I collaborated to get you all the answers you need below :point_down:

3 Likes

Hi @joost_mineur and @Christiaan_Huizer

Joost, thanks for the question and apologies for a late response. I was clearly a little vague in my original verbiage so wanted to take the time to be more precise in my clarification.

To be clear, the pack pre-filtering table rows before they hit Coda is definitely not a universally applicable solution for securely restricting content.

Lets address both of your questions first Joost:

QUESTION 1

Say, you want to retrieve (like for a dashboard) the sales data (from a Xano table) for the last 100 days. I understand how you would setup the filter. Next week, the filter would skip the first 7 days from the previous batch and add the last 7 days. So far, so good. But the first 7 days ars still in the Coda synced table. Obviously you can set a filter on the synced table in Coda, but the data is still there. Eventually, you are going to have data from many years in your Coda table, getting to a point where your doc slows down to a halt.

Sync tables, aka packs, aka the integration we have built for Xano always follows the same dynamic filtering principles. So if you say to Xano: ā€œHey, can you give me just the most recent records from the last 7 daysā€, Those records will continue to change as each day progresses.

You will always get the most recent 7 days based on the current days date. So thereā€™s no fear of, as you put it, ā€œYou are going to have data from many years in your Coda tableā€

The brilliance of the Coda<>Xano pairing is that you can always maintain data from many many years, growing into the millions of records. But In Coda, via your sync tableā€™s filters, you can choose which relevant rows you want to see/interact with at any given point

QUESTION 2

Say you filter the sync data based on the logged in userā€™s name. When another user logs in, the sync data is filtered based on the other userā€™s name. At that point, even if you filter your Coda table based on the same criteria, both userā€™s will have access to the data of the other person (with search or by any of the other means discussed previously in this community).

First off, let me state clearly that we did not create this pack to deal with or address issues of security, access, and permissions. To deal faithfully with large datasets around security, permissions, etc its honestly best not to use Coda at all. Coda wasnā€™t developed to be a true webApp development platform where the data is filtered at the API level.

We built this pack to enable users to continue using Coda as the collaborative platform of choice even when datasizes grow well beyond Codas limits.

Sync tables can never sync in different data based on the logged in user. Personal controls donā€™t work as valid inputs to a sync table filter.

Therefore, the same data that is loaded from Xano for JOHN DOE will always be the same data available for JANE DOE.

So then what did we build this pack for? Whatā€™s its ideal use?

Example 1: Social Media Company

Say a larger influencer marketing company has 50 clients. And for each client, they are keeping track of high level metrics on the success of a social media campaign they are running for that client.

Lets say for each client, there are 30,000 relevant records in relation to their campaign metrics. Thatā€™s a total of 1.5 million records. These sizes are too larger for Coda.

What this enables for the client is that they are able to have one unified source of truth database for all their campaigns in Xano, and then they can use our Xano pack to create unique portals (separate Coda docs) for their clients where they sync in only the clients related records / views.

Example 2: Hereā€™s another cool, and slightly different example

Letā€™s say my global e-commerce business stores all its data in Xano, and I need to provide a subset of this data to a subcontractor who manages distribution and support for a specific region without sharing sensitive info about global operations. Here we can do that safely. Rows for orders or customers from other regions just never make their way into the doc.

Whatā€™s also cool is that you can setup Xano API tokens with fine-grained permissions for who can do what. Combine this with private accounts for pack writes, and we have a doc that only syncs in rows we predetermine (and never anything else) and also constrain individual users of the doc to only perform specific write operations within their job function. Distribution can only modify the orders table, customer support can only reset passwords on the users table, and the accounts team can only modify payment information. And each user class can perform these intuitively directly on the sync table, without us having to build fragile workflows to try control their behavior. Iā€™m sure weā€™ve all launched docs with a ā€œusersā€ table that maps users to specific roles, then implement those role permissions with a mess of Switch() statements, complex layouts, ā€œread-onlyā€ formula columns. Those docs are a nightmare to inherit, and i think our pack solves this difficult problem really easily.

Example 3: Customer Research / Reports

Lets say there is a large organization that operates / sells 20 different products (We actually have a client just like this!) - and for each product they do extensive customer research, cataloging things such as:

  • Customer reviews
  • Bug reports
  • Feature requests
  • Customer interviews, etc

For each product i there are 50,000 records over a couple years, thats around 2 million records. Once again, too large for Coda.

But maybe the teams workflow requires that during a certain day, they are only looking at bug reports for product x. Or during another day they are only looking at open features requests in the last year for product y.

They can declare those filters and sync the relevant data in from Xano and then utilize all the wonderful building blocks of Coda that we know all too well!

I hope that answers the questions raised, and better clarifies our goals for this pack build. Please let me know if anythingā€™s still unclear and if you give the pack a test, let me know how you get along! :slight_smile:

2 Likes

Hey @Pablo_DV. Thanks for the questions. As @Breno_Nunes has already pointed out, sync tables from packs are hard-limited to a maximum of 10,000 rows. Thatā€™s the highest table limit you can set from the Advanced tab in the table options.

To answer your question regarding performance, a native table will be the most performant option, but even native tables with that many rows will probably cause your doc to become sluggish.

What sync table filters tries to achieve is the best of both worlds - your primary data store (XANO) can have millions of rows without any performance issues, and because youā€™re only syncing the rows into Coda that you need for your specific purpose, youā€™re able to keep your doc performant too.

4 Likes

hi @Jono_Bouwmeester ,

thanks for the detailed response, much appreciated.

cheers, Christiaan

3 Likes

Letā€™s say you have a register of all sales transactions on Xano and you want want a multi-year dashboard of sales by month.

Naturally, you canā€™t import millions of rows and summarize in Coda.

So, can you import a transformed, summarized table of totals/averages/etc. by month?

Or can you only import data as-is?

Awesome pack! I will consider using Xano for projects with large datasets. Just exploring the possibilities.

1 Like

Hello @Jono_Bouwmeester,

Sorry about the impatience - it kind of felt like you guys were skipping my question, which turns out to be a wrong assumption.

Thank you for your very elaborate answer. I am glad to hear that the pack will limit and update the sync to only show the filtered records and that it will (magically) get rid of records that donā€™t fit the filter any longer. That allows indeed for a master table in Xano and work with a limited data set in Coda.

About your agency example: I can see some value there, but I have to find a way to update my ā€˜agencyā€™ docs in an automated way. I donā€™t mean the data, but the actual layout of tables and pages. It seems to me like quite a burden and error prone to update 50 or more docs everytime you make a change to the master (or template) doc. We have solved this in the past by exporting our Coda data to firebase and have a secure app with proper logins for our users retrieve the right dataset for these individual users. But then bottleneck is the somewhat limited data capacity of Coda, so a better backend like Xano seems the way to go. In our solution we only have to update one (1) app, rather than a document for each user: I hope to find a solution to do the same with Coda, but I guess we are not there yet. My personal choice would be to have our users access data in a (shared) coda doc, which is so much more flexible (and modifialble) than a ā€˜normalā€™ Appsstore or Playstore app.

I am writing this in the hope it inspires the both of you to perhaps work on some other great packs.

Thanks again,
Greetings, Joost

3 Likes

Hi @Alberto_Delgado

Thanks for the query. XANO does support something called Direct Database Queries, where you effectively run a MySQL query against your tables. Itā€™s only available to paid users (ā€œLaunchā€ plan or higher). Direct Database Query - Xano Documentation

This would allow you to write a statement which outputs your data in a more meaningful way for that particular use case. Something like:

SELECT MONTH(transaction_date) as Month,SUM(total_sales) as Sales FROM orders GROUP BY MONTH(transaction_date)

Thatā€™d output a list of each month with the total sales for that month summed together. So youā€™d get a sync table with 12*Years rows, each with the Month and total Sales for that particular month. Thatā€™d be SUPER efficient in terms of the goal youā€™ve presented.

Direct Database Queries are not something we currently support in the pack, but if thereā€™s enough interest of course weā€™re keen to add additional functionality. Good pack design seems to be finding the right combination of ā€œwhat do Coda users needā€ and ā€œhow do I keep this simpleā€.

Something youā€™d be keen to investigate further, @Alberto_Delgado ?

1 Like

Hi @joost_mineur

I hear ya on the ā€œone shared client docā€ dream - Iā€™ve actually spent a significant amount of time working on a solution for that dilemma in the past, and just havenā€™t been able to get it right without some real hacky stuff.

I do feel like at least having a central database somewhere and being able to intuitively work with it in Coda is a step in the right direction. Hopefully this pack is helpful for that purpose, and moves us all a little closer to a good solution.

This is something weā€™re actively working on, so please feel free to post your feedback and feature requests - weā€™ll review everything! Thereā€™s a form in the launch doc. Hereā€™s that link again:

4 Likes

Awesome!

Thanks for the detailed explanation. Iā€™m glad itā€™s possible to query aggregated data, because such queries would make this pack useful for larger enterprise use-cases.

With that said, I donā€™t have a use-case for it myself. Just curious.

4 Likes

Great work getting this stood up. I assume you have done similar integrations with Airtable?

1 Like

Definitely seen your post there! Have done similar things with Airtable in the past but never specifically with Xano.

Why do you ask?

1 Like