How do I migrate multiple docs' data to new pages and schemas?

[Ed_Wei]

I built a doc for tracking investments as part of my Real Estate Investing coaching. I would like share this system with my cohort by creating a template and removing all of my rows in the template.

However, as I continue building out my system, early adopters will be stuck on earlier versions of the template. How do I migrate people onto new versions?

Would I need to find a process to have people create a new doc based on the new template and then copy data between the documents?

[Piet_Strydom]

Hi Ed,

It is a problem that I am facing with two docs that I am building. One would not be distributed often, the other would hopefully be mass distributed.

I have not really thought about the problem yet, still focused on building the base document itself.

Thinking aloud:
There would be two issues to solve in the process - data and functions, both of which can be solved by either moving the data and functions (any views, extra tables created by the user) from the old doc to the new doc. The other alternative would to copy your new data and functions to the old doc.

Off the cuff, I do not think that the functionality to copy page(s) to a new doc is going to help in this scenario.

1. Moving from old Doc to the new Doc
   Data (And this is may be the easier of the two?):
   One way to solve the data issue would be to make use of the CSV pack that is available, and export the data to a CSV file, and then import into the new Doc.
   Functions
   Any views that the user has created on your tables would need to be recreated. Any “local” tables and views would also need to be recreated.

2. Copying new functionality to the old doc.
   It seems like this will be the more difficult approach, especially if you touch the existing tables. If it is completely new tables, views and data, then it would be relatively easy to just duplicate those pages into the old Doc. But I cannot see that this will often be the case.

3. Coda solves the problem.
   I VERY much doubt that you and I are the only ones facing this problem, hopefully the Codans are hard at work at solving this (very difficult) problem.

How to address this topic, would also depend on the nature of the doc. The first doc that I am working on is to provide some assistance to SAP projects as far as documentation, design and collecting user input goes. In final form this will have 100/ 200 or more tables containing master data and other configuration. It would be a nightmare to push that back though CSV files to a new Doc. On top of that, meetings and design information would be stored in canvas columns, doubt that CSV exchange would work for this. But this would be fairly large implementations, and the company would probably have an internal or external Coda person to assist with upgrades.

My other doc would be very different - it is to help John and Jane Doe to do their personal financial planning. The idea is to record their financial information, and then develop answer to questions like:

• What happens if I die too soon?
• What happens if I don’t die soon enough?
• What happens if I become ill? And die, or don’t die?

So this doc would be very heavy on calculations, and forms/functionality to input data like share investments, 401(k) pensions etc, all updating a central table. So very integrated, and users with no Coda experience.

Hopefully your question will start a discussion here.

Rambling Pete

[Piet_Strydom]

And new thought: There is probably data (contents in tables) as well as data tables (the structure of tables and their relationships with each other.)

And functions could be detailed as forms, formula columns in tables, and canvas level formulas.)

R
P

[Paul_Danyliuk]

Here’s my 5c.

Coda explored the idea of “version control” at some hackathon. The idea would be this: a doc is created of a template and lives its own life as a “branch”. Then someone makes improvements to the template and pushes (“merges”) the edits into each doc using the template. Edits are granular enough, e.g. “add column”, “rename this”, “move that”, so that applying the list of operations onto a doc that already has its own history shouldn’t result in conflicts. And if there were, Coda would offer a way to resolve these.

That said, Coda hackathon projects notoriously have a motto “build things that don’t ship” There’s obviously a lot of work to bring those 48-hour-experiments to production. It might just not be the priority.

So in absence of an official solution, I was thinking of how to best facilitate migration myself. I don’t have any plan or advice yet, only these ideas:

• It will definitely be easier to migrate the data. Take a new template, make copies, distribute to people, and populate each copy with the data from their previous version doc.

• There must be a way to package the data on the old doc side and then unpackage on the new doc side. You can’t just iterate over rows from whatever tables and upsert them with API / Packs — you’ll hit rate limiting before you even know it.

• Since it can’t reliably be done through API, black magic has to be involved. Unpackaging the parcel of migrated data should happen on client side.

• There must be a Row ID system in place that would preserve original IDs after they’re inserted into a new doc. Partly because you’ll have to implement a mechanism that would re-link all lookups between newly created rows.

• All that said, perhaps when thinking of migration you have to consider the least amount of data that you absolutely need to bring over. E.g. migrating the task tracker, consider leaving all the old tasks in the old doc and only bringing over e.g. the overall stats.

So, in general case not an easy task. In individual cases could be manageable.