Complex Coda Docs — Mapping them out for sustainability and continuous development

Hey guys,

I’m currently making a rather complex Coda doc and have some concerns, mainly evolving around the sustainability of a HUGE Coda doc with 10+ collaborators.

Some scenario’s:

  • We discover something that’s not working properly and need to improve it (accross different views)
  • Coda comes out with an update (e.g.: granular permissions)
  • We have a new idea to implement

Once this huge Coda doc has been made, I can imagine it to be tedious to find the right items to correct and adapt to certain changes. Which to edit where so that the workflow remains optimised.

I have a background in Engineering (University degree, no practical experience, went solo right away), so I know a lot of different modelling techniques, but I’m not sure which one would be the best to use for modelling and mapping out a Coda doc so that the model can be used for little tweaks and improvements based on radical changes.

The whole business process also needs to be mapped out more, so if it is something that combines mapping out a business process with mapping out the Coda doc, that’d be great!

I’m looking forward to any tips on Frameworks, Modelling Languages, apps and so on.

Thank you!
Sonny

(The Coda doc I’m talking about would probably have about 50-100 tables and 30+ sections. It’s a content management system that starts with the content (multiple hours), dissects into edited content and then into shorter clips/trailers/socialmedia content. All managed within Coda. There will be revisors, editors, managers, writers, etc. collaborating with one another. )

Hey Sonny,

Not sure if this is what you’re after, but here are some quick tips. These are the best practices I discovered while making my own docs and building docs for hire. Right now it’s just a list — I’ll expand more on it in my upcoming blog codatricks.com

The most important piece of advice is:
Always assume two roles: doc builder and doc user.

Which means, always have a view into your data that’s convenient for a doc builder (where nothing is grouped or hidden, and all the data structure is apparent), and always provide good UI/UX for end users (e.g., well-formatted views, prominently visible controls, links etc).

Here’s a set of tips:

  • Keep all your master tables separate from user-facing content. Ideally place each master table in a separate section, all sections in a dedicated folder named “Database (do not touch)” or such. Only use views throughout the rest of the doc. This is the most frequent sin I see throughout people’s docs.
  • Keep master tables ungrouped, unsorted, unformatted, with all columns visible. It’s much easier for a doc builder to work with this than with any other display setting.
  • When changing table structure (e.g. adding columns, editing formulas), always do so in the master tables. Then make new columns visible in views. Not the other way around.
  • Establish some naming convention to tell apart master tables and helping tables from views. E.g. prefix master table names with DB and helper tables with _.
  • Always prefer verbose table and column names over cryptic short names. It’s better to name a column with 6–8 words (e.g. “Number of non-failed contracts starting this month”) than name it e.g. “Contracts” and stumble on it each time later down the road.
  • Break down formulas into smaller pieces. Split down a column formula into several steps (several columns) if you feel it becomes hard to follow, even if you’re not going to reuse any intermediate calculations. As a software engineer, use your best judgement as you would when splitting one long expression into multiple temporary variables.
  • Use chaining instead of wrapping (e.g. Table.Filter(...).First(), not First(Filter(Table, ...))). Almost all Coda functions support this syntax. It makes formulas much easier to understand for everyone on your team and for you later because it reads like a chain of steps.
    More
  • Put effort into formatting your formulas with line breaks (shift+enter) and indents (2 spaces works best). This takes time, but makes formulas much more maintainable.
    More
  • Use correct column types. When using lookup columns, pay extra attention that a) you select a right table, and b) you properly select whether multiple items selection is turned on. When writing lookup formulas, be explicit if you want to only select a single value by using .First() in the end — otherwise the type would be inferred as “a list of one element” and you’ll run into issues when using that result in further calculations.
    More
  • Avoid using .ToText(). Usually it’s misused to convert a row reference into a string, e.g. to remove links / “blue text” or when it’s needed to compare a lookup column to some string. This will break if you change your display column. If you find yourself typing ToText(), stop and think again. Maybe what you really want is dereference a specific column instead.
  • Discourage any possibility for users to mess up table display settings. Instead of making end users change filters, grouping, sorting etc via dialogs (the hidden icons in table headers), provide obvious in-canvas controls like here.

This is what comes from the top of my head. I probably forgot something. And yeah, this is basically an outline for my upcoming article on best Coda practices on my upcoming blog. It’s just much harder to write a proper article than type out this reply in a single take :slight_smile:

14 Likes

This post should be pinned :round_pushpin:

6 Likes

Dear @Paul_Danyliuk, :four_leaf_clover:

I learned this the hard way and in a time consuming manner, somehow this should be included on top of the formulas page.

Thanks again taking the time to summarize this precious content :handshake:

:bulb:
*I would really like to advise users to check out the PARA method, developed by Tiago Forte.
His method is very valuable for ALL types of digital content

2 Likes

Hey Paul,

This was not exactly what I was after, but very much welcome nonetheless, some great rules you put out there, they will definitely help. Love that you took out the time to write it out and subscribed to your newsletter :wink: Looking forward to some more value bombs!

I’m now learning that mapping/modelling our process is highly needed, I’ve used BPMN to model the basics, but thought I’d ask here if someone has discovered a certain modelling language or framework that works really well with Coda Docs, aimed for sustainability and scaleability.

As I’ve said, I’m a complete amateur when it comes to actual real life applications haha, I’ve learned all the theory in university, but that’s where it stops. I’ve been lucky enough to work with a partner that has established a relatively successful business model, but we both have no experience with actual scaling. We’re now at 10 employees and while business is going great, daily operations are suffering. So we need to look at ways to do things better and prepare for scale. That’s where Coda came in. We’re moving away from static sheets and docs to dynamic coda docs focussed on collaboration and automation.

Hope that makes sense.

Cheers,
Sonny

Thanks! Will definitely check it out.

Woops, been replying with my other account :man_facepalming:

Well, I don’t have any theoretical knowledge about project management, so cannot help you with that (I just had to google what BPMN meant, lol). But what I’m pretty sure about is that Coda is flexible enough so that you can model virtually any workflow with it.

Like, I totally see how it could replace Jira, for example.

(shameless plug: bending Coda and pushing its limits to make the trickiest docs is my top skill — people hire me when they hit roadblocks with Coda themselves. And yeah, what you can do with Coda despite its constraints is A LOT. Virtually anything from Sheets with proper approach can be recreated in Coda)

Coda is indeed awesome. There’s no other updates to which I’m more excited than Coda updates! Looking forward to what they’ll add in the future.

Totally agree with what you’re saying, and love the plug, keep hustling man! There’s money to be made with Coda for sure.

Cheers,
Sonny

Yes - this is a concern I share as well. There are two dimensions to this issue (as I see it) -

  1. Change management
  2. Process model impact when change occurs

#1 is the essential idea that intimates that changes must be managed holistically when there are dependencies. #2 factors in the impact of change - however well-intentioned and smart it may seem - on the entire process.

I worry about #1 a lot simply because there are no viable ways (that I know of) to utilize versioning techniques with Coda at the present time. The API doesn’t provide access to all document elements so we cannot easily integrate with existing version control systems.

#2 requires the ability to simulate changes on business process logic, and thus probably necessitates deep integration.

For two clients I have created rudimentary API dependency maps that are built in Google Apps Script but presented in Coda docs. The systems use charts from D3 to help visualize the relationships between Coda tables that are populated and updated through API processes. Nothing fancy - just enough information to help makers and users understand what leans on what in their document-centric applications.

1 Like

Hey Bill,

Thank you very much for your extensive replies.

You seem to know what you’re doing and I have already learned from both your replies.
It indeed seems that Coda is missing some features when factoring for sustainability.

Cheers,
Sonny

Yes, I think that’s a fair take-away if we can agree on the definition of “sustainability”. :wink:

Is it possible we are expecting too much from Coda itself? I mean, after all - this is a “document” for heaven’s sake! In the realm of innovative new approaches to information management, shit like this is going to bubble up and we probably need to let it percolate a bit before opening the big umbrella of expectations.

Haha, we might be. As said above, there are no other updates I’m more looking forward to than those on Coda. I’m really excited to see where they’ll take this tool in the future and have a lot of trust in their capabilities.

@Paul_Danyliuk do you have any advice on how to best space/format a long formula when chaining / not wrapping? When I have a long formula it makes sense to me to write:

if(
1=1,
true,
false
)

And I’d “indent 1=1, true, false” two spaces (this formatter won’t let me do that in this reply).

What are you best practices for formatting a long chained formula for human readability?

@Johg_Ananda, @Paul_Danyliuk probably knows far more about this in terms of the mechanics, but can you not reduce this formula example to:

(1 = 1)

?

@Bill_French one can reduce that formula to simply true, but I’m pretty sure it was just for demonstration and perhaps some comic relief :slight_smile:

@Johg_Ananda to paste a block of code and keep the spaces, use Markdown code block formatting: three grave accents before (on a separate line) and three grave accents after:

image

Let me reply in a few minutes. Short answer: use your best judgement when to break down formulas into multiple lines. I’m coming from Java background, so I tend to use Java code style -like. And people coming from C would code their calculations in C instead of using Coda at all :upside_down_face:

Indeed, but I was asking in terms of a real equation such as:

({value1} = {value2})

… instead of:

If({value1} = {value2}, true, false)

@Johg_Ananda Okay, here’s a more expanded reply. I’d illustrate it with a formula I wrote yesterday:

RunActions(
  [DB Scenario metric uplifts].Filter(
    Scenario = thisRow
    AND [Value driver].IsNotBlank()
    AND thisRow.[Select value drivers].Contains([Value driver]).Not()
  ).DeleteRows(),
  If(
    thisRow.[Select value drivers].IsNotBlank() AND [DB Scenario metric uplifts].Filter(Scenario = thisRow AND [Value driver].IsBlank()).Count() = 0,
    Sequence(0, 5).FormulaMap(
      [DB Scenario metric uplifts].AddRow(
        Scenario, thisRow,
        Year, CurrentValue,
        Ramping, [Template ramping].Nth(CurrentValue + 1)
      )
    ),
    _Noop()
  ),
  If(
    thisRow.[Select value drivers].IsBlank(),
    _Noop(),
    thisRow.[Select value drivers].FormulaMap(
      RunActions(
        thisRow.ModifyRows(thisRow.[_Current value driver], CurrentValue),
        thisRow.[_Current value driver].[Metrics Impacted].FormulaMap(
          RunActions(
            thisRow.ModifyRows(
              thisRow.[_Current metric impact], CurrentValue
            ),
            thisRow.[_Add current metric impact]
          )
        )
      )
    )
  )
)

As I said above, I use best judgement when deciding whether to break down into separate lines or not. The goal is formula readability in the constraints of a pretty narrow window. If breaking down into separate lines makes reading the formula harder and not easier, it would make sense to not do that, right?

Example of that:

In the DeleteRows formula, I’m breaking down the conditional in Filter into separate lines:


This is what I’m usually doing BTW with long boolean expressions with lots of ANDs.
However, further down I’m not breaking a similar long filter expression into separate lines:

This is easier for me to read because I know that to jump to the if expr=true clause, I don’t need to scan the formula line by line, but can simply jump the block of text that overflows, right to the next indented line. Whereas if I broke down the expression like this:
image
it would require me to scan each line, looking for a comma.

My rules of thumb:

  • RunActions() — always break down and indent, each action starting on a new line. If actions are too long, consider adding empty lines between actions.
  • FormulaMap() — always break down and indent.
  • Filter() — keep inline if the condition is simple (only one operation e.g. X = Y or X.Contains(Y).Not()) or breaking down would only make overall readability worse (e.g. when surrounded by lots of other code), otherwise break down and indent.
  • If() — keep inline if the condition is simple and the true/false values are simple as well, otherwise break down and indent.
  • Switch() — always break down and indent. Write the expression on a separate line, then case and value pairs each on the same line if the case is just a value. Otherwise consider using SwitchIf().
  • SwitchIf() — always break down and indent. Write expressions and calculations on separate lines. Consider adding empty lines between expression/calculation pairs if either part gets broken down into multiple lines itself:
SwitchIf(
  thisRow.[New lines in this row].Not(),
  "",

  thisRow.[Cohort type] = "Adds",
  thisRow.Scenario.[Metric uplifts].Filter(
    Metric = [New Lines/Services]
    AND Year = thisRow.[Year index]
  ).[Ramped uplift].Sum(),

  thisRow.Scenario.[Metric uplifts].Filter(
    Metric = Winbacks
    AND Year = thisRow.[Year index]
  ).[Ramped uplift].Sum()
)
  • AddRow(), ModifyRows() — only consider keeping inline if you’re setting one column->value pair, otherwise break down into separate lines, with column and its value being on the same line:
    image

Also if one of the paths in an If() is long (i.e., takes more lines than a formula editor can display), and the condition is easy to swap (e.g. just replacing .IsBlank() with .IsNotBlank(), consider writing a shorter path, e.g. _Noop(), first, so that you can see it while still seeing the condition:

1 Like

OK I’m doing that. I thought you were ‘never’ wrapping, whereas you’re using a combination of chaining and wrapping. I agree that for short simple formulas chaining makes sense (first(), sum()) but that for the larger architectural formulas wrapping and indenting makes sense. Thanks I’m always learning from you @Paul_Danyliuk.

Well, it’s still chaining. What I meant was to write in the style of:

Table.Filter(
  ...
).Sum()

instead of the Excel-like

Sum(
  Filter(
    Table,
    ...
  )
)

I am NEVER writing things like First(), Sum() like that, no matter how long the expression.

That’s independent of line breaks and indents :slight_smile:

As for wrapping, I’m still using it for If(), SwitchIf(), RunActions() and some string/math functions where you cannot write e.g. 1.Max(2) instead of Max(1, 2), or it would be stupid to do so :slight_smile:

If a list of parameters is homogeneous (e.g. Min(1, 2, 3, 4, 5)), then by all means write it as a function. But if the 1st parameter is the object onto which the formula applies the rest of the parameters (e.g. a table, followed by a list of column-value pairs), then put the object outside and use chaining.