How to handle backtick escapes in Coda API rich valueFormat text data

If you return rows from the Coda API with the parameter valueFormat=rich it will fully escape the text as described here:

E.g. the response text might look like:

```This is plain text```

What’s the proper / canonical way to parse text the extra ``` symbols?. For example, say I receive a list of this text like:

```A```, 0, ```B```, ```,```, ``` this is a```code``` block ```

I have a bit of logic around this in my Advanced Doc Connections (aka Cross-doc Plus) pack. Happy to share privately if you aren’t building a competing pack.

How would you get a list like that? When fetching a list of text via the API I get it back as an array of individual strings:

"Column 8": [
  "```cat```",
  "```dog```",
  "```monkey```"
]

In general I think the logic is that you should consider the value plain text if it both starts and ends with triple ticks. I thought that this could be an issue with text values which contain only a code block, but in those cases we appear to drop the closing triple ticks:

"Text": "```\nThis is just code",

Would be so grateful! Not competitive, I’m building the Archive Pack which backs up your Coda data to an external server.

In fact, if you would be willing to share the code for syncing the file types that would be amazing. I’d be happy to run it against my test suite, too

Ah, ok that makes sense and helps a lot. So the logic should be something like:

switch text {
  case ( text.regexMatch("^```.*```$") ):
    return text.regexReplace("^```", "").regexReplace("```$", ""); 
  case ( text.regexMatch("^```") ):
    return concat(text, "```");
  default:
    return text   
}

Is the final ``` needed at the end of a code block? What happens if there’s some text that has a codeblock then more regular text after it?

Also running tests on this, but figured a canonical community solution would probably be benefit many future people.

^ I think that would only remove the first triple ticks, and you’d want to remove the ending one as well.

As for the ending triple tick being needed for code blocks, it seems to depend on the destination for that markdown and what their parser accepts. In a brief survey some seem to support unclosed triple ticks and others don’t. Notably the parser used by the Pack value hint Markdown does seem to require it.

It probably does make sense to close the triple ticks, but you’ll probably need to count how many there are in the doc, and if there are an odd number then to add a closing one at the end.

Ooop, forgot the second regex replace, fixed the above.

I’m returning this data to Coda, so I’m just trying to make it work correctly in the cells like it does with Cross Doc.

What I’ve found is that backticks seem to get automatically escaped.
image

This means I can just add a sequence to unescape the escapes right at the end.

The above is kinda pseudocode anyway. I’ll share my exact solution in a subsequent post.

function recoverText(text) {
  // converts ```\`\`\`test\`\`\```` to test
  if (text.match(/^```.*```$/)) {
    text = text.replace(/^```/, "").replace(/```$/, "");
  }

  text = text.replace(/\\`/g, "`");

  return text;
}

Boy does performing this search make the sync slow though…

I’m not sure if will make a big difference, but you can eliminate a few regular expressions but using startsWith, endsWith, and substring.

function recoverText(text) {
  if (text.startsWith("```") && text.endsWith("```")) {
    text = text.substring(3, text.length - 3);
  }
  text = text.replace(/\\`/g, "`");
  return text;
}
1 Like

This is the piece of code where I’m unwrapping it in my pack:

...
} else if (/^```(?!=\n).*```$/.test(item)) {
  return item.slice(3, -3).replace(/([_*>`\\\{\}\[\]\(\)#\-\=\.\!])/g, "\\$1");
}
...

Never noticed any slowness there tbh. Or at least I wouldn’t think this could be the slowest part.

P.S. I’m supplying the result into a coda.ValueHintType.Markdown column in a sync table, so I actually want to preserve markdown if it’s proper markdown, and only unwrap the plain text from those backticks, and escape all markdown-like characters in the plain text value.

1 Like

Sorry for the tangent - but after reading the comments I realized I should learn Regex - at least basics, any good reference materials for noobs?

I’ve found the best way to learn is doing.

It’s a steep learning curve, the symbols are really arcane, and they change depending on the context [^abc].* means “nothing that matches a, b, or c” whereas ^abc.* means “nothing that begins with ‘abc’”

I think I remember once doing this: https://regexone.com/

2 Likes

Is it worth spending time to learn? because it looks like hieroglyphics haha

3 Likes

Is it worth spending time to learn?

In my opinion, yes 100%. Outside of programming it’s supported by many applications in the find and replace dialog, and can save you a ton of time if you need to filter or manipulate lists of data. There are Coda formulas that uses them as well!

3 Likes

Looks intimidating but only until you grasp the principles. Then it becomes your favorite tool to work with text :slight_smile:

I recorded a lesson on regex a while ago:

2 Likes

Definitely worth it. You’ll use it all the time.

1 Like

Yeesh - I never knew regex was that beloved. @Paul_Danyliuk I’ll check out your lesson - The amount of coda I have learned from your posts are insane… Side note: I really appreciate that you always make your stuff copyable. You could totally make your examples read only and bait people into some sort of payment for the doc (I’m not saying that having people pay for a service (doc) is wrong) - I just personally appreciate it.

3 Likes

Btw, I don’t mean to diminish your contributions @Connor_McCormick1 . I have seen a ton of stuff from you recently that Is really interesting…I’m more of the “Can i cheat on you for the test” type of coda community goer haha…

2 Likes

Haha thanks for your nice note, I’m so with you, @Paul_Danyliuk’s contributions have been enormously helpful to me, too. Imo the Coda community is what it is today only because of Paul’s consistent and brilliant contributions.

3 Likes