Pro hack: Trigger cross-doc sync with an action (hidden formula)

Digging into document JSON files, just discovered a way to trigger cross-doc table sync with an action. It’s a bit tricky: the action must be triggered by Automation Bot, so you’ll have to set up automation.
Fair warning: uses hidden formulas (although they seem harmless)

  1. Create a button for your imported table with the following chain of actions:
RunActions(
  YourTableName._StartSync(),
  Crossdoc::_SyncTableTable([Your cross-doc connection], YourTableName),
  YourTableName._FinishSync()
)
  1. Since this button cannot be clicked by you, you must set up some sort of automation to click it for you. What you’re interested here is setting a trigger-based automation (e.g. on a cell value changed).

Now whenever you need to trigger sync programmatically, just change the value in that observed cell (e.g. set a timestamp to Now()), wait some time until it triggers the automation, which in turn presses the sync action button, which forces the cross-doc table to sync.


Still, a proper user-clickable action to directly request table sync is very much anticipated.

Please subscribe to codatricks.com. With the quarantine in effect, I might finally have time to start writing this blog.

12 Likes

2 posts were merged into an existing topic: CrossDoc - Add Row Sync Time

How do you suggest this happens? If we are using a button to do a CrossDoc Action and then want it to sync the table, the sync will probably be ‘too quick’ for the sync to pick up the new data. Are you suggesting a delay function?

I am suggesting to try and see whether it will be too quick or maybe not.

I have a theory that it may actually work.

  1. Cross-doc action doesn’t return immediately, but takes some time (e.g. you can see that the button doesn’t become available for a while). I noticed it on large (~125 MB) docs, that Crossdoc::ModifyRows could take a good minute or two to complete. Hence I have a feeling that this action is “synchronous” — in terms that it completes only when the data is guaranteed to be saved in the snapshot.

  2. I don’t know the nature of Automation Bot’s delay, but I think it might be caused by some operation queue on Coda servers. I could see saving snapshots being part of that operation queue.

So it’s quite possible (although I’m not fully sure) that when you first do the action and then force sync via the method I described, it could actually perform in order every time. I haven’t tested it myself, but it’s a good chance for you to do that.

(sorry for moving posts here and there; I thought this was a continuation of conversation in CrossDoc - Add Row Sync Time)

By that sentence I meant that it usually takes some time between cell value changed and automation triggered. So it’s not instant, and with this process you have to wait until automation runs.

ok so I think you are saying that by using the automation bot it will be hopping into a queue that should have the ‘addrow’ function in it, so perhaps it will wait until that is complete before initiating the sync?

This is a theory. To test it I’d have to make some docs and set them to perform this some 100 times at least and see if every time it would work. I haven’t done that yet.

1 Like

@Paul_Danyliuk :pray: thank you for this. I was able to implement and after shallow testing I can report that it does function as desired; activating a crossdoc addrow action button that is paired with the above pro-hack adds the row back to the source table and then initiates a sync which brings said added row in through the sync. As you said I will need more testing to see its fidelity, but early tests show it works! This is a huge step closer to two-way sync! :pray:

2 Likes

Thanks for reporting back and glad that it works (and glad to have my theory confirmed).

Please report back if it at some point it fails.

Also please keep in mind that this is all undocumented functionality that may change / break at any moment. So while you can rely on it while it lasts, accept that at some point it won’t work anymore.

@Paul_Danyliuk thanks for this idea - it taught me a few things along the way and it is addressing something that has been a real problem since I started playing with crossdoc tables.

I’ve implemented your pro-hack as follows:

  • I have button which opens a new row (add row) for editing and it has a button column for “save”'ing the entry - this is how I’m trying to simulate a form entry in a modal dialog for adding some data.
  • The “save” button runs the Crossdoc::AddRow to send the data to my remote table. I’ve wrapped it with a RunActions call so that it does that and also updates the timestamp in the Trigger table as per your solution.

I’m finding that my data is sent across and my sync is triggered, but I’m finding that the sync is triggering before my remote data is saved in the snapshot that gets sync’d.

Are there any ideas for how I can delay that sync trigger to give more time for the snapshot to be updated before the sync is triggered?

I was trying to see if I could leverage the timestamp where I would update the timestamp to Now()+Seconds(10) and then have the automation only trigger if Now() was later than or equal to the timestamp, but I haven’t found a way to make that happen.

Oh. This means that my approach doesn’t work reliably then.

You can delay the sync trigger by wrapping your timestamp-updating ModifyRows action with a _Delay(ms), e.g.

ModifyRows(TriggerTable.Timestamp, Now())._Delay(10000)

for a 10 second delay.

This is not going to be much more reliable though. The time of snapshot creation is non-deterministic.

A reliable (but super-mundane) way to approach this would be to set up an automation on the receiving doc to crossdoc-update the timestamp remotely from that doc into this :slight_smile: which would then trigger the automation in this doc to start the sync.

1 Like

I would like to test this but I’m not seeing a way to use a Crossdoc Action in an Automation. Do I need to setup up layer of indirection to support that (e.g. when my receiving table is updated, update a trigger table and have an automation that pushes a button to do the Crossdoc Action to update the timestamp on the remote Trigger table?

Thanks!

Let’s say Doc A is a data source and Doc B is where you have the sync table from Doc A. I see the process like this:

  1. Doc B inserts a row into Doc A through a Cross-doc action
  2. Doc A has an automation listening on new/updated rows in that table you just inserted a row to.
  3. When automation triggers, Doc A updates that trigger cell in Doc B through a Cross-doc ModifyRow action.
  4. Doc B has an automation listening on that cell change, as per the trick. This triggers the startsync/synctabletable scenario, and the table eventually updates.

So the idea is to replace who’s modifying that cell and triggering the sync. Previously it was Doc B who updated that value. Now it should be Doc A remotely into Doc B. This should work because the automation in Doc A will only fire when the added row has actually landed into the snapshot.

Regarding setting this up, I personally always prefer to set up any actions in buttons, not in the automation settings directly. I think editing buttons is easier, partially because it preserves newlines.

3 Likes

Thanks for clarifying @Paul_Danyliuk.

I set this up and it worked. There’s still a bit of delay before all is complete and the sync completes, but it’s working more reliably now.

This is a clever approach - thanks for taking me through it and teaching me many things along the way.

That said, will this delay be a permanent barrier or are there plans to make it more “instantaneous”?

I ask as in my use case, I’m asking a user of the doc for input data, storing that in a remote doc, and then I want to have the user see what they just input via the sync’d table. This is equivalent of a web app presenting a modal form, the user providing some information and clicking “save”, and that information showing up in a table of like information immediately so the user has a sense of their input being “saved”.

If there’s not a more instantaneous sync solution, I may have to remove the “table of like information” so that the user just submits information and it goes off into the ether, but they’re not waiting for it to show up like a traditional web app would (by instantaenously making a call to a backend that will pull the updated data from a database, for example).

Huge thanks again for this clever pro-hack.

I wouldn’t know — I’m not working at Coda :slight_smile: But given how Coda works under the hood (which I had a chance to learn purely by reverse-engineering it myself), I doubt that any improvements to this particular use case will happen anytime soon.

And yeah, if you want to build the UX where the user immediately sees what they have submitted, you should probably build that UI around the “local” table, not the sync one. The user adds the row to that table (via a form-like layout, for example), it gets submitted but they see it already. Then when it’s synced back (into a separate, sync table), it’s looked up into that local table and e.g. a checkbox “Saved” becomes true for that row as a confirmation. Something like that.

The problem with this approach though is that users can delete rows from this local table in Doc B, thinking it would also delete rows remotely on your Doc A.

(and yeah, I’ve built a solution like that for a client a few months ago, and in the end chose to base the end user’s UI on a sync table and just warn them that the data would not appear immediately)

2 Likes

Thanks, at least I’m netting out in the same spot that you ended up, even with starting to warn the users. :slight_smile:

I had the same concern about local edits to the table and that’s why I wanted to source of truth to be the remote table (similar to a remote data store / database concept).

Thanks again for all of your insights!

1 Like

Paul what do you mean by this?

Ah, only that the line breaks in the formula (indented code formatting like I usually do) are gone in automation formulas, making them harder to comprehend and edit afterwards.

ezgif.com-optimize (82)

ahh ok yes! I agree. It would be nice if linebreaks would remain in code blocks here too :slight_smile:

You mean the community? You wrap a code block with three accents before and three accents after, on separate lines:

(also it was another reply to you :wink: )