How to reference Objects in a table column (Pack Building)

Background:
I have a large-ish data set (~5k rows) that I want to pull into Coda for reference and analysis. These are invoices so there is a need to bring in new ones as they are created but I do NOT need to sync old ones since they are fairly static once created.

Sync Table:
I could create a sync table for this, but it seems really stupid since this will query all ~5k rows and I really only need the most recent 200. The issue is if I filter to only the most recent ones the sync table will remove the other 4,800 rows.

Custom Function:
I’ve created a custom function that grabs the rows I want and outputs (I think) an array of objects. I then combine this with FormulaMap to input these objects into a table. This is where things get weird.

Issues:

  1. The table sees them as objects, but not really. They don’t have the icon or rounded edges, but when I mouse over them I can see other attributes
  2. Despite seeing some other attributes, most of the attributes defined in the schema are missing. This was because I was passing the reference schema and not the actual schema.
  3. Any formulas in other columns that attempt to reference the attributes on the object return a blank string
  4. I see the following error in the Pack Log: Formula Invoice failed: Unexpected token R in JSON at position 0

I’ve also tried using formatColumn to tell Coda that these are objects that follow a schema. The issue there is that it seems this was designed to only work for a function that alters the value and I can’t really tell what that value is. Any function I try to run on it (in the pack or in Coda itself) returns very unexpected values. (JSON.parse errors out, referencing elements directly fails).

Am I doing something entirely wrong? This seems like it should be incredibly simple but somehow is actually impossible.

Some Relevant Code Snippets:

Schema and Reference Schema

const InvoiceSchema = coda.makeObjectSchema({
  properties: {
    uuid: {
      type: coda.ValueType.String,
      required: true
    },
    id: {
      type: coda.ValueType.String,
      required: true
    },
    data: { type: coda.ValueType.String },
    serviceFee: { type: coda.ValueType.Number },
    year: { type: coda.ValueType.String },
    startDate: { type: coda.ValueType.String },
    endDate: { type: coda.ValueType.String },
    totalSales: { type: coda.ValueType.Number },
    company: CompanyReferenceSchema,
    location: LocationReferenceSchema
  },
  displayProperty: "id",
  idProperty: "uuid",
  featuredProperties: ["uuid", "id", "company", "location", "serviceFee"]

});

const InvoiceReferenceSchema = coda.makeReferenceSchemaFromObjectSchema(
  InvoiceSchema, "Invoice");

Fetcher Function:

async function fetchInvoices(nextToken = null, context) {
  let invoices = [];
  let payload = {
    nextToken: nextToken,
    perPage: 500,
    ...
  };
  let response = await context.fetcher.fetch({
    method: "POST",
    url: "<url>",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify(payload),
  });
  let data = response.body;
  if (data.nextToken) {
    let additional_responses = await fetchInvoices(data.nextToken, context);
    //invoices = additional_responses;
  }
  for (let item of data.items) {
    invoices.push(formatInvoice(item))
  }

  return invoices;
}

function formatInvoice(item) {
  return {
    uuid: item.id,
    data: item.data,
    id: item.id,
    company: { uuid: item.data.company?.id },
    location: { uuid: item.data.location?.id },
    year: item.data.year,
    startDate: item.data.startDate,
    endDate: item.data.endDate,
    totalSales: item.data.sales.totalSales,
    
  }
}

pack.addFormula({
  name: "FetchRecentInvoices",
  description: "Fetches any invoices in the last 30 days.",
  parameters: [],
  resultType: coda.ValueType.Array,
  items: InvoiceSchema,
  execute: async (args, context) => {
    const response = await fetchInvoices(null, context);
    return response;
  },
});

Attempt at Column Format:

pack.addColumnFormat({
  name: "Invoice",
  instructions: "Invoice Objects",
  formulaName: "Invoice",
  
})

pack.addFormula({
  name: "Invoice",
  description: "Shows an invoice",
  resultType: coda.ValueType.Object,
  schema: InvoiceSchema,
  parameters: [
    coda.makeParameter({
      type: coda.ParameterType.String,
      name: "input",
      description: "f",
    }),
  ],
  execute: async function ([input], context) {
    return JSON.parse(input)
  }


})

Why don’t you just create a sync table - and in your execute code just filter out any rows that are old?

Also what happens if you change the “items” in your “fetchRecentInvoices” to be “invoiceSchema”

Filtering the list in a sync table will remove the rows. I don’t want to remove them, I just want to avoid resyncing the entire table when 90% of it is static.

Good catch on the Item type. I had caught that after posting and fixed it. Still can’t get the column type to properly be the object. The data is all there, but I can’t seem to get it to work like an object (e.g. trying to reference a portion of the object in another column doesn’t work. Invoice.totalSales returns nothing.)

Can you share your doc?

Unfortunately I don’t think so. I’m pulling data from my company and it’s not meant for external consumption. :confused:

Dang. Well I think I know what’s going on but it’s hard to help without a doc share. Maybe @Eric_Koleda can provide some more insight.

Can you give me a few hints? I can try editing some stuff to see if it helps.

@Scott_Collier-Weir

Well I think I know what’s going on

Any way you can give me even something vague? This one is really giving me fits. :pray::pray:

Any way you can share doc with me privately???

I’m sorry I really don’t feel comfortable with that without some pretty serious clean-up of the data. I’m trying to understand what information seeing the doc would give? I thought I had shared all the relevant details but perhaps I missed some? Or were you looking to be able to just rapidly try things out?

Hi @Barry_Brinkley1 - I think the simplest solution would be just to toggle on the advanced option “Keep un-synced rows”:

image

This allows your sync table to hold on to old rows, even if the latest sync didn’t return them. I think that should allow you to keep your sync’s snappy (just return the new stuff) while not losing the old stuff.

In regards to your original problem, Pack objects aren’t terribly portable currently. That is to say that you can’t easily move them between tables, since the information about their structure gets lots on the journey. The simplest approach is to break the object apart into separate columns of data, although that can get to be a bit annoying when your objects have lots of properties.

P.S. - I moved this conversation into the Making Packs channel, which the Packs team and I monitor more closely.

1 Like

@Eric_Koleda thanks for the reply. I swear I checked to see if I had that option as I’d seen it on other packs but somehow wasn’t able to find it.

I’m kind of shocked that this hasn’t come up more. It seems like being able to define a column type without an unnecessary function on it would solve this issue (and in general be pretty handy). Hopefully that’s on the roadmap at some point. :slight_smile:

Also appreciate moving the thread. Long time reader first time caller, wasn’t sure where to post.

It actually does come up a bit, but there just isn’t an obvious solution and the team hasn’t prioritized the work yet. I’ll pass this thread along to them though.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.