Handling files returned by API

Making Packs
1

TL;DR: Need some help with file encoding for files returned by external API

For my QuickBooks Invoicing Pack, I’m trying to fetch invoice PDFs from the API, and show them to the user (either as a download link like in @Courtney_Milligan1’s Export Tables Pack, or as an attachment column).

The API docs say the invoices/{invoice#}/pdf endpoint returns a PDF in application/pdf format.

An example response.body:

%PDF-1.5
4 0 obj
<</Type /Page/Parent 3 0 R/Contents 5 0 R/MediaBox [0 0 612 792]/Resources<</Font<</FAAAAH 7 0 R/FAAAAJ 9 0 R/FAAABC 12 0 R>>/XObject<</X1 14 0 R>>>>/Group <</Type/Group/S/Transparency/CS/DeviceRGB>>>>
endobj
5 0 obj
<</Length 15 0 R/Filter /FlateDecode>>stream
�\b&�FJ��h]#i|N�'7XK�w;��7�}�m��D�Z����[        ��D
...

So I’m taking a hint from the raw image data Packs SDK docs, and structuring things like this with temporaryBlobStorage:

const response = await context.fetcher.fetch({
  url: url,
  method: "GET",
  headers: { Accept: "application/pdf" },
});
let fileBase64 = response.body;
let buffer = Buffer.from(fileBase64, "base64");  // This step doesn't complain
let temporaryFileUrl = await context.temporaryBlobStorage.storeBlob( // This step fails
  buffer,
  "application/pdf"
);
return temporaryFileUrl;

In the CLI, this executes without error, returning something like https://not-a-real-url.s3.amazonaws.com/tempBlob/1593a504-e08f-4019-88f6-3cc8bfff63d8

When uploaded to Coda, this error gets thrown when we hit the storeBlob() line:
Error 13 INTERNAL: Request message serialization failure: Failure: Cannot coerce to Uint8Array: object

This feels to me like an issue interpreting the encoding of the API response? I don’t have a lot of experience with this but one thing that’s notable is that the response has a bunch of normal text characters off the top, and eventually gets into unrepresentable (binary?) data. Any thoughts? Is my response really base64 or perhaps something else?

2 Likes
2

Hi @Nick_HE - Working with binary content has proven to be a bit tricky in my experience. Two options I think would work:

  1. If that Accept header isn’t required then you can use the simpler storeUrl method instead, as shown in the “Private Images” section of that same page. It basically bundles up the fetch and store into one operation.
  2. If you do need to set that header, than make sure to set isBinaryResponse: true on your fetch request, as shown here. This will cause response.body to be a Buffer, which is exactly what you need for storeBlob.
2 Likes
3

Thanks @Eric_Koleda, both of these solutions worked perfectly!

First I tried isBinaryResponse: true which did the trick (and as you say, since response.body is already a buffer, I no longer needed Buffer.from())

Commenting out the application/pdf header assured me that it still defaulted to that, and I confirmed that a simple context.temporaryBlobStorage.storeUrl(url); worked. I think I’ll go with this one for brevity.

Edit: changed my mind back to using context.fetcher.fetch with isBinaryResponse, because:

  • I want control over cacheTtlSecs
  • When using in a sync table, it helps to explicitly specify "application/pdf"as a second parameter to storeBlob() (which doesn’t seem to be available on storeUrl())
1 Like
4

I have it working both as an action - e.g. people might tie it to a button with OpenWindow(InvoicePDF(...)) - as well as in a sync table, with codaType: coda.ValueHintType.Attachment.

In the table column, the file name appears as image.jpeg. Viewing the attached file (and/or downloading it) shows a long, seemingly random hex file name.

I see a bug warning in the docs: “Attachments currently only work within a sync table. Additionally, file attachments may by shown with the wrong file name.”

I’m assuming this bug relates to the “image.jpg” file name preview in the column itself. Is there a way for me to customize the column display, or the name of the file when viewing/downloading?

1 Like
5

shows a long, seemingly random hex file name.

That’s certainly unexpected, and indicates that perhaps the content type isn’t set correctly on the response. What is the file extension of the file URL?

Is there a way for me to customize the column display, or the name of the file when viewing/downloading?

No, unfortunately that isn’t possible at the moment. It is a known a known issue that we are tracking, and obviously the user experience isn’t great when the column displays an inaccurate file name.

1 Like
6

The API request URL has no extension, it’s just .../invoices/{invoice#}/pdf

The temporary Coda URL that’s generated has no extension, e.g.:
https://not-a-real-url.s3.amazonaws.com/tempBlob/57d9c05e-cf95-43c7-b8c3-f1d39a76893d

MacOS Chrome saves it as .pdf, though it doesn’t preview the .pdf bit in the save dialog (I think Chrome is just like that though, never showing extension).

For the name itself - it’s entirely possible QuickBooks is providing this file name (I’m not sure what exactly I would expect, and I think I can recall instances where I can get PDF downloads with nonsense names like that from the UI), but ideally I’d like to be able to provide my own.

1 Like
7

Ah, I see. When you were talking about random hex you were talking about the file name, not file content as I had misread it. Yes, that name is not configurable today, and we don’t plan to make it so. That said, we are working on allowing developer to set the Content-Disposition header, such that it will download with a dev-supplied name. Stay tuned!

1 Like
8

That’s perfect. Not too fussed about its name in the preview, but the ability to pre-fill a reasonable downloadable file name takes care of my use case here.

1 Like
9

Hi @Eric_Koleda,
adding to this topic, I get the same error as @Nick_HE when uploading via the CLI, but not when directly creating the pack via Pack Studio. I’ve discovered this while migrating my pack to use the CLI, so this was with the exact same code and content returned by the API.

Unfortunately I cannot apply your solutions as I need to encapsulate the returned Base64 data inside a JSON response or would need to add another pack.addNetworkDomain line.

If I paste my code back in Pack Studio and build it there, it work without any problems !

Any ideas on what I could check ? A potential problem with my local dev environnement ?

Thanks !

1 Like
10

Hi @Martin_Portevin - Which error are you referring to specifically? This was a long thread and I want to make sure I understand your problem correctly.

2 Likes
11

Hi @Eric_Koleda , sorry for not being specific enough, the error is this one:

When hitting the storeBlob() line:
Error 13 INTERNAL: Request message serialization failure: Failure: Cannot coerce to Uint8Array: object

1 Like
12

Ah, yes, that unfortunately is still an open issue. It has something to do with how we polyfill Buffer, but I need our engineers to dig in and figure out where the issue is. I’ll ping the team again to see if we can get some attention on this issue.

2 Likes
13

Thank you for your answer, at least I know I don’t have to waste time fiddling with some obscure bug on my local machine. :slight_smile:

1 Like