I am following the instructions here:
in order to create a pack using a Github Codespace
I get to the step to run a Hello World test and then I get a TypeError as below.
Is this related to the mismatch of Node version which was caused a similar problem with using the Desktop CLI Pack Demo recently?
Interestingly the Replit demo from here GitHub - coda/packs-starter: A starter project template for creating a Coda Pack. works but not the Codespaces demo.
@richardkaplan ➜ /workspaces/hello-pack (main) $ ls
CODEOWNERS.txt LICENSE README.md node_modules pack.ts package-lock.json package.json replit.nix test tsconfig.json
@richardkaplan ➜ /workspaces/hello-pack (main) $ npx coda execute pack.ts Hello "World"
coda execute <manifestPath> <formulaName> [params..]
Execute a formula
Options:
--version Show version number [boolean]
--help Show help [boolean]
--fetch Actually fetch http requests instead of using mocks. Run
"coda auth" first to set up credentials.
[boolean] [default: true]
--vm Execute the requested command in a virtual machine that
mimics the environment Coda uses to execute Packs.
[boolean] [default: false]
--dynamicUrl For a dynamic sync table with a variable source location,
specify the URL to test here. [string]
--timerStrategy Options: none, error, fake. [string] [default: "none"]
--maxRows For a sync table, the maximum number of rows to sync.
[number] [default: 1000]
TypeError: Cannot set property crypto of #<Object> which has only a getter
at node_modules/@codahq/packs-sdk/dist/testing/injections/crypto_shim.js (/tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/bundle.js:148:21)
at __init (/tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/bundle.js:15:58)
at ../../tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/browserify-bundle.js (/tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/bundle.js:155:7)
at __require2 (/tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/bundle.js:18:52)
at /tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/bundle.js:8646:10
at Object.<anonymous> (/tmp/coda-packs-49fbafe8-865b-4ba0-8c5c-90e25427e5071SZLGa/bundle.js:8647:3)
at Module._compile (node:internal/modules/cjs/loader:1275:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1329:10)
at Module.load (node:internal/modules/cjs/loader:1133:32)
at Module._load (node:internal/modules/cjs/loader:972:12)
@richardkaplan ➜ /workspaces/hello-pack (main) $
Hi @Richard_Kaplan - Thanks for flagging this. The starter project was still using the older SDK version (1.2.3) but we recently released a new version (1.3.0) that we hope will fix this issue. I just upgraded the project to the newest SDK version, can you try again?
Worst case you should be able to run nvm use 18 to downgrade to Node 18 which should work around this issue.
Hi @Eric_Koleda
Thank you for that help. Re-creating the Codespace and downgrading to Node 18 did work.
I do have a follow-up question though.
As we discussed in a separate thread recently, I am trying to parse XML in the format in this URL/API:
https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=27919949&retmode=xml&rettype=abstract
After going over the various examples and alternatives you and @Xyzor_Max suggested previously, I realized that there are some edge cases in the XML I am trying to work with so it makes sense for me to parse this myself rather than using the auto-parsing in Coda. So as per your documentation I used the isBinaryResponse:true flag in the fetcher. I expected the XML to be passed through as in the above URL (as a string or buffer).
But it seems that even with the isBinaryResponse:true flag set, the auto-parser is still parsing into JSON and that is creating some problems with incorrect parsing.
This is my pack.ts code:
/*
Start making Packs!
Try out the hello world sample below to create your first build.
*/
// This import statement gives you access to all parts of the Coda Packs SDK.
import * as coda from "@codahq/packs-sdk";
import { clear } from "console";
import * as xml2js from "xml2js";
// This line creates your new Pack.
export const pack = coda.newPack();
pack.addNetworkDomain("nih.gov");
// Here, we add a new formula to this Pack.
pack.addFormula({
// This is the name that will be called in the formula builder.
// Remember, your formula name cannot have spaces in it.
name: "PubMed",
description: "PubMed Entrez Metdata Import",
// If your formula requires one or more inputs, you’ll define them here.
// Here, we're creating a string input called “pmid”.
parameters: [
coda.makeParameter({
type: coda.ParameterType.String,
name: "pmid",
description: "PMID Number for a citation",
}),
],
// The resultType defines what will be returned in your Coda doc. Here, we're
// returning a simple text string.
resultType: coda.ValueType.String,
// Everything inside this execute statement will happen anytime your Coda
// formula is called in a doc. An array of all user inputs is always the 1st
// parameter.
execute: async function([pmid], context) {
const efetch = `https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&&retmode=xml&rettype=abstract&id=${pmid}`;
let fetcher = context.fetcher;
const response = await fetcher.fetch({ url: efetch, method: "GET", isBinaryResponse: true });
const responseBody = await response.body;
console.log(responseBody);
console.log(responseBody["PubmedArticle"][0]["MedlineCitation"][0]["Article"][0]["Abstract"][0]["AbstractText"]);
const abstract = responseBody["PubmedArticle"][0]["MedlineCitation"][0]["Article"][0]["Abstract"][0]["AbstractText"][0];
return abstract;
},
});
This is the response I get - note from the Console.Log output the response.body has been parsed into JSON even though I am hoping it would be passed through directly as an XML string (or buffer). Can you help with this?
Hi @Richard_Kaplan - Thanks for flagging. We had previously found and fixed this XML bug in the Packs runtime, but forgot to address it in the Pack CLI as well. I just submitted a PR (#2525) that fixes it in the CLI as well. It’s not been released yet, but you can install directly from the git repo to get access to that fix:
npm install git+https://github.com/coda/packs-sdk.git
Thanks @Eric_Koleda
I re-installed the Codespace from scratch, reverted to nvm 18 again, and installed the fix.
Still the same error:
To clarify @Eric_Koleda -
Your pull request is regarding the repository coda/packs-sdk
I am using coda/packs-starter at GitHub - coda/packs-starter: A starter project template for creating a Coda Pack.
Is the fix for isBinaryResponse:true applicable to both repositories?
Hi @Richard_Kaplan - The fix was to coda/packs-sdk, and it makes it so coda execute works correctly for XML content when isBinaryResponse: true is set. Since that fix hasn’t been pushed to NPM yet, I was suggesting that after you clone the starter project you update the dependency to pull from GitHub instead of NPM.
While I have see it working correctly on my local machine, for some reason when I try to re-install the dependency in a GitHub Codespace it crashes. I haven’t been able to work out the cause though.
I think the most expedient solution for now would be to keep on the automatic parsing of XML to JSON and see if you can use that.
What edge cases are you thinking of?
Thank you @Eric_Koleda
The key issue is actually a bit more than just a rare “edge case.” It affects any abstract with multiple paragraphs.
Take for example this content which should be rendered like this:
The raw XML is fairly straightforward to understand or to parse with an algorithm in Javascript/Node:
But the autoparsing misunderstands the XML and reorders the paragraph headings with the JSON term “Label:” below the paragraph.
Moreover, the XML uses <sup> and <sub> tags for superscripts and subscripts; I would be fine if it just ignored those, but instead it moves them to JSON below the paragraphs. There is no context whatsoever for where those subscripts and superscripts should go; thus there is no algorithm anywhere which could properly transform this JSON back into rendered text properly. Anything which had been a superscript or subscript would be lost - even if I were willing to accept the formatting challenge and have that content in the same font as the body of the text.
I would be fine for the moment working on my local machine, in a Github Codespace, or in Replit [which I have noticed sometimes works with Coda when Github Codespace does not]. Is it possible to get isBinaryResponse:true working in any of these?
I see, that does make the default xml2json conversion quite useless.
Yes, just follow the quickstarts for your local machine or Replit (if that works) and then use the command from a few posts up to re-install the Pack SDK from GitHub to get the latest code.
Hello @Eric_Koleda
I tried both the GIthub Codespace quickstart and the Replit quickstart from here:
I switched to Node 18
I ran the npm command to update the SDK
In each case I can run the Hello World example but my code above to pass through unparsed XML does not work
I get this error:
~/packs-starter-2$ npx coda execute pack.ts PubMed 27919949
<Buffer 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 3f 3e 0a 3c 21 44 4f 43 54 59 50 45 20 50 75 62 6d 65 64 41 72 74 69 63 6c 65 53 65 74 20 ... 7164 more bytes>
TypeError: Cannot read properties of undefined (reading '0')
at Object.execute (/tmp/coda-packs-7c8baeaa-9a96-46ae-bcd2-8f353409d34cGWqwmR/bundle.js:8909:60)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Object.findAndExecutePackFunction (/home/runner/packs-starter-2/node_modules/@codahq/packs-sdk/dist/runtime/thunk/thunk.js:33:16)
at async findAndExecutePackFunction (/home/runner/packs-starter-2/node_modules/@codahq/packs-sdk/dist/testing/execution.js:83:18)
at async executeFormulaOrSyncFromCLI (/home/runner/packs-starter-2/node_modules/@codahq/packs-sdk/dist/testing/execution.js:176:19)
at async Object.handleExecute [as handler] (/home/runner/packs-starter-2/node_modules/@codahq/packs-sdk/dist/cli/execute.js:22:5)
~/packs-starter-2$
If you can PM me a Github username then I can add you as a collaborator on the repository - perhaps that would help to debug it?
That error and logging looks correct. response.body is now a Buffer as expected. You can turn that buffer back to a string using responseBody.toString() and then use the XML as you wish.
That works except that neither response.body nor response.body.toString() contains the XML tags. I get back pure text stripped of any XML coding; thus it would be challenging to parse out the individual metadata items.
How do I retrieve the text including the XML tags as a string?
Hi @Eric_Koleda
Not sure if you just changed/fixed something but have not responded yet - but in any event it now works fine in both Replit and my local Node environment to return the full XML code as a string with tags attached.
That’s the biggest obstacle I was working through. I can finish the rest as standard Javascript/Node coding.
Thanks for your help!
Nice, glad you got it working. Let me know if there are any other bumps along the way.
1 Like
