I am building a Pack which retrieves XML data from an API
Most of the Pack is working. However I am able to parse the XML contents using Javascript in my local test environment but that does not work within the Pack.
This is the pertinent code
var xml = response.body;
var abstract = xml.getElementsByTagName('AbstractText')[0].innerHTML;
When I try to use this code in my Pack I get this error:
xml.getELementsByTagName is not a function
How do I fix this?
What is your local test environment? Were you just testing this out in a browser console?
response.body from a fetcher returns a JSON object IIRC, or plain text â not a node object. You have to parse the response as XML first, and only then youâll be able to traverse the hierarchy. Also not sure if that would include .innerHTML because thatâs something relevant to DOM only, not any XML.
I have tested it in a browser console, using Keyboard maestro as a custom HTML prompt, and also on the web in JSFiddle and in CodePen (see below). It works in all of those.
While I can write basic Javascript it is not my full-time occupation and my knowledge falls off on the differences between Javascript vs. Node. I am not completely clear on whether for Coda Packs I need to refer to documentation for Javascript or Node or JQuery; one of the errors I got in the Coda online editor suggested I had to use NPM to install JQuery but that did not make much sense to me for a sandboxed environment like that.
Anyway any suggestions on how to change my approach for this to work would be much appreciated.
you tested this in your browser.
and your browser supports the function
getElementsByTagName()
and objects like
innerHTML
but your pack does not run in a browser. it runs on the Coda servers and so has no access to browser functions and objects. this is why it throws that compile-time error.
so instead, you will need look at the response.body and see if it is already an object, or contains xml.
if it contains xml then you will need to convert it into an object and then you will be able refer to the items within it.
in the past i have used the xml2js library to do this, but i dont know if it is available within the coda pack builder environment.
are you sure the API you are using only has an XML endpoint? it is rare these days. there is usually a json option which is native to javascript and thus native to coda packs.
sorry i cannot be more helpful
max
Thanks for the advice.
The API is quite old; it predates virutally all modern programming languages.
It does have limited support for ASN.1 (ASN.1 - Wikipedia) which is a predecessor to JSON and it can produce pure text output as well; but it is not easy to parse either of those for the specific metadata I am seeking.
I believe to use a library such as xml2js I need to use the Command Line Interface rather than the online Pack Studio. Is that correct?
Is it possible to convert the XML to a String without using libraries so I can use Regex to parse out the elements from the XML?
Unless itâs really simple, generally using regex to parse XML is asking for trouble. Youâll want to use a library (fast-xml-parser seems to be a popular one, but I havenât used it so canât specifically endorse it).
Per our documentation, libraries are only available when using the CLI and thatâs generally the recommended approach. We have a guide there on how to start using the Packs CLI.
In some cases, you might be able to find the libraryâs source code (like on this CDN, for instance) and paste the contents into your Pack Studio editor and see if you can use it, but itâs likely youâll run into a bunch of errors youâll need to fix up due to these libraries often being âminifiedâ so they take up less space (and as a result use syntax that the Pack Studio might not like). So sticking with the CLI might be easier in the end for these more advanced use cases where you need libraries.
Thanks - much appreciated
Thatâs quite a spirited argument against using Regex to parse XML 
The source code for the fast-xml-parser via the CDN gives tons of errors when added to the Pack Studio so that does not seem to be viable.
So it appears that command line will be the answer. Thank you.
OK I am following instructions here:
I get to the step where I enter
npx coda execute pack.ts Hello "World"
and it fails - see below. Any suggestions?
Richards-Pro:pubmed-pack richardkaplan$ npx coda init
pubmed-pack@1.0.0 /Users/richardkaplan/github-repositories/pubmed-pack
âââ (empty)
git version 2.38.1
added 1 package, and audited 669 packages in 4s
75 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
up to date, audited 669 packages in 3s
75 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
pubmed-pack@1.0.0 /Users/richardkaplan/github-repositories/pubmed-pack
âââ @codahq/packs-sdk@1.2.3
Unknown command: "set-script"
Did you mean this?
npm run-script # Run arbitrary package scripts
To see a list of supported npm commands, run:
npm help
patch-package 6.5.1
⢠Creating temporary folder
⢠Installing mold-source-map@0.4.1 with npm
⢠Diffing your files with clean files
â Created file patches/mold-source-map+0.4.1.patch
đĄ mold-source-map is on GitHub! To draft an issue based on your patch run
npx patch-package mold-source-map --create-issue
removed 1 package, and audited 668 packages in 2s
75 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
Richards-Pro:pubmed-pack richardkaplan$
Richards-Pro:pubmed-pack richardkaplan$ npx coda execute pack.ts Hello "World"
coda execute <manifestPath> <formulaName> [params..]
Execute a formula
Options:
--version Show version number [boolean]
--help Show help [boolean]
--fetch Actually fetch http requests instead of using mocks. Run
"coda auth" first to set up credentials.
[boolean] [default: true]
--vm Execute the requested command in a virtual machine that
mimics the environment Coda uses to execute Packs.
[boolean] [default: false]
--dynamicUrl For a dynamic sync table with a variable source location,
specify the URL to test here. [string]
--timerStrategy Options: none, error, fake. [string] [default: "none"]
--maxRows For a sync table, the maximum number of rows to sync.
[number] [default: 1000]
TypeError: Cannot set property crypto of #<Object> which has only a getter
at node_modules/@codahq/packs-sdk/dist/testing/injections/crypto_shim.js (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:148:21)
at __init (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:15:58)
at ../../../../private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/browserify-bundle.js (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:155:7)
at __require2 (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:18:52)
at /private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:8663:10
at Object.<anonymous> (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:8664:3)
at Module._compile (node:internal/modules/cjs/loader:1246:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1300:10)
at Module.load (node:internal/modules/cjs/loader:1103:32)
at Module._load (node:internal/modules/cjs/loader:942:12)
Richards-Pro:pubmed-pack richardkaplan$
I ran the above on my local machine
Separately I followed the instructions to clone your sample Github repository and set up an online Codespace. I did that got basically the identical error:
Richard
I have not used XML for about a decade, but I did do a lot of XML based work back in the day when it was all-the-rage.
While it is nice to be able to parse the whole XML tree and then use X-Path to select the parts you want to extract, for many cases you dont need to be that sophisticated.
You can always use simple regex to find the contents of a single tag, or multiple occurances of a single tag - if you know the tag-name is unique and you dont need to bother with the position of that tag in the overall structure. You just scan the whole XML tree looking for anything between <myTag> and </myTag>, for example.
If you can share a sample of the XML you are working with and the tag/tags you are looking to extract, then I could suggest some regex patterns to find what you are looking for.
If you prefer to not âpublishâ the XML, you can message me on this platform or email max.xyzor@agile-dynamics.com
Thanks @Agile_Dynamics
It is a public/free API so there are no worries about âpublishing itâ
An example is here:
I do think I could use Regex for the key items I am looking for - such as <ArticleTitle> and <Abstract>. The complicated parts are the authors since there can be one or many; the nested structure can get quite complex. But I donât really need that for my purposes for this project.
But the problem is this - I succeeded with the online Pack Studio at importing the XML into Coda.io. But Coda.io does not display all of the fields; it creates what appears to be a card with limited information but not the most important fields that I need.
If there is a way for me to stick with Pack Studio and import the entire XML document as a text document, then I believe I would be fine using Regex to parse out the data I am looking for.
Some simple options areâŚ
(a) you can return the entire XML from your pack as a string by returning response.body. Then you can use the various Regex functions available in Coda to scan for and extract the items you want
(b) you can write javascript regex code inside your pack to extract the items you want and return them as an array of strings from your pack
Personally, I prefer (a) as I can easily use the Coda formula language to look for the item i want, so I would just use the pack to execute the API request and hand back the raw response to me in Coda
In the simplest case, you could use myXML.Split(âArticleTitle>â) to get a list of texts where every second one is an ArticleTitle.
myXML.Split('ArticleTitle>').WithName(L, // split into list L
Sequence(2,Count(L),2) // take every 2nd item
.ForEach(
L.Nth(CurrentValue).Left(-2) // strip off last 2 chars '</'
)
).NumberedList() // remove this, its just for demo purposes
Do the same again with Split(âAubstract>â) to collect the abstracts.
For recurring items within larger items, it can be done as well, but gets more complicated and is lots of fun.
Let me know if you want me to have a go at the nested Authors within each Article
Respect
Max
Hi @Richard_Kaplan - I think there may be a simpler solution to this problem. As mentioned in the documentation, the Fetcher will automatically parse XML responses into a JavaScript object using the xml2js library:
This means you should be able to use the Pack Studio, since no libraries are required. If you do want to continue to use the CLI, that specific error is due to an incompatibility with Node version 19. We have a fix in place but had to pause releasing it last week. That fix should go live this week. In the mean time you can downgrade to Node version 18 to resolve that crypto error.
aha!
that explains why you are seing a âcardâ displayed.
its coda displaying a complex object
so you are getting an object that is easy to parse in coda or in javascript
OK - thank you both - @Eric_Koleda and @Agile_Dynamics - those are both excellent solutions and I suspect will work. It turns out I will be traveling much of this week so may not be able to resume this project for a week or two. But I will ponder both ideas in the interim and will definitely get back to you when I resume working on this. Much appreciated.
** Deleted Original Post **
@Eric_Koleda and @Agile_Dynamics
Sorting out the XML formatting seemed tough to do using the web app interface for Pack Studio.
The Node command line interface for Coda packs has now has been fixed - I was able to get the Pack working that way - which gives a lot more flexibility in terms of other libraries I may need in the future.
Thanks for your help!