xml.getELementsByTagName is not a function

I am building a Pack which retrieves XML data from an API

Most of the Pack is working. However I am able to parse the XML contents using Javascript in my local test environment but that does not work within the Pack.

This is the pertinent code

var xml = response.body;

var abstract = xml.getElementsByTagName('AbstractText')[0].innerHTML;

When I try to use this code in my Pack I get this error:

xml.getELementsByTagName is not a function

How do I fix this?

What is your local test environment? Were you just testing this out in a browser console?

response.body from a fetcher returns a JSON object IIRC, or plain text — not a node object. You have to parse the response as XML first, and only then you’ll be able to traverse the hierarchy. Also not sure if that would include .innerHTML because that’s something relevant to DOM only, not any XML.

I have tested it in a browser console, using Keyboard maestro as a custom HTML prompt, and also on the web in JSFiddle and in CodePen (see below). It works in all of those.

While I can write basic Javascript it is not my full-time occupation and my knowledge falls off on the differences between Javascript vs. Node. I am not completely clear on whether for Coda Packs I need to refer to documentation for Javascript or Node or JQuery; one of the errors I got in the Coda online editor suggested I had to use NPM to install JQuery but that did not make much sense to me for a sandboxed environment like that.

Anyway any suggestions on how to change my approach for this to work would be much appreciated.

you tested this in your browser.
and your browser supports the function
getElementsByTagName()
and objects like
innerHTML

but your pack does not run in a browser. it runs on the Coda servers and so has no access to browser functions and objects. this is why it throws that compile-time error.

so instead, you will need look at the response.body and see if it is already an object, or contains xml.

if it contains xml then you will need to convert it into an object and then you will be able refer to the items within it.

in the past i have used the xml2js library to do this, but i dont know if it is available within the coda pack builder environment.

are you sure the API you are using only has an XML endpoint? it is rare these days. there is usually a json option which is native to javascript and thus native to coda packs.

sorry i cannot be more helpful

max

Thanks for the advice.

The API is quite old; it predates virutally all modern programming languages.

It does have limited support for ASN.1 (ASN.1 - Wikipedia) which is a predecessor to JSON and it can produce pure text output as well; but it is not easy to parse either of those for the specific metadata I am seeking.

I believe to use a library such as xml2js I need to use the Command Line Interface rather than the online Pack Studio. Is that correct?

Is it possible to convert the XML to a String without using libraries so I can use Regex to parse out the elements from the XML?

Unless it’s really simple, generally using regex to parse XML is asking for trouble. You’ll want to use a library (fast-xml-parser seems to be a popular one, but I haven’t used it so can’t specifically endorse it).

Per our documentation, libraries are only available when using the CLI and that’s generally the recommended approach. We have a guide there on how to start using the Packs CLI.

In some cases, you might be able to find the library’s source code (like on this CDN, for instance) and paste the contents into your Pack Studio editor and see if you can use it, but it’s likely you’ll run into a bunch of errors you’ll need to fix up due to these libraries often being “minified” so they take up less space (and as a result use syntax that the Pack Studio might not like). So sticking with the CLI might be easier in the end for these more advanced use cases where you need libraries.

Thanks - much appreciated

That’s quite a spirited argument against using Regex to parse XML :slight_smile:

The source code for the fast-xml-parser via the CDN gives tons of errors when added to the Pack Studio so that does not seem to be viable.

So it appears that command line will be the answer. Thank you.

1 Like

OK I am following instructions here:

I get to the step where I enter

npx coda execute pack.ts Hello "World"

and it fails - see below. Any suggestions?

Richards-Pro:pubmed-pack richardkaplan$ npx coda init
pubmed-pack@1.0.0 /Users/richardkaplan/github-repositories/pubmed-pack
└── (empty)

git version 2.38.1

added 1 package, and audited 669 packages in 4s

75 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

up to date, audited 669 packages in 3s

75 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
pubmed-pack@1.0.0 /Users/richardkaplan/github-repositories/pubmed-pack
└── @codahq/packs-sdk@1.2.3

Unknown command: "set-script"

Did you mean this?
    npm run-script # Run arbitrary package scripts

To see a list of supported npm commands, run:
  npm help
patch-package 6.5.1
• Creating temporary folder
• Installing mold-source-map@0.4.1 with npm
• Diffing your files with clean files
✔ Created file patches/mold-source-map+0.4.1.patch

💡 mold-source-map is on GitHub! To draft an issue based on your patch run

    npx patch-package mold-source-map --create-issue


removed 1 package, and audited 668 packages in 2s

75 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Richards-Pro:pubmed-pack richardkaplan$ 
Richards-Pro:pubmed-pack richardkaplan$ npx coda execute pack.ts Hello "World"
coda execute <manifestPath> <formulaName> [params..]

Execute a formula

Options:
  --version        Show version number                                 [boolean]
  --help           Show help                                           [boolean]
  --fetch          Actually fetch http requests instead of using mocks. Run
                   "coda auth" first to set up credentials.
                                                       [boolean] [default: true]
  --vm             Execute the requested command in a virtual machine that
                   mimics the environment Coda uses to execute Packs.
                                                      [boolean] [default: false]
  --dynamicUrl     For a dynamic sync table with a variable source location,
                   specify the URL to test here.                        [string]
  --timerStrategy  Options: none, error, fake.        [string] [default: "none"]
  --maxRows        For a sync table, the maximum number of rows to sync.
                                                        [number] [default: 1000]

TypeError: Cannot set property crypto of #<Object> which has only a getter
    at node_modules/@codahq/packs-sdk/dist/testing/injections/crypto_shim.js (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:148:21)
    at __init (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:15:58)
    at ../../../../private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/browserify-bundle.js (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:155:7)
    at __require2 (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:18:52)
    at /private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:8663:10
    at Object.<anonymous> (/private/var/folders/k2/qk1vv2g101x5lg2rxsp_vyw80000gn/T/coda-packs-4e7a603b-740d-461a-b732-68435f8378a4Yil87u/bundle.js:8664:3)
    at Module._compile (node:internal/modules/cjs/loader:1246:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1300:10)
    at Module.load (node:internal/modules/cjs/loader:1103:32)
    at Module._load (node:internal/modules/cjs/loader:942:12)
Richards-Pro:pubmed-pack richardkaplan$

I ran the above on my local machine

Separately I followed the instructions to clone your sample Github repository and set up an online Codespace. I did that got basically the identical error:

Richard

I have not used XML for about a decade, but I did do a lot of XML based work back in the day when it was all-the-rage.

While it is nice to be able to parse the whole XML tree and then use X-Path to select the parts you want to extract, for many cases you dont need to be that sophisticated.

You can always use simple regex to find the contents of a single tag, or multiple occurances of a single tag - if you know the tag-name is unique and you dont need to bother with the position of that tag in the overall structure. You just scan the whole XML tree looking for anything between <myTag> and </myTag>, for example.

If you can share a sample of the XML you are working with and the tag/tags you are looking to extract, then I could suggest some regex patterns to find what you are looking for.

If you prefer to not ‘publish’ the XML, you can message me on this platform or email max.xyzor@agile-dynamics.com

Thanks @Agile_Dynamics

It is a public/free API so there are no worries about “publishing it”

An example is here:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=30090998&retmode=xml&rettype=abstract

I do think I could use Regex for the key items I am looking for - such as <ArticleTitle> and <Abstract>. The complicated parts are the authors since there can be one or many; the nested structure can get quite complex. But I don’t really need that for my purposes for this project.

But the problem is this - I succeeded with the online Pack Studio at importing the XML into Coda.io. But Coda.io does not display all of the fields; it creates what appears to be a card with limited information but not the most important fields that I need.

If there is a way for me to stick with Pack Studio and import the entire XML document as a text document, then I believe I would be fine using Regex to parse out the data I am looking for.

Some simple options are…

(a) you can return the entire XML from your pack as a string by returning response.body. Then you can use the various Regex functions available in Coda to scan for and extract the items you want

(b) you can write javascript regex code inside your pack to extract the items you want and return them as an array of strings from your pack

Personally, I prefer (a) as I can easily use the Coda formula language to look for the item i want, so I would just use the pack to execute the API request and hand back the raw response to me in Coda

In the simplest case, you could use myXML.Split(“ArticleTitle>”) to get a list of texts where every second one is an ArticleTitle.

myXML.Split('ArticleTitle>').WithName(L,  // split into list L
  Sequence(2,Count(L),2)                  // take every 2nd item
  .ForEach(
    L.Nth(CurrentValue).Left(-2)          // strip off last 2 chars '</'
  )
).NumberedList()                          // remove this, its just for demo purposes

Do the same again with Split(‘Aubstract>’) to collect the abstracts.

For recurring items within larger items, it can be done as well, but gets more complicated and is lots of fun.

Let me know if you want me to have a go at the nested Authors within each Article

Respect
Max

Hi @Richard_Kaplan - I think there may be a simpler solution to this problem. As mentioned in the documentation, the Fetcher will automatically parse XML responses into a JavaScript object using the xml2js library:

This means you should be able to use the Pack Studio, since no libraries are required. If you do want to continue to use the CLI, that specific error is due to an incompatibility with Node version 19. We have a fix in place but had to pause releasing it last week. That fix should go live this week. In the mean time you can downgrade to Node version 18 to resolve that crypto error.

2 Likes

aha!

that explains why you are seing a “card” displayed.
its coda displaying a complex object

so you are getting an object that is easy to parse in coda or in javascript

OK - thank you both - @Eric_Koleda and @Agile_Dynamics - those are both excellent solutions and I suspect will work. It turns out I will be traveling much of this week so may not be able to resume this project for a week or two. But I will ponder both ideas in the interim and will definitely get back to you when I resume working on this. Much appreciated.

1 Like

** Deleted Original Post **

@Eric_Koleda and @Agile_Dynamics

Sorting out the XML formatting seemed tough to do using the web app interface for Pack Studio.

The Node command line interface for Coda packs has now has been fixed - I was able to get the Pack working that way - which gives a lot more flexibility in terms of other libraries I may need in the future.

Thanks for your help!

2 Likes