Is there any way to view pack execution/error logs of Pack users by the pack makers/owners ?
I can see that users got error (in Pack stats) but not sure what goes wrong. Unless users email me or contact me with error info, I have no way to know.
If we can see the error logs (not sensitive parameters or data) , we can make the Pack better & fix the bugs. I am not sure if we can see the logs now. I couldn’t find it.
Hi @Rupjyoti_Nath - Unfortunately no, this isn’t possible today. Because the logs can contain sensitive information users must opt-in by first sharing the doc with the Pack maker. We recognize this makes it more difficult to maintain your Pack and improve quality, but we need to balance both the needs of the Pack maker and user.
Hi @Eric_Koleda Thank you for your reply. I understand the privacy of the data. I was just curious to know if any way exists to get some logs.
In future I do think that if possible Coda can analyze the source code of logs of a Pack & then allow Pack makers to view the approved logs. I am not sure how difficult would this be to implement. It’s just my thought.
Anyway, users can contact the makers for any bug or issue, so we can fix it without getting the actual logs. Also , they can share the doc as you suggested. That seems okay for now.
I should clarify: if a user shares their doc with you, you can then open the Pack Maker tools in that doc and see their logs. So it is possible to see the user’s logs, but only if they share the doc with you first.
The best way right now is asking the user to share a doc reproducing the issue, but as you mentioned the user will have to contact you first!
Maybe in the future Coda can prompt users that experience errors to contact the Pack Maker or send a bug report?
Understood Eric. Thanks.
Hi Leandro, that’s an awesome idea. Users can then easily provide error/bug info & they won’t hesitate to report tiny issues as well.
Perhaps I’m under-simplifying this answer, but isn’t the pathway to deep analytics a superset of errors? Shouldn’t the Pack owners know about failures as and when the users encounter them?
I have a hunch that the right way to support users who encounter errors is to:
Hi Bill,
I already use User Visible Error but still I don’t know about the error if user doesn’t reach out to me.
Regarding temp blob storage, I have used in my Pack to return file. For logs , it’s definitely a very good idea to get a file of logs. Then again, the user has to provide the logs.
Regarding saving logs to another service, it’s also a good idea, actually could be best idea because users don’t need to contact you & you still get the logs. But, I am worried about the Privacy of the users. Of course Coda shows the list of domains to user but getting data out of Coda to a 3rd party service even if the Pack is not an integration of this service, would force the users to think about using the Pack & will worry about the Privacy of the data that goes out to the logging service. It is possible though with a proper Terms of Service & Privacy Policy provided by the Pack maker/owner.
Coda might need to verify the source code to see if it gets any sensitive user data.
This is a reasonable concern. However, analytics about an internal code failure is arguably your data, not the user’s data. I believe all Pack Makers have a right to know who [exactly] is using your Pack. Installing a Pack requires Coda identity and by extension, the Pack installer’s identity. So that ship has sailed long before we capture identity in a failure log. The wrinkle is that with custom Packs, the number of parties getting user activity is expanding as we chat.
There’s also your (and Coda’s) right to examine logs for nefarious activities; failures are not always the outcome of unintentional actions.
The entire privacy envelope has been overplayed to encompass every aspect of technology. This is an over-reach by compliance authorities and users who not only want the benefits of discrete identity, but demand these benefits and all while expecting complete anonymity.
In my view, all Packs should gather analytics if for no other reason than to thwart security attacks, which ultimately are intended to protect user privacy.
In general there is a lot more that Coda can do to provide Pack makers with the tools and information they need to maintain their Packs and ensure quality. I can appreciate the challenge of getting a report with error rates but not being able to investigate those errors proactively.
However a key part of Pack security is ensuring that Pack makers can’t access the data from their users, so any approach we took towards error reporting would need to be carefully thought through. I don’t doubt a solution can be found, but it will likely be a non-trivial amount of work, and not something we are actively working on at this time.
That said, is it Coda’s position that Pack Makers should try to capture analytics to provide good support to their Pack users? And if so, what are considered best practices or guidelines? Perhaps a simple policy doc that frames a recommended strategy could be also used as boiler-plate for every Pack privacy policy.
There is one additional approach that I have tested and it seems like a way to avoid the privacy issue. Instead of sending failure analytics to another service, send them to the same Coda document where the Pack is installed. This should dispel any concerns that your data is being harvested while also making the log transparent only to the parties who have authenticated access.
Such a Coda-based log could also include preconfigured features that allow users to share the logs to Pack Makers.