AI Truth Monitoring and Fact Checking?

[Bill_French]

One of the biggest challenges and fears that come with AI is a concern for content safety and truthfulness. The advent of easily accessible AI features has added a new risk surface in almost every solution that embraces AI for productivity and other benefits.

We have a new writer on the team who’s been known to make stuff up.

As is often the case for all new technology, it tends to improve. But until it does, we may need to monitor this new team member in some circumstances.

With that looming crisis for CSOs in mind I managed to make this work in Coda for Coda AI.

To be clear, these are just sample demos - AI content on the left, and VERIFAI’s interpretation of it on the right. It is not intended to be used this way, but this is an ideal way to show what it does. However, fact and safety verifications ARE intended to be used by whomever can benefit from them and wherever it makes sense. Of course, Coda makes this easily possible.

Regardless of your testing and monitoring protocols, AI issues are difficult to mitigate; you can never guarantee perfection, so plan how to spot and deal with problems that arise upfront. Common approaches include setting up a monitored channel for users to share feedback (e.g., thumbs up/down rating) and running a user study to proactively solicit feedback from a diverse mix of users — especially valuable if usage patterns are different from expectations.

Using the VERIFAI Pack, it is possible to monitor AI components in the background and deliver assessments through other packs or webhooks to other data targets. Imagine a log of instances where fiction has been presented in Coda documents by AI components that were unintended. I’m pretty sure that’s exactly what management, Makers, and CSOs are needing as potentially poorly-constructed AI prompts are about emerge across Coda document systems.

VERIFAI provides the underlying framework to begin to start building AI features while embracing productive approaches to ensuring safe and healthy AI outputs. I’m happy to chat with anyone who has more stringent compliance and monitoring requirements.

My question to the community - is this needed?

[Xyzor_Max]

@Bill_French

VERIFAI

@Bill_French

i was travelling when this was published, so i am only catching-up on the backlog now.

i love what you are showing here, it provides a lot of useful information about the AI result grom a prompt.

so, YES, it is needed!

have you published this pack or tool ? (cant find it in the gallery !)

any pointers on how to implement?

max

[Bill_French]

@Xyzor_Max

have you published this pack or tool ? (cant find it in the gallery !)

This was going to be a hackathon submission but I decided to shelve it because no one seemed to care (at the time) and it is based on PaLM 2.

@Xyzor_Max

any pointers on how to implement?

There are a few challenges asking Coda AI to check itself (rats → cheese, etc). And the idea of asking an independent LLM to check another LLM is dicey but likely less dicey when using PaLM 2 since it has seven safety features built into it.

I think this idea has merit, but the amount of effort to do this in the Coda ecosystem outweighs the likely return on that investment. Money needs to be chasing this idea before I’d stop doing other stuff to work on this concept.