How I check what an AI claims.
You cannot tell from an AI answer whether it is true. So I build systems that show a claim only if its quote appears word for word in the source.
The risk
An AI that reads a long document answers in good sentences. Some of what it says is in the document, and some only sounds as if it were. From the answer you cannot tell which.
You cannot rely on that when the document is a contract, a tender or a regulation.
A quote for every fact
The model has to attach a word-for-word quote from a source to every fact it states. A separate step without AI then searches the stored text of the source for the exact quote. The result shows each claim with its quote.
A conclusion the model draws from several facts is kept only if every one of those facts passed.
What happens to the rest
A claim whose quote is not in the source is left out. A claim that can be neither confirmed nor ruled out is listed separately. The result also lists what the question needed and the sources did not show.
Which sources count
Your own documents and official sources, such as a register or a regulator, come first. Everything else from the web comes second and is marked as such in the result.
An example: ThesisCheck
ThesisCheck is my product that tests an investment thesis against a company's filings. It checks every claim this way, on filings from the US, the UK, Australia and Japan. A report is typically ready in about 5 minutes. An analyst needs hours to read the same filings.
What you can use
The same steps work for any AI that reads long documents for you, such as contracts, tenders, regulations or manuals.
- Store the source text yourself, so there is something to check against
- Have the model quote word for word for every fact
- Check each quote against the source with code
- List what could not be confirmed