Authors

Amenah Alghanim, Winfred Videkar

Abstract

Verity AI is an AI-powered verification tool for flagging, checking, and correcting factual claimsin text and uploaded documents. It responds to the difficulty researchers and other accuracydependentprofessionals face when assessing model-generated or documentary information.The project focuses on reducing hallucination, nondeterministic answers, and fabricatedreferences by combining repeated consistency checks, claim-level analysis, evidence retrieval,source assessment, and confidence scoring. Intended users include academic researchers,legal professionals, technical writers, and content teams that need traceable support for factualstatements.

Verity AI was presented through a functioning website prototype connected to a backendverification pipeline. A user can paste text or upload a document, after which the systemhandles the material as individual claims rather than asking one large model to process theentire submission. Requests are sent through the GroqCloud SDK, selected to address theapproximately 30-second delays the team had encountered and to improve interaction speed.

In the first model stage, Llama 3.1, configured with a temperature of zero, reads the raw textand converts it into a structured JSON array of claims. These claims pass to a three-tierevidence-retrieval process. The first tier searches Semantic Scholar Graph for peer-reviewedpublication metadata, while the second uses Unpaywall to locate openly accessible researchpapers. An additional fallback search is triggered if the earlier tiers return no usable evidence.Llama 3.3, also set to zero temperature, performs deeper comparison and final classification,producing a correct, incorrect, or uncertain outcome.

The website contains separate consistency and fact-checking sections. The consistencyfunction repeats the same prompt through independent runs and compares the responses,while the fact checker evaluates pasted text or uploaded material and can display a correction with its supporting reference. Confidence combines contextual entailment, weighted at 60%,with source reliability, weighted at 40%. Peer-reviewed publications receive the strongestsource weighting, followed by government and educational websites, while commercial sitesreceive lower scores. The presentation described an acceptance threshold of approximately70%, below which a response is automatically rejected.

license

Creative Commons BY-SA 4.0