Introduction. In regulated quality assurance (QA) workflows, companies must demonstrate that work has been performed according to approved procedures. Procedure documents define requirements, while record documents provide evidence of performed activities. Reviewing this relationship manually can be slow, detailed, and difficult to scale, especially when evidence is incomplete, fragmented, or distributed across document sections.
Research questions. This thesis investigates how a retrieval-augmented and evidence-audited LLM framework can support requirement-level compliance assessment between procedure-derived requirements and record-document evidence. The project focuses on grounded requirement-level assessment, differences between non-retrieval, record-retrieval, and reference-assisted stages, and the role of deterministic evidence auditing in identifying supported, partial, missing, weakly matched, and contradicted evidence.
Methods. The thesis is designed as an industrial case study with a mixed methods evaluation. Document analysis is the primary method, supported by expert-informed review and quantitative comparison of framework outputs. A research prototype was implemented and evaluated using one procedure document, original company records, synthetic records, and one referenced supporting document. The framework includes requirement extraction, staged assessment, evidence grounding, deterministic audit, and metric-based evaluation.
Results. The results show that retrieval changes the quality of requirement-level assessment by increasing access to candidate record evidence and improving traceability to source documents. The record-retrieval stage provided a clearer connection between requirements and record evidence than the non-retrieval baseline. The reference-assisted stage was useful when requirement interpretation depended on the referenced supporting document. The deterministic audit helped separate grounded evidence from unsupported generated claims and made missing, partial, weakly matched, and contradicted cases easier to identify.
Discussion. The findings suggest that retrieval-augmented and evidence-audited LLM frameworks can support QA documentation review, but their outputs should remain subject to human review. The project is limited by the size and scope of the original dataset, the use of synthetic records, document-processing choices, and the selected LLM backends. Future work should evaluate the framework on larger and more varied QA documentation, improve retrieval and audit methods, and study practical integration into regulated review workflows.