Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

Abstract: The rapid evolution of software systems has made traditional testing methods unsuitable to provide quality, speed, and responsiveness to applications in modern development. This paper ...

Abstract: The growing complexity of software systems and the need for more rapid, high-quality software releases have created the need for intelligent and automated testing mechanisms. Drawing on ...