Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
We’re entering a new renaissance of software development. We should all be excited, despite the uncertainties that lie ahead.