Description
RAG-LLM Evals & Test Automation for Beginners. This course introduces beginners to the fundamentals of automated evaluation and testing of large language models (LLMs) with an incremental retrieval-generation (RAG) architecture. As these models become increasingly popular in businesses, there is a need for a specific methodology to measure their performance. The course begins by explaining the architecture and mechanism of language models and then explores key evaluation metrics. Participants are introduced to the specialized RAGAS framework and learn how to write scripts to calculate evaluation metrics and validate the results using Pytest. The course covers seven commonly used metrics and teaches how to design an automated testing framework for single- and multi-loop interactions. It also provides methods for generating test data suitable for evaluation. The ultimate goal is to equip engineers with the skills necessary to create robust and automated evaluation systems for RAG-based language models.
What you will learn
- Introduction to designing custom LLMs:
- How to design custom large language models (LLM) using the Retrieval Augmented Generation (RAG) architecture.
- Evaluation criteria:
- Common benchmarks/criteria used in evaluating RAG-based LLMs.
- Introduction to RAGAS:
- Introducing the RAGAS assessment framework for assessing/testing LLMs.
- Generating practical scripts:
- Generate practical scripts to automate and verify the scoring of LLMs criteria.
- Automate scenarios:
- Automate scenarios such as single turn interactions and multi turn interactions with LLMs using the RAGAS framework.
- Test data generation:
- Generating test data for evaluating LLM criteria using the RAGAS framework.
- Creating the Pytest RAGAS framework:
- Creating the Pytest RAGAS evaluation framework to verify the criteria of (custom) RAG LLMs.
This course is suitable for people who:
- Software engineers.
- Quality Assurance Engineers.
- Software testers/experimenters.
Course details
- Publisher: Udemy
- Instructor: Rahul Shetty Academy
- Training level: Beginner to advanced
- Training duration: 8 hours and 39 minutes
- Number of lessons: 57
Course syllabus in 2025/9
RAG-LLM Evals & Test Automation for Beginners Course Prerequisites
- Python, PyTest basics are required. But wait! we have 2 dedicated sections at the end of this course which gives you the necessary knowledge on Python & Pytest required to follow the course.
- Basic knowledge on API Testing
Course images
Sample course video
Installation Guide
After Extract, view with your favorite player.
Subtitles: None
Quality: 720p
Download link
Downloadly
Rapidgator link
File(s) password: www.downloadly.ir
File size
3.7 GB
