AWS Flash – Operationalize Generative AI Applications (FMOps/LLMOps)

Module 1: Introduction to LLMOps

Introduction to LLMOps

LLMOps Roles

Challenges in operationalizing LLMs

Module 2: LLM Selection

Use-case benchmarking of LLMs

Priority-based decision making

Module 3: LLM Evaluation

Evaluation methods

Evaluation prompt catalog

Evaluation framework and metrics

Benchmarking framework and metrics

Module 4: Retrieval Augmented Generation (RAG)

LLM customization

Embedding models

Vector databases

RAG workflows

Advanced RAG techniques

Module 5: LLM Fine-tuning

Continual pre-training vs. fine-tuning

Parameter-efficient fine-tuning (PEFT)

Fine-tuning architecture

Module 6: LLM Monitoring

LLM monitoring

LLM guardrails

Module 7: LLM Governance and Security

Security and governance best practices

Security and governance tools

Module 8: LLMOps Architecture

LLMOps lifecycle