Name: PHAROS Training Series - Course 9 "RAG End-to-End: Architecture, Retrieval, Generation and Evaluation"
Start: 2026-07-07T11:00:00+03:00
End: 2026-07-07T15:15:00+03:00
Location: No location set

PHAROS Training Series - Course 9 "RAG End-to-End: Architecture, Retrieval, Generation and Evaluation"

Tuesday, 7 July 2026 - 11:00

Monday, 6 July 2026
Tuesday, 7 July 2026
11:00 Why RAG? Hallucinations, knowledge cut-offs and grounded generation - George Drosatos (ATHENA RC)
Why RAG? Hallucinations, knowledge cut-offs and grounded generation
- George Drosatos (ATHENA RC)
11:00 - 11:15
This opening session explains why Retrieval-Augmented Generation is needed when Large Language Models must answer with factual, current and verifiable information. It introduces the main limitations of standalone LLMs, including static training knowledge, knowledge cut-offs, hallucinations and weak handling of specialised domain terminology. Using Greek public-service information as a motivating example, the session shows how RAG changes the task from generating from model memory to answering from retrieved evidence. Participants will see how an external knowledge source can support more transparent, context-aware responses, especially in domains where users need reliable guidance, citations and clear caveats. The session establishes the central concepts used throughout the webinar: user query, retrieval, evidence, grounded answer and source attribution. By the end, participants will understand the practical motivation for RAG and why retrieval quality matters before any answer is generated by the model. It clearly frames RAG as a reliability pattern for specialised knowledge-intensive applications.
11:15 RAG architectures: Naïve, Advanced and Modular - George Drosatos (ATHENA RC)
RAG architectures: Naïve, Advanced and Modular
- George Drosatos (ATHENA RC)
11:15 - 11:40
This theory session presents the main architectural paradigms used in Retrieval-Augmented Generation systems. It starts with Naïve RAG as the baseline pipeline: document ingestion, chunking, embedding generation, vector indexing, retrieval and grounded answer generation. It then explains how Advanced RAG improves this baseline through better preprocessing, metadata enrichment, query rewriting, hybrid retrieval, reranking, context compression and evidence filtering. The session also introduces Modular RAG, where routing, rewriting, retrieval, fusion, generation and verification can be organised as configurable components rather than a fixed linear chain. Participants will learn how each architecture changes the quality, reliability, latency and maintainability of a RAG application. The session prepares participants to recognise which design pattern is appropriate for a simple educational prototype, a more robust domain assistant or a production system that must adapt to different question types and knowledge sources. It also clarifies how the same concepts appear later in the hands-on notebook exercises.
11:40 Hands-on 1: dataset loading, document preparation and first RAG pipeline overview - Sotiris Gyftopoulos (ATHENA RC)
Hands-on 1: dataset loading, document preparation and first RAG pipeline overview
- Sotiris Gyftopoulos (ATHENA RC)
11:40 - 12:10
This first hands-on tutorial introduces the Google Colab notebook and the running example used throughout the practical part of the webinar. Participants will load a small Greek-language public-service dataset and inspect how documents are represented before indexing. The session focuses on orientation rather than optimisation: understanding the structure of the corpus, identifying the available fields, examining source information and seeing how raw documents become inputs to a RAG pipeline. Participants will review the full notebook flow, from setup and data loading to chunking, embeddings, retrieval, answer generation and evaluation. They will also inspect intermediate outputs so that the pipeline remains transparent and debuggable. By the end of the session, participants will understand the dataset, the role of metadata, the overall sequence of implementation steps and how the notebook will be used to build a practical RAG system progressively. This foundation helps them follow later implementation choices with confidence during exercises.
12:10 Ingestion, chunking, metadata and embeddings for RAG - George Drosatos (ATHENA RC)
Ingestion, chunking, metadata and embeddings for RAG
- George Drosatos (ATHENA RC)
12:10 - 12:35
This methodology session focuses on preparing knowledge so that it can be retrieved accurately by a RAG system. It explains why document preparation is not a minor preprocessing task but a core design decision that affects downstream retrieval and answer quality. Participants will learn how text can be extracted from PDFs, Markdown, HTML or structured files while preserving headings, sections, tables, article numbers, source identifiers and other traceable information. The session then introduces chunking strategies, including fixed-size, overlapping, semantic and structure-aware chunking, and discusses how chunk size influences retrieval precision and context completeness. It also covers metadata fields that support filtering, citation and source attribution. Finally, the session explains embeddings and vector indexing, with emphasis on multilingual or Greek-aware representations, index choice, latency, scaling, filtering and maintainability in prototype and production RAG settings. These foundations prepare participants for the following indexing and retrieval implementation exercises in Colab, including practical debugging.
12:35 Hands-on 2: chunking, metadata and vector indexing - Sotiris Gyftopoulos (ATHENA RC)
Hands-on 2: chunking, metadata and vector indexing
- Sotiris Gyftopoulos (ATHENA RC)
12:35 - 12:50
This second hands-on tutorial applies the knowledge preparation concepts introduced in the methodology session. Participants will create document chunks from the loaded public-service dataset, attach useful metadata and generate embeddings for each chunk. The session demonstrates how metadata such as document title, section, source identifier and chunk position helps preserve traceability and makes later retrieval more controllable. Participants will then build a local vector index in the Colab environment and run an initial similarity search to inspect the retrieved evidence. The emphasis is on visibility and debugging: participants will read example chunks, check whether they are coherent, verify that metadata is correct and examine whether the first retrieval results are meaningful. By the end, participants will have a working indexed knowledge base and will understand how data preparation decisions shape everything that follows in the RAG pipeline. This result becomes the basis for hybrid retrieval, reranking and answer generation exercises.
12:50 Break
Break
12:50 - 13:05
13:05 Retrieval strategies: semantic, lexical, hybrid retrieval, HyDE, RRF and reranking - George Drosatos (ATHENA RC)
Retrieval strategies: semantic, lexical, hybrid retrieval, HyDE, RRF and reranking
- George Drosatos (ATHENA RC)
13:05 - 13:30
This methodology session examines the retrieval stage, which determines what evidence the language model can use when generating an answer. It compares dense semantic retrieval, sparse keyword retrieval, structured retrieval and hybrid retrieval, explaining when each modality is useful. Participants will see why semantic search is valuable for paraphrases and user-friendly language, while keyword or BM25-style search is important for exact administrative terms, identifiers, article numbers, dates and rare phrases. The session also introduces query enhancement techniques such as rewriting, expansion and HyDE, which can bridge the gap between conversational user queries and formal source language. It then explains fusion, Reciprocal Rank Fusion and reranking as ways to combine complementary retrieval signals and improve the final Top-K evidence. Participants will understand how retrieval choices affect recall, precision, context noise, latency and final answer reliability. The session prepares participants to experiment with these choices in tutorial exercises using realistic Greek queries.
13:30 Hands-on 3: semantic search, BM25/hybrid retrieval and reranking - Sotiris Gyftopoulos (ATHENA RC)
Hands-on 3: semantic search, BM25/hybrid retrieval and reranking
- Sotiris Gyftopoulos (ATHENA RC)
13:30 - 13:55
This third hands-on tutorial lets participants compare retrieval strategies directly in the Colab notebook. Using representative Greek-language public-service questions, participants will run semantic search, keyword-based search and hybrid retrieval over the same indexed dataset. They will inspect returned chunks, compare ranking behaviour and identify cases where one retrieval strategy succeeds while another misses important evidence. The session also introduces practical analysis of false positives, missing evidence and noisy context. Participants will apply reranking to candidate chunks and compare the Top-K context before and after reranking, observing how ordering affects the evidence available to the generator. The tutorial encourages participants not to trust retrieval scores blindly, but to read and evaluate the retrieved passages. By the end, participants will understand how to choose retrieval strategies for different query types and how to diagnose retrieval weaknesses systematically. These observations support later grounding, citation checking and end-to-end answer evaluation activities in the notebook.
13:55 Generation in RAG: context construction, grounding, citations and answer constraints - George Drosatos (ATHENA RC)
Generation in RAG: context construction, grounding, citations and answer constraints
- George Drosatos (ATHENA RC)
13:55 - 14:20
This methodology session explains how retrieved evidence is transformed into a grounded answer through prompt construction and generation. Participants will learn the main components of a grounded RAG prompt: the user query, retrieved evidence, system instructions and required answer format. The session discusses how prompt instructions can constrain the model to answer only from available sources, cite supporting evidence, avoid unsupported claims and state when the retrieved context is insufficient. It also covers context construction decisions, including how much evidence to include, how to order passages and how to separate facts from caveats. Special attention is given to source attribution and citation granularity, particularly in public-service or administrative domains where users need traceability. By the end, participants will understand that generation quality depends not only on the LLM, but also on retrieval quality, prompt design and evidence organisation. These principles are then implemented in the following hands-on end-to-end RAG tutorial.
14:20 Hands-on 4: end-to-end RAG answer generation with source attribution - Sotiris Gyftopoulos (ATHENA RC)
Hands-on 4: end-to-end RAG answer generation with source attribution
- Sotiris Gyftopoulos (ATHENA RC)
14:20 - 14:40
This fourth hands-on tutorial connects retrieval and generation into an end-to-end RAG chain. Participants will start from a user question, retrieve relevant chunks, build a grounded prompt, call the selected language model and return an answer with source attribution. The notebook will expose each intermediate step so participants can see exactly what the model receives before generating a response. They will compare answers produced with and without retrieved context, highlighting the role of evidence in reducing hallucinations and improving factuality. The session also demonstrates how to handle insufficient evidence by producing cautious responses rather than unsupported claims. Participants will inspect whether cited sources actually support the generated answer and prepare the chain for a simple user-facing interface. By the end, participants will have a runnable RAG workflow that links retrieval, prompt construction, generation and transparent citations. This workflow provides practical foundations for later evaluation and deployment discussions in the webinar.
14:40 RAG evaluation: retrieval quality, faithfulness, groundedness and RAGAS/DeepEval concepts - Sotiris Gyftopoulos (ATHENA RC) George Drosatos (ATHENA RC)
RAG evaluation: retrieval quality, faithfulness, groundedness and RAGAS/DeepEval concepts
- Sotiris Gyftopoulos (ATHENA RC)
- George Drosatos (ATHENA RC)
14:40 - 15:00
This evaluation session explains how to assess RAG systems beyond a single final answer score. Participants will learn why evaluation must decompose the pipeline into retrieval, generation, citation quality and end-to-end behaviour. The session introduces retrieval metrics such as Precision@k, Recall@k, Mean Reciprocal Rank and NDCG, showing how they describe the evidence made available to the model. It then discusses answer-level criteria, including correctness, faithfulness, groundedness, citation quality, completeness and relevance. Participants will also be introduced to RAGAS and DeepEval concepts for automated RAG evaluation, regression testing and structured comparison of system variants. The emphasis is diagnostic: metrics should identify failure modes and guide concrete improvements, such as better chunking, query handling, filtering, reranking or prompt constraints. By the end, participants will understand how to evaluate and iterate RAG systems systematically. This prepares them to maintain RAG quality as data, prompts and models evolve in operational settings, after deployment too.
15:00 Deployment considerations and wrap-up - George Drosatos (ATHENA RC) Sotiris Gyftopoulos (ATHENA RC)
Deployment considerations and wrap-up
- George Drosatos (ATHENA RC)
- Sotiris Gyftopoulos (ATHENA RC)
15:00 - 15:15
This wrap-up session discusses how a Colab-based RAG prototype can be prepared for real-world deployment. Participants will review the main components of a deployed RAG application, including user interface, API or orchestration chain, retriever service, vector or hybrid store, LLM service, monitoring and logging. The session highlights production concerns such as latency, cost, data privacy, access control, source updates, model versioning, failure logging and regression testing. It also revisits the specific challenges of Greek-language public-service RAG, including morphology, administrative terminology, tables, articles, codes, appendices and vague user questions. Participants will receive an end-to-end design checklist covering information need, risk level, document preparation, chunking, metadata, retrieval strategy, grounding, citations, evaluation and deployment safeguards. The session closes the webinar by connecting the theoretical concepts and hands-on notebook into a coherent pathway for operational RAG applications. It also identifies next steps for adapting the tutorial to participants' own organisational domains and use cases.