SQL for Data Engineering Training Workshop

Course overview

SQL for Data Engineering Training Workshop is a practical programme designed to develop the advanced SQL skills required to build, transform, validate and optimise data pipelines and analytical data platforms.

Unlike general SQL courses that focus primarily on querying databases, this workshop focuses on how SQL is used by data engineers in real-world data workflows. Participants learn how to use SQL for data extraction, cleansing, transformation, deduplication, incremental loading, data validation, analytical modelling and production ETL/ELT pipelines.

Course objectives

• Apply advanced SQL techniques to extract, transform, clean, validate and integrate data from multiple sources. Develop SQL-based ETL and ELT workflows using CTEs, window functions, joins, MERGE/UPSERT operations and reusable transformation logic.
• Implement reliable data-engineering patterns including incremental loading, deduplication, Slowly Changing Dimensions, reconciliation and data-quality validation.
• Optimise SQL workloads for large datasets using execution plans, indexing, partitioning, query optimisation and efficient data-transformation techniques.
• Optimise SQL workloads for large datasets using execution plans, indexing, partitioning, query optimisation and efficient data-transformation techniques.
• Develop production-ready SQL solutions that integrate with modern data platforms, analytics engineering tools, cloud warehouses, lakehouses and automated data pipelines.

Who should attend

Course content

• Module 1: SQL Foundations for Data Engineering Role of SQL in data engineering, SQL and relational databases, OLTP versus OLAP, SQL data-engineering lifecycle, source systems, staging systems, analytical systems, data warehouses, data lakes, lakehouses,
• Module 2: Advanced SQL Querying SELECT statements, filtering, sorting, aliases, expressions, CASE statements, NULL handling, data types, string functions, date and time functions, mathematical functions, conditional logic, DISTINCT, LIMIT/TOP, query comp
• Module 3: Joins & Data Integration with SQL INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL OUTER JOIN, CROSS JOIN, SELF JOIN, multi-table joins, anti-joins, semi-joins, joining datasets from different business domains, join conditions, duplicate records after j
• Module 4: Aggregation & Analytical SQL GROUP BY, HAVING, aggregate functions, conditional aggregation, distinct counts, grouped calculations, analytical metrics, summary tables, aggregation strategies, business metrics, data profiling using SQL and aggre
• Module 5: Subqueries, CTEs & Modular SQL Subqueries, correlated subqueries, Common Table Expressions, recursive CTEs, multiple CTEs, nested transformations, reusable SQL logic, modular SQL design, readable SQL, transformation pipelines using CTEs and deb
• Module 6: SQL Window Functions OVER clause, PARTITION BY, ORDER BY, ROW_NUMBER, RANK, DENSE_RANK, NTILE, LAG, LEAD, FIRST_VALUE, LAST_VALUE, running totals, moving averages, cumulative calculations, ranking datasets, deduplication using window functions
• Module 7: SQL Data Cleaning & Transformation Missing values, NULL handling, duplicate detection, deduplication, inconsistent data, data standardisation, data-type conversion, string cleansing, date cleansing, invalid records, conditional transformations,
• Module 8: Set Operations & Advanced Data Manipulation UNION, UNION ALL, INTERSECT, EXCEPT, set-based transformations, comparing datasets, identifying changed records, reconciliation, source-to-target comparison, exception identification and multi-source
• Module 9: SQL for ETL & ELT Pipelines SQL extraction, staging tables, transformation layers, loading strategies, ETL versus ELT, raw/staging/analytics architecture, SQL transformation workflows, reusable SQL scripts, transformation dependencies, pipeline
• Module 10: Incremental Loading & Change Data Processing Full loads versus incremental loads, watermarking, timestamps, high-water marks, incremental filters, MERGE, UPSERT, insert/update detection, delete detection, Change Data Capture concepts, incremen
• Module 11: Slowly Changing Dimensions & Data Warehousing SQL Dimensional modelling, fact tables, dimension tables, star schemas, surrogate keys, natural keys, Slowly Changing Dimensions Type 1, Type 2 and Type 3, historical tracking, effective dates, cur
• Module 12: SQL Data Quality & Validation Data-quality dimensions, completeness, accuracy, consistency, validity, uniqueness, referential integrity, NULL checks, duplicate checks, record-count validation, source-to-target reconciliation, schema validation
• Module 13: SQL Views, Temporary Tables & Stored Procedures Views, materialised views, temporary tables, table variables, stored procedures, reusable SQL components, parameterised SQL, procedural logic, database automation, scheduled SQL processing, reusa
• Module 14: SQL Performance Optimisation Query execution, execution plans, table scans, index scans, joins and query plans, indexing strategies, composite indexes, partitioning, query rewriting, filtering early, reducing unnecessary columns, avoiding inef
• Module 15: SQL for Modern Data Platforms SQL Server, PostgreSQL, MySQL, Oracle, Snowflake, BigQuery, Amazon Redshift, Databricks SQL, Microsoft Fabric, Spark SQL, cloud data warehouses, lakehouse SQL, cross-platform SQL considerations, portability and pl
• Module 16: SQL with dbt & Analytics Engineering Analytics engineering concepts, dbt fundamentals, dbt models, staging models, intermediate models, analytical marts, SQL transformations, incremental models, dbt tests, documentation, dependencies, lineage,
• Module 17: SQL for Big Data & Spark Spark SQL fundamentals, SQL over large datasets, DataFrames and SQL, distributed SQL processing, partitioning, large-scale joins, aggregations, Spark SQL optimisation, SQL with Delta Lake, lakehouse queries and integra
• Module 18: Production SQL, Automation & Data Pipeline Reliability Production SQL standards, parameterisation, reusable scripts, idempotency, error handling, transaction management, logging, data reconciliation, retry and recovery strategies, SQL testing,
• Module 19: SQL for Analytics, BI & AI-Ready Data Building analytical tables, data marts, semantic datasets, BI-ready datasets, Power BI integration, reporting tables, feature datasets, machine-learning data preparation, AI-ready datasets, data aggregatio
• Module 20: End-to-End SQL Data Engineering Capstone Source-system analysis, raw data ingestion, staging-layer development, SQL cleansing, transformation, deduplication, incremental loading, dimensional modelling, data-quality validation, analytical-layer

General notes

• SQL Server, PostgreSQL, MySQL, Oracle, Microsoft Fabric, Snowflake, BigQuery, Amazon Redshift, Databricks SQL, Apache Spark/Spark SQL, dbt, Python, Power BI, Git/GitHub, cloud data warehouses, data lakes, lakehouses and SQL development environments such as SSMS, Azure Data Studio and DBeaver.
• Advanced SQL transformation scripts, complex multi-table SQL queries, SQL data-cleaning workflows, CTE-based transformation pipelines, window-function solutions, SQL-based ETL pipelines, SQL-based ELT workflows, incremental loading scripts, MERGE/UPSERT solutions, Slowly Changing Dimension implementations, SQL data-quality validation framework, source-to-target reconciliation queries, analytical data marts, optimised SQL queries, SQL performance-analysis reports, dbt SQL models, Spark SQL transformations, production-ready SQL pipeline scripts, end-to-end SQL data engineering project

Frequently asked questions

Schedule & fees