Schulungsübersicht
Databricks Platform and Lakehouse Fundamentals
- Databricks Lakehouse architecture and components
- Organizing workspaces and catalogs
Databricks Workspace and Notebooks
- Workspace navigation and notebook-based development
- Structuring code into reusable notebooks
Apache Spark Architecture and Execution
- Spark runtime architecture and execution model
- Lazy evaluation and the job DAG
PySpark DataFrames and the DataFrame API
- DataFrame abstractions and schemas
- Core DataFrame operations and column expressions
Translating SQL to PySpark DataFrames
- Translating core SQL clauses to DataFrame operations
- Window functions and aggregations in PySpark
Reading and Writing Data in Databricks
- Reading from common file and database sources
- Writing and partitioning data in the Lakehouse
Delta Lake and Table Management
- Delta tables and ACID transactions
- Time travel and schema evolution
Data Cleaning and Transformation Patterns
- Data cleaning and type conversion
- Building reusable transformation logic
User-Defined Functions and Modular Code
- Python UDFs and pandas UDFs
- Modularizing procedural logic into functions
Performance Tuning and Optimization
- Partitioning and caching strategies
- Diagnosing bottlenecks with the Spark UI
Structured Streaming Fundamentals
- Batch versus streaming processing models
- Streaming DataFrames and basic aggregations
Databricks Jobs and Workflow Orchestration
- Scheduling notebooks as jobs and tasks
- Building multi-step workflows with dependencies
Unity Catalog and Data Governance
- Unity Catalog architecture and namespaces
- Access control and data lineage
Testing, Debugging, and Production Practices
- Unit testing PySpark logic
- Debugging and code quality standards
End-to-End Financial Services Use Cases
- Building an end-to-end banking ETL pipeline
- Translating legacy SQL processes to PySpark
Migrating SQL Workloads to PySpark
- Migration strategy and planning patterns
- Incremental conversion of SQL workflows to PySpark
Voraussetzungen
- Erfahrung mit Python-Programmierung, einschließlich Funktionen und Datentypen
- Verständnis von SQL, einschließlich Joins, Aggregationen und Unterabfragen
- Keine Vorerfahrung mit Databricks oder PySpark erforderlich
Zielgruppe
- Data Engineers, Datenanalysten und Datenexperten
- Teams, die bestehende SQL-basierte Workflows auf Databricks und PySpark migrieren
Erfahrungsberichte (1)
Ich mochte es, dass es praktisch war. Ich liebte es, die theoretischen Kenntnisse mit praktischen Beispielen anzuwenden.
Aurelia-Adriana - Allianz Services Romania
Kurs - Python and Spark for Big Data (PySpark)
Maschinelle Übersetzung
