Databricks Data Engineer

One route through SQL, Python and PySpark, in the order a Databricks data engineering loop actually asks for them.

20 questions across 6 modules, in the order the concepts build. Each one is graded by running it.

1
SQL foundations
Group, rank and de-duplicate — the three things every SQL round opens with.
SQLFree0/3
2
Windows and time
Carry a value across rows: running totals, as-of joins and late data.
SQLLocked
- Running balance per accountMedium
- Rolling 7-day active usersMedium
- Month-over-month revenue growthMedium
- Latest price as of each orderMedium
- Late arriving events per visitorMedium
3
Harder SQL shapes
Sessions, cohorts and islands — the questions that separate the round.
SQLLocked
- Sessionize clickstream eventsHard
- Gaps and islands in device uptimeHard
- Cohort retention by signup monthHard
4
Python for pipelines
Parse messy input, validate it, and flatten it into something loadable.
PythonLocked
- Parse fixed-width log linesEasy
- Validate JSON records against a schemaEasy
- Flatten nested JSON into a fact tableMedium
- Merge overlapping time rangesMedium
5
Python at scale
Bounded memory, one pass, and ordering work that has dependencies.
PythonLocked
- Chunked aggregation over a large CSVMedium
- Sessionize a clickstream in PythonMedium
- Top K events in a streamHard
- Pipeline task execution orderHard
6
PySpark
The same thinking, on a cluster, where a shuffle has a price.
PySparkLocked
- Deduplicate customers, keep latest recordMedium