Putting it together: one order's journey

Follow one order through everything in this chapter.

1. Generation. Asha taps Pay in a shopping app. The app's PostgreSQL database (OLTP, normalized) inserts a row in orders and two rows in order_items. The app also emits a payment_success event.
2. Ingestion. CDC picks up the inserts within seconds. Every night an incremental batch job also pulls the payment gateway's settlement file through its API.
3. Storage. Everything lands untouched in the raw layer of the lake, as partitioned Parquet.
4. Transformation. An idempotent job cleans types, removes duplicate events, quarantines bad rows and builds fct_order_lines (grain: one row per product per order) plus dim_customer, dim_product and dim_date: a star schema.
5. Serving. The sales dashboard (OLAP) shows yesterday's revenue by city. The data science team pulls a feature table for a repeat-purchase model.
6. Undercurrents. An orchestrator runs the steps in order with retries. Tests check row counts and that amount_paise is never negative. Asha's phone number is masked for everyone except the support team. Lineage shows which dashboards depend on fct_order_lines.

The app team is upstream of you. The dashboard and the model are downstream. The contract on orders tells you when its schema will change.

What's next in this course

The rest of the course teaches the tools for each step: