Posts about Data Engineering

Simple validations to prevent garbage data in real pipelines: nulls, duplicates, types, and alerts with Python.

How to organize raw, staging, and marts layers in a real data pipeline. A practical follow-up to the ETL vs ELT article.

Technical guide to designing maintainable scrapers in Python: selectors, fallback, validation, logs and real best practices.