Design, develop and maintain efficient and reliable ETL processes and data pipelines for the ingestion, transformation, and integration of data from multiple source agencies
Design, build, and manage scalable data ingestion, transformation, and data-serving solutions across modern hybrid cloud platforms, ensuring reliability, security, reusability, and compliance with enterprise architecture and governance standards.
Perform ETL (Extract, Transform, Load) processes on large and complex transaction datasets to support business reporting, modelling, and analytical needs.
Conceptualize and own the data architecture for multiple large-scale projects, while evaluating design and operational cost-benefit tradeoffs within systems
Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
Engineer the People Data Middle Layer: Utilise Microsoft Fabric Lakehouse to consolidate various HR data sources. Drive heavy data transformations and build complex business logic primarily within Microsoft Fabric's scheduled pipelines and notebooks, leveraging PySpark/Python for efficient data ingestion, transformation, and distribution.
Leads the architecture, design, and implementation of ETL processes, workflows, and data pipelines for the ingestion, transformation, and integration of data from multiple source agencies.