Job Description
We are looking for an experienced
Architect Lead – Data Engineering
to lead the design and implementation of enterprise-scale data platforms, data pipelines and cloud-based data solutions.
Key Responsibilities:
Lead the architecture and design of scalable data engineering and data platform solutions.
Design and develop high-volume data pipelines using
Apache Spark and PySpark
.
Implement data warehousing and analytics solutions using
Snowflake
.
Design workflow orchestration using
Apache Airflow
.
Develop modern data lake/lakehouse solutions using
Apache Iceberg
.
Design real-time and event-driven data pipelines using
Apache Kafka
.
Develop ETL/ELT solutions using
DBT and enterprise ETL frameworks
.
Work with GBI ETL Framework and ensure integration across enterprise data platforms.
Leverage cloud computing technologies to build secure, scalable and highly available data solutions.
Provide technical leadership, architecture guidance and mentoring to data engineering teams.
Drive performance tuning, data quality, scalability and reliability of data platforms.
Collaborate with business and technology stakeholders to translate requirements into technical solutions.
Explore and implement appropriate
Artificial Intelligence
capabilities within data platforms.
Requirements:
Minimum 10
years of experience
in data engineering, big data or data architecture.
Strong hands-on experience with
Spark, PySpark, Snowflake, Airflow, Kafka and Iceberg
.
Strong experience in ETL/ELT frameworks and
DBT
.
Experience with cloud-based data platforms and modern data architecture.
Experience designing large-scale data pipelines and lakehouse architectures.
Exposure to Artificial Intelligence / AI-based data solutions.
Strong analytical, problem-solving, communication and technical leadership skills.
Degree in Computer Science, Information Technology, Engineering or a related discipline.
OX Consultancy is an IT recruitment firm and we offer a range of services that cover permanent, contract and flexible on demand recruitment. Our team have enormous experience enabling the right solution for your needs. Whether your business need is to transform talent acquisition, hire permanent staff, or plug in the gaps around your workforce, we can adapt our services to meet your business needs.