We are seeking an experienced BAU Lead – Data Engineering to lead data operations, production support, and continuous improvement in a cloud-first environment. The role requires strong technical leadership and hands-on expertise in AWS, Snowflake, Python, SQL, Spark, and Airflow/MWAA, with experience managing BAU data platforms and production issues.
Key Responsibilities
- Lead a team of Data Engineers in BAU operations, production support, incident management, and platform stability.
- Design and develop scalable data pipelines, data feeds, and integrations across data lakes and warehouses.
- Work with business and technical stakeholders to understand data requirements and deliver fit-for-purpose solutions.
- Lead migration of data transformation workloads from Oracle, MS SQL, Hive, and Impala to Snowflake, Spark, Python, and AWS Glue.
- Design and optimise AWS data solutions using S3, Glue, DMS, RDS, Kinesis, Lambda, MWAA, IAM, and Step Functions.
- Develop and maintain datasets, data models, ingestion pipelines, and data management processes.
- Improve data quality, pipeline performance, scalability, and operational efficiency.
- Support critical production incidents, troubleshoot root causes, and drive permanent fixes and continuous improvement.
- Collaborate with Data Scientists, Architects, DevOps, and other technology teams on data and analytics initiatives.
- Establish and maintain data security, governance, documentation, and compliance standards.
- Provide technical leadership, mentoring, code reviews, and guidance on engineering best practices.
- Contribute to data strategy, architecture, technology evaluation, and continuous improvement initiatives.
Key Requirements
- 8+ years of Data Engineering experience, including 5+ years of hands-on experience in a Lead/production support role.
- Strong hands-on expertise in AWS and Snowflake.
- Strong programming and data engineering skills in SQL, Python, PySpark, Spark, and UNIX Shell.
- 3+ years of experience with AWS Glue, S3, RDS, Kinesis, Lambda, Airflow/MWAA, and Step Functions.
- Strong understanding of data lakes, data warehouses, ETL, data ingestion, data integration, and data flows.
- Experience working with large-scale datasets and technologies such as Snowflake, AWS Redshift, or Google BigQuery; Snowflake is highly preferred.
- Strong knowledge of Airflow/MWAA and cloud-based data pipeline optimisation.
- Experience with CI/CD, Git, SDLC, Agile/Lean methodologies, and production deployments.
- Understanding of distributed systems, data streaming, scalable data processing, and cloud architecture.
- Experience in data modelling, data quality, data mining, and segmentation.
- Bachelor’s degree in Computer Science, Engineering, Science, Technology, or a related STEM discipline.
- Strong stakeholder management skills with the ability to communicate complex technical solutions to business, technical, and senior management teams.
Soft Skills
- Strong team leadership, coaching, and mentoring capabilities.
- Excellent communication and stakeholder management skills.
- Comfortable working with senior leadership and geographically distributed teams.
- Proactive, technically curious, self-motivated, and solution-oriented.
- Strong customer focus with the ability to work effectively in a fast-paced Agile environment.
EA License Number: 23C2060 Registration ID is R22109715
Disclaimer: The company is committed to ensuring the privacy and security of your information. By submitting this form, you consent to the collection, processing, and retention of the information you provide. The data collected (which may include your contact details, educational background, work experience and skills) will be used solely for the purpose of evaluating your qualifications for the position you're applying for. Your data will be stored securely and retained for the duration necessary to fulfill our hiring process. If you are not selected for the position, your data will be kept on file for a limited period in case future opportunities arise. You have the right to access, correct, or delete your data at any time by contacting us at Quess Singapore | A Leading Staffing Services Provider in Singapore (quesscorp.sg)
This is in partnership with the Employment and Employability Institute Pte Ltd (“e2i”).
e2i is the empowering network for workers and employers seeking employment and employability solutions. e2i serves as a bridge between workers and employers, connecting with workers to offer job security through job-matching, career guidance and skills upgrading services, and partnering employers to address their manpower needs through recruitment, training, and job redesign solutions. e2i is a tripartite initiative of the National Trades Union Congress set up to support nation-wide manpower and skills upgrading initiatives. By applying for this role, you consent to Quesscorp Singapore’s PDPA and e2i’s PDPA