- Singapore
Lokasi Kerja
Penerangan Kerja
Tanggungjawab
Data Pipeline Engineer
Role Overview
We are looking for a Data Pipeline Engineer with expertise in multi-source data integration and cloud-
native data platforms. The role focuses on building scalable ingestion frameworks, enabling cross-
agency data sharing, and implementing governance-driven data engineering solutions on AWS.
Key Responsibilities
Develop and maintain 15 data ingestion pipelines from diverse source systems, including REST
APIs, SFTP file transfers, and database extracts.
Design and implement ETL transformations based on business and technical data mapping
specifications.
Build secure cross-agency data-sharing mechanisms using AWS Lake Formation cross-account
permissions and resource links.
Implement data lineage and metadata tracking to support governance, auditability, and
compliance requirements.
Develop robust data quality frameworks capable of handling schema drift, delayed data arrivals,
and source system outages.
Build and manage Infrastructure as Code (IaC) assets using Terraform or AWS CDK.
Execute unit testing, integration testing, and cross-platform data validation activities.
Support User Acceptance Testing (UAT) with business stakeholders to validate data quality,
timeliness, and access controls.
Document source connectivity patterns, data pipelines, governance controls, and architecture
designs.
Participate in Agile ceremonies, code reviews, and technical design discussions.
Required Skills & Experience
5+ years of experience in data engineering and cloud data platforms.
Strong hands-on expertise with AWS Glue, AWS Lake Formation, Amazon S3, and Amazon
Athena.
Proficiency in Python, PySpark, and SQL.
Experience integrating data from APIs, SFTP systems, and relational databases.
Strong understanding of metadata management, data governance, and secure data sharing
models.
Experience with Terraform, AWS CDK, or similar IaC frameworks.
Knowledge of distributed data processing and scalable ETL architectures.
Good to Have
Experience with OpenLineage or data lineage/governance platforms.
Exposure to multi-tenant data platform architectures.
Experience delivering solutions for government agencies or public sector organisations.
1. Education
Bachelor’s degree in computer science, information technology, engineering, or a related discipline.
Pay: $5,000.00 - $12,000.00 per month
Work Location: In person
Peringatan Penting
Jangan pernah kongsikan maklumat bank atau kad kredit anda semasa memohon pekerjaan. Elakkan membuat sebarang pembayaran atau mengisi survey yang tidak berkaitan. Jika ada yang mencurigakan, sila laporkan iklan pekerjaan ini segera.