- Islandwide (Singapore) Singapore

Working Location
Job Description
Responsibilities
Responsibilities:
Pipeline Development& Orchestration
- Design, build, and maintain data pipelines using Python, SQL, and orchestration tools
- Develop and manage Directed Acyclic Graph (DAGs) / flows in using orchestration tools like Apache Airflow and Prefect
- Ensure pipelines are idempotent, scalable, and fault-tolerant
- Implement logging, monitoring, and alerting for pipeline observability
Package &Dependency Management
- Install, upgrade, and manage Python packages in controlled environments
- Maintain ************* / dependency manifests with version pinning
- Resolve dependency conflicts and ensure compatibility across environments (dev, UAT, prod)
- Support deployments in restricted or air-gapped environments where require
Security Remediation& Library Fixes
- Analyse vulnerability reports from security scanning tools (e.g., CVE findings)
- Upgrade or replace vulnerable libraries while maintaining pipeline stability
- Fix broken imports, deprecated APIs, and compatibility issues arising from library updates
- Collaborate with security teams to ensure compliance with organisational standards
Code Refactoring &Optimization
- Refactor legacy code across:
* Data ingestion APIs
* Data transformation(Pandas/SQL)
* Model training and inference pipelines
* Orchestration workflows
- Improve code modularity, readability, and performance
- Ensure backward compatibility and minimal disruption to production systems
Data Processing &Integration
- Perform data transformation and validation using Pandas and SQL
- Integrate streaming data pipelines using Kafka (producers/consumers)
- Ensure schema consistency and data quality across pipeline stages
Testing, Deployment& Support
- Implement unit and integration tests for pipelines
- Support workflows for deployment of data pipelines
- Troubleshoot pipeline failures and perform root cause analysis
- Provide production support and continuous improvement of data workflows
Streaming and Integration Skills
- Working knowledge of Kafka (topics, partitions, consumers, producers)
- Experience handling schema evolution and message serialization/deserialization
Platform Awareness Skills
- Working knowledge of Kafka (topics, partitions, consumers, producers)
- Experience handling schema evolution and message serialization/deserialization
Security &Reliability
- Experience resolving vulnerabilities from security scans
- Understanding of secure coding practices
- Experience working in regulated or high-security environments
Core Mandatory Skills:
- Strong Python programming (modular design, error handling, logging)
- Advanced SQL (joins, window functions, optimization)
- Hands-on experience with Pandas and Kafka for data processing
- Experience with orchestration tools:- Apache Airflow
- Prefect (or equivalent)
- Experience with package and dependency management (pip, virtual environments)
Experience Requirements:
- Preferably at least2-3 or more years of experience in data engineering
- Prior experience working with production data pipelines
- Prior experience handling dependency conflicts, library upgrades, and refactoring in live systems
- Ability to work across multiple layers (API / data / orchestration / ML)
Interested candidates may send their CV to MAC (Reg No. R1221300) ************* quoting the job title in the Subject line. We regret that only shortlisted candidates will be notified.
Important Information
Never provide your bank or credit card details when applying for jobs. Do not transfer any money or complete unrelated online surveys. If you see something suspicious, Report this Job ad.