Kerja Sepenuh Masa, Data Engineer di CloudMile - Maukerja

About the Role

We are looking for a hands-on Data Engineer to design, build, and operate scalable cloud data platforms for enterprise customers.

You will work across modern data warehouse and lakehouse environments, with a primary focus on platforms such as Google BigQuery, Snowflake, Amazon Redshift, AWS Glue, and dbt. You will contribute throughout the delivery lifecycle, from requirements gathering and solution design to implementation, testing, deployment, and production support.

This role is suitable for an engineer with approximately five years of relevant experience who is technically strong, delivery-oriented, and able to lead smaller project work-streams while mentoring junior engineers.

Key Responsibilities

Data Engineering & Pipeline Development

• Design, develop, and maintain scalable, reliable, and secure data ingestion, transformation, and orchestration pipelines.
• Build batch and, where required, near-real-time data pipelines integrating data from databases, APIs, SaaS applications, files, and streaming platforms.
• Develop reusable data ingestion frameworks, utilities, and deployment patterns to improve delivery speed and consistency.
• Optimise pipelines for performance, scalability, maintainability, reliability, and cost efficiency.

Data Warehousing & Transformation

• Design and implement data warehouse, data lake, and lakehouse solutions using platforms such as BigQuery, Snowflake, and Amazon Redshift.
• Develop dimensional models, data marts, semantic layers, and curated datasets to support analytics, reporting, and AI use cases.
• Prepare and maintain high-quality historical and near-real-time datasets for forecasting models, such as demand forecasting, sales forecasting, inventory planning, capacity planning, and anomaly detections.
• Build and maintain transformation workflows using dbt, including models, tests, documentation, macros, snapshots, and deployment pipelines.
• Apply best practices in data modelling, partitioning, clustering, workload optimisation, query tuning, and cost management.

Cloud Data Platform Engineering

• Build and manage cloud-native data solutions on Google Cloud Platform and/or AWS.
• Develop AWS Glue jobs, crawlers, catalogues, workflows, and supporting components where applicable.
• Work with cloud storage, IAM, networking, secrets management, logging, monitoring, and CI/CD pipelines.
• Support environment setup and release processes across development, UAT, and production environments

Data Quality, Governance & Reliability

• Implement data quality checks, reconciliation processes, monitoring, alerting, and incident response procedures.
• Support metadata management, lineage, access controls, and data governance requirements.
• Ensure solutions follow organisational security, compliance, and operational standards.
• Produce clear technical documentation, including architecture diagrams, data models, runbooks, and handover materials.

Project Delivery & Technical Leadership

• Lead assigned technical work-streams within data engineering projects and ensure deliverables are completed on time and to the required quality.
• Translate business and technical requirements into practical data engineering designs and implementation plans.
• Collaborate with solution architects, analytics engineers, BI developers, AI engineers, project managers, and customer stakeholders.
• Mentor junior engineers through code reviews, technical guidance, troubleshooting, and engineering best practices.
• Participate in effort estimation, project planning, technical risk identification, and stakeholder updates.

Requirements

Experience

• Around 5 years of hands-on experience in data engineering, data platform engineering, analytics engineering, or related roles.
• Experience delivering production-grade data pipelines and cloud data platform solutions.
• Experience working in customer-facing project delivery, consulting, or enterprise environments is preferred.
• Experience leading smaller technical work-streams or mentoring junior engineers is advantageous.

Core Technical

• Strong hands-on experience with one or more cloud data warehouses, particularly:
• Google BigQuery
• Snowflake
• Amazon Redshift
• Strong experience with dbt, including data modelling, transformations, testing, documentation, and deployment practices.
• Experience with AWS Glue, including ETL jobs, Glue Data Catalog, crawlers, workflows, and integration with S3, Redshift, or other AWS services.
• Strong SQL skills, including complex transformations, optimisation, stored procedures where applicable, and performance tuning.
• Proficiency in Python for data processing, automation, API integration, and pipeline development.
• Good understanding of data modelling concepts, including dimensional modelling, star schemas, slowly changing dimensions, and data marts.
• Experience with orchestration tools such as Apache Airflow, Cloud Composer, AWS Step Functions, or similar tools.
• Familiarity with Git, CI/CD pipelines, infrastructure-as-code, and containerisation is preferred.

Additional Skills

• Experience with data integration tools such as Fivetran, Airbyte, Informatica, Matillion, or similar platforms is advantageous.
• Familiarity with Apache Spark, Databricks, Kafka, or other distributed data processing technologies is a bonus.
• Experience with data visualisation and BI platforms such as Looker, Power BI, Tableau, or QuickSight is beneficial.
• Familiarity with data governance, data catalogues, lineage, access control, and data quality frameworks is advantageous.
• Exposure to AI, machine learning, RAG, or Generative AI data pipelines is a bonus.

Communication & Collaboration

• Strong problem-solving skills and ability to troubleshoot complex data and platform issues.
• Able to communicate technical concepts clearly to both technical and non-technical stakeholders.
• Comfortable working in cross-functional and customer-facing delivery teams.
• Proactive, detail-oriented, and accountable for delivery quality

Preferred Qualifications

• Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Science, or a related field.
• Relevant cloud certifications are advantageous, such as:
• Google Cloud Professional Data Engineer
• SnowPro Core Certification
• AWS Certified Data Engineer – Associate
• AWS Certified Solutions Architect
• dbt Fundamentals or dbt Analytics Engineering Certification

What We Offer

• Opportunities to work on enterprise-scale data, analytics, and AI initiatives across multiple cloud platforms.
• Exposure to modern technologies including BigQuery, Snowflake, Redshift, dbt, cloud-native orchestration, and Generative AI.
• A collaborative environment with experienced architects, engineers, and technology partners.
• Clear opportunities for career progression into technical lead, solution architect, or data platform leadership roles.
• Competitive salary, benefits, training, and certification support.