jobs in LyncLab

Kerja Sepenuh Masa, Data Engineer (Geospatial Architecture - Infrastructure) di LyncLab Federal Territory - Maukerja

Data Engineer (Geospatial Architecture - Infrastructure)

LyncLab

KL City, Federal Territory

Kongsi
Simpan

Lokasi Kerja

  • Jalan Sultan Mizan Zainal Abidin, Kompleks Kerajaan Kuala Lumpur Federal Territory Malaysia

Penerangan Kerja

Tanggungjawab

About the Role

We are looking for a Data Engineer to design, build and run the data platform behind our maritime products: live vessel tracking, marine traffic and risk studies, and the data science and AI systems built on them. You will own the whole path, from the edge server at a receiver site, through stream processing, to the analytical and operational databases, together with the servers and networks it runs on. The role is hands-on, and part of it is on site: installing servers, connecting them to the network and bringing them up from a bare operating system.


Key Responsibilities

Streaming Ingestion & Pipelines

  • Build and run publishers that move raw AIS/NMEA streams from edge servers and external feeds into the streaming platform, with local buffering, reconnect with back-off, and no silent message loss.
  • Administer the streaming platform: topic, partition and key design, retention, consumer groups and offsets, authentication and access control.
  • Develop and operate stream-processing jobs for decoding, enrichment, latest-position state and event detection (anchoring, berthing, zone entry and exit, proximity), handling event time, late data, checkpoints and backpressure.
  • Build incremental, idempotent batch pipelines, and run backfills and reprocessing from the raw archive.
  • Measure freshness, lag and loss at every stage, and trace a missing record to the stage that dropped it.


Data Architecture & Geospatial Modeling

  • Design raw → staging → curated models for trajectories, reference data (ports, anchorages, zones, boundaries) and spatial outputs (geofencing, proximity, grid aggregates), covering geometry storage, spatial indexing, partitioning, clustering and naming conventions.
  • Plan and execute large-scale data migrations with change data capture, reconciliation and minimal-downtime cutover.


Data Warehouse & Databases

  • Write advanced SQL to pull data for analysts, data scientists and client studies, and turn repeated pulls into curated tables or parameterized templates.
  • Manage warehouse datasets (partitioning, clustering, expiration, schema evolution, access control, data catalog), and keep loads complete, deduplicated and reconciled with source counts.
  • Control cost and performance: query cost estimates and limits, materialized and incremental tables, and spend monitoring in the warehouse; indexing, query plans, maintenance, replicas and retention in operational databases.


Infrastructure & Networking

  • Set up servers end to end at client premises, data centers and receiver stations: rack and cable them, connect them to switches, configure IP addressing, VLANs, routing, DNS, NTP and firewalls, and install and harden the operating system.
  • Deploy and support on-premise installations, working within each client’s security and change-control rules.
  • Run cloud infrastructure (virtual machines, managed databases, object storage, DNS, CDN and certificates) and containerized services that come back by themselves after a reboot; plan capacity and test backups and restores.


Reliability, Data Quality & Governance

  • Lead CI/CD, infrastructure automation, environment separation and version-controlled schema migrations.
  • Run monitoring, alerting, SLAs and runbooks; lead incident response and write post-incident reviews that prevent repeats.
  • Own the data quality framework (automated checks, anomaly detection, reconciliation, quality SLAs), and benchmark data sources against each other on coverage, latency and completeness.
  • Enforce governance: ownership, lineage, access control, retention and secrets management across all data stores.


AI Enablement & Collaboration

  • Build curated datasets, knowledge graphs and data interfaces that let agentic AI tools query maritime and geospatial data reliably, with guardrails on quality, lineage and access.
  • Partner with data scientists on study data and production models, mentor others, and document architecture, tables and runbooks.


QualificationsMust-Have
  • Degree in Computer Science, Software or Data Engineering, or equivalent experience
  • 5+ years building and operating production data pipelines, including streaming.
  • Fluent in Python, plus at least one of Java, Scala, Rust or Go.
  • Advanced SQL with hands-on BigQuery and PostgreSQL: schema design, partitioning, indexing, query tuning and cost control.
  • Kafka in production, and a stream-processing framework (Flink preferred, in Java or PyFlink; Kafka Streams, Spark Structured Streaming or Beam also considered).
  • Workflow orchestration with Airflow, cron or similar schedulers.
  • Linux administration and networking (TCP/IP, subnets and VLANs, routing, DNS, firewalls, VPN, TLS); able to take a server from bare metal to a running service.
  • Sound engineering practice: idempotency, replayability, observability, Git, CI/CD, Docker, and data quality checks in production pipelines.
  • Experience with geospatial data and analytics-ready data models.
  • Willing to travel to client sites and receiver stations; government projects may require security vetting.


Preferred

  • Streaming and storage: Flink state, checkpoint and backpressure tuning; PostGIS, TimescaleDB, H3 and spatial indexing (GiST, BRIN).
  • Domain: AIS/NMEA pipelines, or similar telemetry streams from edge or IoT devices.
  • On premise: managed switches, RAID, out-of-band management (IPMI, iDRAC, iLO), and government or regulated networks.
  • Platform: Terraform or Ansible, Kubernetes, Prometheus/Grafana, GCP (Cloud Run, Dataflow, Composer), CDC (Debezium, Kafka Connect), Keycloak/OIDC.
  • AI: knowledge graphs and ontology design; AI-assisted development (e.g. Claude, Codex) in a production workflow.


What We Look For

  • Owners, who treat an outage or a data gap as theirs to find, fix and prevent.
  • Comfortable moving between a server rack, a Kafka topic and a SQL query.
  • Architecture-first: clear conventions, and a sound balance of cost, performance and maintainability.
  • Growth-minded and proactive in raising engineering standards in a fast-moving team.

Peringatan Penting

Jangan pernah kongsikan maklumat bank atau kad kredit anda semasa memohon pekerjaan. Elakkan membuat sebarang pembayaran atau mengisi survey yang tidak berkaitan. Jika ada yang mencurigakan, sila laporkan iklan pekerjaan ini segera.

Lebih Lanjut