Kerja Sepenuh Masa, Senior Databricks Engineer [Python - SQL - DataBrick - Azure] di Luxoft Malaysia Pulau Pinang - Maukerja

Project Description:

• We are looking for a versatile Data Solutions Engineer capable of supporting the complete data lifecycle, from data ingestion and engineering to analytics, visualization, and API enablement.
• The ideal candidate combines strong Data Engineering fundamentals with the ability to work closely with business stakeholders, develop reporting solutions, and contribute across multiple data-related initiatives. Exposure to Data Science and AI use cases is considered an advantage.
• This role requires a flexible mindset, as the engineer may rotate between Data Engineering, Data Analytics, and Data Visualization initiatives depending on project priorities.

Responsibilities:

• Design, develop, and maintain scalable data pipelines.
• Build and optimize ETL/ELT processes.
• Develop data ingestion, transformation, and aggregation workflows.
• Ensure data quality, reliability, and performance.
• Develop solutions using cloud-based data platforms.
• Build and optimize data processing notebooks and workflows.
• Work with Data Lakes, Lakehouse architectures, and distributed data processing technologies.
• Support CI/CD and deployment activities for data solutions.
• Collaborate with business users to gather and clarify requirements.
• Translate business needs into analytical and technical solutions.
• Develop KPI calculations, reporting logic, and support data exploration for business insights.
• Build dashboards and reports to enable self-service analytics and decision-making.
• Develop and maintain APIs to expose data products and integrate with consuming applications.

Mandatory Skills Description:

Programming:

• Python
• SQL

Data Engineering:

• Databricks or Apache Spark / PySpark

Cloud Platform:

• Microsoft Azure

Examples:

• Azure Databricks, Azure Data Factory, Azure Data Lake Storage, Azure, Synapse Analytics, Microsoft Fabric

Data Processing:

• ETL / ELT
• Data Modeling
• Data Pipeline Development

APIs:

• FastAPI (preferred)
• Flask / REST API development experience acceptable
• Data Visualization

At least one of:

• Power BI
• Tableau
• Looker
• Similar BI platform

Nice-to-Have Skills Description:

• Data Science & AI
• Machine Learning fundamentals
• Predictive Analytics
• AI/ML model integration
• Statistical analysis
• Advanced Data Platform
• Azure Synapse
• Microsoft Fabric
• Azure ML
• Snowflake
• DevOps
• Azure DevOps
• Git
• CI/CD pipelines