We are looking for a Data Lead – Data Engineering to drive technical diagnostics, optimization, and stabilization of enterprise data platforms supporting large-scale ETL and batch processing systems.
Key Responsibilities
- Strong expertise in: Azure Data Factory (ADF),Databricks (PySpark / Spark SQL preferred),SSIS and SQL Server
- Hands-on experience with: Batch orchestration tools (Control-M / Autosys),ETL/ELT pipeline design and optimization
- Strong experience in: SQL development and performance tuning, Troubleshooting and debugging complex data pipelines
- Familiarity with: Monitoring and alerting tools (Azure Monitor, Log Analytics)
- Exposure to large-scale data processing and batch system
- Review and optimize ADF pipelines, SSIS packages, and Databricks workflows for performance, scalability, and
- Analyze batch scheduling and orchestration frameworks (Control-M / Autosys), including dependencies, triggers, and SLA adherence
- Investigate and resolve pipeline failures, job delays, and runtime issues, ensuring faster recovery and minimal business
- Evaluate source-to-target integration flows, SQL logic, and transformation layers for efficiency and correctness
- Define and improve error handling, retry mechanisms, restart ability, and reprocessing strategies across ETL and batch
- Optimize runtime performance of data pipelines, including query tuning and batch execution improvements
- Establish and enhance monitoring and alerting frameworks using tools like Azure Monitor, Control-M dashboards, and custom logging solutions
- Identify automation opportunities to reduce manual intervention and improve operational efficiency
- Ensure data quality, consistency, and reliability across ETL processes and reporting outputs
- Support modernization initiatives, including migration from legacy ETL (SSIS) to cloud-based platforms like Databricks
- Collaborate with data engineers, architects, business, and support teams to drive improvements and ensure stable production systems
Avatar Techno Services Pte Ltd. is a Consulting and Solution Provider company in the Information Technology Industry, specialized in providing sound, secure and cost-effective IT solutions to Small & Medium enterprises thereby empowering and accelerating their growth.
Incorporated in 2019, Avatar Techno Services was initially under the banner of ILC Pte Ltd., providing IT Consulting services in Singapore to a handful companies in Singapore and overseas.
Within a short-span, we have grown from simple operation providing mono-service, to a team of dedicated staff, each of which is a professional in its own field.
We have a solid reputation in the technology industry for providing excellent services to our clients. Our values are represented by our integrity, thought leadership, and commitment to maintaining a high-level of excellence in the constantly evolving world of Information Technology. We strive to build a long-lasting relationship with our customers and client satisfaction is our key performance indicator. We take pride in becoming a preferred partner for most of our clientele.
Growing over the years with our maturity in service and excellence, we progress with a clear vision of brilliance in IT.
Avatar Techno Services provides customers consulting, integration, and implementation services and specialized in bringing business process capabilities, industry experience, and solution & market expertise that can help our clientele to deploy IT solutions, maximize functionality, and rapidly achieve Return on Investment