Data Pipeline Engineering: Architect, design, and maintain reliable, high-throughput pipelines that ingest, route, and process vast volumes of unstructured data. You will integrate AI models for automated data classification and information extraction, while ensuring latency and throughput meet mission-critical standards.
Responsible for the research and development of data platform for Xiaomi Internet businesses, driving end-to-end design and delivery of large-scale data systems.
Collaborate with internal and external stakeholders to identify and acquire relevant data sources for AI/ML projects, leveraging on both structured and unstructured data to enrich the organization’s data assets.
Design, implement, and maintain scalable data pipelines using the Databricks Lakehouse Platform using technologies such as Apache Spark and Delta Lake.
Bachelor's degree or above in Computer Science, Software Engineering, Data Science or a related field. 3+ years of professional big data development experience in the internet industry, with proven experience in end-to-end real-time data warehouse implementation and data ingestion system construction.
Design and implement scalable ETL/ELT pipelines on modern cloud data platforms - AWS (Glue, Step Functions, Lambda, S3), Databricks, Spark or Microsoft Fabric.
Experienced professional with strong expertise in back-end development and data analytics. Proficient in Python, and SQL, with hands-on experience in cloud-native development on Azure and Infrastructure as Code (IaC) using Terraform Enterprise.
We are looking for AI Engineers for a strategic initiative to build Agentic AI use cases around Trading Surveillance. Ideal candidate is someone with strong pyspark background coupled with proven experience build Agentic AI pipelines for code conversion from Q KDB to Pyspark.