Design end-to-end LLM evaluation plans for business scenarios such as dialogue and financial trading. Build evaluation metric systems and rubrics, transforming subjective model performance judgments into quantifiable, reproducible, and explainable evaluation conclusions.
Lead the design and construction of evaluation datasets. Define evaluation dimensions and scenario coverage, establish high-quality data annotation guidelines and quality control processes, and build benchmarks that authentically reflect business needs and have discriminative power.
Analyze model capability boundaries and failure modes based on evaluation results. Produce actionable improvement recommendations and collaborate with algorithm and product teams to drive model iteration, making evaluation a critical component of the R&D loop.
...
Conduct research, technical evaluation, ingestion, integration, cleansing, standardization, computation, storage, and servicing of financial market data, covering securities master data, real-time and historical market data, fundamentals, corporate actions, indices, and business-required product and risk data; responsible for source ingestion, raw retention, and stable delivery to knowledge engineering pipelines for content-type data such as announcements, news, and research reports.
Design scalable unified data models and integration frameworks, handling different markets' trading calendars, time zones, currencies, security identifiers, listing relationships, lifecycles, and data corrections, supporting rapid onboarding of new markets and sources.
Build batch-stream unified data pipelines centered on Flink, continuously optimizing latency, throughput, query performance, stability, and cost, while supporting consumer trading products, research analysis, and AI scenarios.
...
Factor Mining & Validation: Discover, construct, and validate trading factors from multi-source data including market data, fundamental data, and on-chain data. Continuously iterate the factor library to identify effective alpha signals.
Factor Prediction Modeling: Design and optimize prediction models using machine learning and deep learning methods to improve signal accuracy and stability while controlling overfitting and strategy decay.
Strategy Design & Backtesting: Lead the design, backtesting, and live deployment validation of trading strategies — covering signal generation, portfolio construction, risk control, and execution optimization. Take ownership of strategy P&L and risk performance.
...
Participate in the Margin team’s full software development lifecycle, from requirements analysis and test planning to execution, defect tracking, product delivery, and maintenance, with a strong focus on Java / Rust-based trading and risk systems.
Work closely with software engineers, product managers, designers, and operations teams to provide insights and feedback on system design, testing strategies, and implementation for margin trading products and risk control features.
Set up and manage testing environments, developing detailed and well-structured test plans and cases for complex trading workflows, margin calculations, liquidation processes, and high-concurrency systems.
...
Participate in the testing of Binance backend microservices, ensuring high-quality delivery across service APIs and integration boundaries.
Understand business scenarios and technical design (domain models, database schemas, service-to-service contracts); design, develop, and execute test cases, and organize test case reviews.
Execute functional, integration, and regression testing at the API and service layer; identify and track bugs, and ensure timely fixes and delivery.
...