Kerja Sepenuh Masa, Senior AI Infrastructure Engineer (LLM Inference - GPU Optimization) di GPU Wise - Maukerja

Company Description

GPUWise is an AI infrastructure technology company focused on Inference Efficiency Assurance for enterprises, AI-native companies, and infrastructure providers operating private, self-hosted, or rented GPU infrastructure.

GPUWise is developing an intelligent inference efficiency platform designed to help organizations understand and optimize GPU utilization, batching efficiency, queue behavior, latency, throughput, and inference economics.

Our approach combines independent, metadata-only observability with intelligent workload management, performance benchmarking, and controlled runtime optimization. The platform is designed to operate within customer-controlled infrastructure without requiring access to prompts, sensitive user data, or proprietary model weights.

From observe-only Shadow Audits to controlled production optimization, GPUWise aims to make AI inference infrastructure more efficient, measurable, reliable, and economically sustainable.

Role Description

We are seeking a Senior AI Infrastructure Engineer (LLM Inference & GPU Optimization) for a full-time, remote position.

This is a hands-on engineering role focused on designing, developing, integrating, and optimizing production-grade AI inference infrastructure.

You will work on GPU inference performance, intelligent request scheduling, batching optimization, workload management, observability, and scalable runtime systems.

The role requires a strong understanding of how LLM inference engines operate, how GPU resources are consumed, and how intelligent software can improve infrastructure efficiency without compromising latency, reliability, or security.

You will contribute to the development of GPUWise's core inference efficiency technology and help validate its performance across real-world AI infrastructure environments.

Key Responsibilities

AI Inference & GPU Optimization

• Design, develop, and optimize GPU-based LLM inference systems.
• Integrate with inference frameworks such as vLLM, NVIDIA Triton Inference Server, TGI, and other model-serving technologies.
• Analyze GPU utilization, memory usage, batching efficiency, request queues, and inference throughput.
• Identify performance bottlenecks and opportunities to improve inference efficiency.

Runtime Engineering & Workload Management

• Develop intelligent request scheduling, admission control, and micro-batching capabilities.
• Build latency-aware workload management and prioritization mechanisms.
• Implement runtime guardrails, safe fallback strategies, and rollback controls.
• Optimize performance while maintaining strict service-level objectives (SLOs).

Performance Benchmarking & Observability

• Develop benchmarking frameworks to measure throughput, time to first token (TTFT), P50/P95/P99 latency, and GPU resource efficiency.
• Benchmark improvements against appropriately tuned native inference engines.
• Build performance monitoring and observability capabilities using Prometheus, Grafana, NVIDIA DCGM, or equivalent tools.
• Translate technical performance improvements into measurable infrastructure efficiency and cost metrics.

Infrastructure & Platform Engineering

• Develop backend services, APIs, and integrations supporting GPUWise's inference control plane.
• Build scalable and secure deployment solutions for customer-controlled cloud, private, and hybrid environments.
• Work with Linux, Docker, Kubernetes, and distributed systems.
• Implement automated testing, performance validation, logging, and reliability controls.
• Support secure, metadata-only telemetry collection without accessing sensitive inference content.

Engineering Collaboration

• Work closely with the founding and engineering teams on technical architecture and product development.
• Support technical proof-of-concepts, customer pilots, and infrastructure integrations.
• Investigate complex production performance issues and recommend architectural improvements.
• Contribute to engineering standards, documentation, and long-term platform scalability.

Required Qualifications

• 5+ years of relevant software engineering, infrastructure engineering, distributed systems, or performance engineering experience, including hands-on work with AI inference infrastructure.
• Strong programming proficiency in Python, including asynchronous programming, backend services, and performance-sensitive applications.
• Hands-on experience with at least one LLM inference framework, such as vLLM, NVIDIA Triton, TGI, or TensorRT-LLM.
• Solid understanding of LLM inference architecture, including prefill, decoding, KV cache management, batching, and request scheduling.
• Experience with Linux systems, Docker, Kubernetes, and GPU-based compute environments.
• Understanding of distributed systems, concurrency, API design, and production reliability.
• Experience with GPU performance monitoring, profiling, troubleshooting, and optimization.
• Familiarity with observability tools such as Prometheus, Grafana, OpenTelemetry, NVIDIA DCGM, or NVML.
• Understanding of cloud infrastructure, networking, access controls, and secure deployment practices.
• Strong analytical, debugging, and problem-solving abilities.
• Ability to work independently in a remote engineering environment.
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Preferred Qualifications

• Experience with CUDA, C++, Go, or Rust.
• Familiarity with NVIDIA GPU architecture and profiling tools such as Nsight Systems.
• Experience optimizing production LLM inference services at scale.
• Knowledge of continuous batching, prefix caching, speculative decoding, or advanced inference-serving techniques.
• Experience with load testing, benchmarking, and GPU capacity planning.
• Understanding of inference economics, GPU-hour