Senior Software Engineer, Reliability and Efficiency
- ๐บ๐ธ United States
- On-site
- Senior
- 2 months ago
- AI
- System Design
- Incident Management
- Devops
- Linux
- Python
- C++
- Java
- Bash
- CI/CD
- Kubernetes
- AWS
- GCP
- Azure
- IaC
- Terraform
- Prometheus
- Grafana
- Machine Learning
About the Team
The USDS JV AI Infra SRE team works with engineering and product teams to build and run globally distributed, fault-tolerant systems for TikTok's state-of-the-art Recommendation and Search engines. Engineers on this team will deliver on production ownership, driving observability and automation across complex service mesh architectures.
Operating within a highly secure and strictly isolated USDS environment, our team is currently in a critical 0-to-1 infrastructure building phase. We are looking for resilient engineers who are comfortable navigating evolving toolsets and established workflows, eager to bridge the gap between existing systems and our rapid technological growth today to build the robust foundation and advanced agentic automation tools for tomorrow.
Responsibilities
- Partner with cross-functional teams throughout the entire service lifecycle, from system design consulting and architecture reviews to deployment, operation, and continuous refinement, acting as a strong communicator to align engineering efforts during critical incidents and daily operations.
- Build tools, platforms, and automation to improve service reliability, scalability, R&D efficiency, and operational workflows, while maintaining a healthy balance between deep software development and daily operational duties .
- Ensure high availability and performance of large-scale, multi-region systems by proactively monitoring service health, latency, and key metrics.
- Lead complex infrastructure migrations and system architecture upgrades in a highly restricted compliance environment.
- Drive robust Capacity Management, including resource allocation, stability optimization, and error attribution, while ensuring strict SLA/SLO compliance.
- Drive sustainable incident management, blameless postmortems, and systemic root-cause resolution.
Minimum Qualifications
- Bachelor's degree or above in Computer Science, Software Engineering, or a related technical field, or equivalent practical experience.
- 3+ years of hands-on SRE, DevOps, or Systems Engineering experience managing large-scale, highly reliable systems.
- Strong foundation in Linux operating systems, internal system performance, and networking fundamentals.
- Experience programming in at least one standard language: Go, Python, C/C++, or Java.
- Demonstrated experience in designing, troubleshooting, and maintaining complex distributed systems, with a proven ability to navigate highly dynamic production environments, troubleshoot high-complexity traffic issues, and modernize operational workflows via Bash/shell scripting.
- Familiarity with modern CI/CD practices and automated deployment pipelines.
- High resilience and a strong 'system sense' to navigate ambiguity and break down complex, real-world problems. You should thrive in developing or restricted infrastructure environments, turning these constraints into a unique opportunity to pioneer standard developer tools from the ground up.
Preferred Qualifications
- Hands-on experience with container orchestration (e.g., Kubernetes) and service mesh architectures.
- Familiarity with cloud platforms (AWS/GCP/Azure) and Infrastructure-as-Code (e.g., Terraform).
- Experience with modern observability tooling (e.g., Prometheus, Grafana, Distributed Tracing).
- Track record of identifying opportunities to streamline manual operations and successfully designing/coding self-serve tools or automation to eliminate them.
- Interest or experience in leveraging LLMs, Agentic AI, or advanced machine learning concepts to enhance operational efficiency, automate troubleshooting, or build next-generation infra tooling.
- Strong technical curiosity and adaptability, with a proven track record of navigating evolving tech stacks and embracing new technical challenges.
- Experience with designing and building distributed systems and/or infrastructure is a strong plus.
Senior Software Engineer, Reliability and Efficiency ยท TikTok USDS