Senior Software Engineer, Data Foundation – USDS

About the Team

The TikTok U.S. Data Security (USDS) team is responsible for the security, integrity, and compliance of TikTok’s data within the United States. The Data Platform Team builds a scalable, reliable, cost-efficient infrastructure that ensures data integrity and empowers every business unit to win through self-service tools and scientific insights.

We are seeking a Senior Engineer to lead efficiency, stability, and AI-driven innovation initiatives. You will be responsible for optimizing the performance and efficiency of our core data engines—Apache Spark, Kafka, ClickHouse, and Doris—while pioneering AI-powered solutions that enhance platform intelligence, automate operations, and drive cost optimization at a petabyte scale.

Responsibilities

Performance & Efficiency

• Optimize Apache Spark performance and resource utilization at petabyte scale
• Enhance ClickHouse and Doris OLAP systems with storage-compute separation and query optimizations
• Implement AI-driven resource allocation and cost optimization strategies

Platform Intelligence & Automation

• Develop automated job diagnosis and failure analysis systems using AI/ML techniques
• Build intelligent monitoring and self-healing capabilities for data pipelines
• Create AI-powered tools for query optimization and data discovery

Stability & Operational Excellence

• Design robust disaster recovery and data integrity mechanisms
• Integrate AI/ML capabilities to enhance platform efficiency and developer productivity

Technical leadership

• Drive technical roadmap and mentor junior engineers
• Explore new technologies at the intersection of distributed systems and AI

Minimum Qualifications

• Education: BS/MS in Computer Science, Engineering, or a related technical field
• Distributed Systems: 3+ years of experience building and maintaining large-scale distributed systems
• Stack Expertise: In-depth knowledge of Apache Spark, ClickHouse or Doris

Preferred Qualifications

• Experience with Kafka & Flink at scale
• Kubernetes (K8s) experience
• AI/ML systems experience