Data Engineer, Video Platform

About the team

TikTok video system is a world-leading video platform that provides multi-media storage, delivery, transcoding services. As a part of US Tech Service department, we are responsible for building the next generation video processing platform which provides excellent experiences for billions of users around the world.

About the role

As a Data Engineer, you will be instrumental in shaping strategic decisions within the Video Architecture Platform. This role emphasizes building, optimizing, and maintaining robust data processing architectures that support new data and ETL pipelines. Your expertise will ensure that data flows reliably and accurately across various systems, empowering stakeholders with high-quality, actionable insights that impact quality of service, quality of experience and cost optimizations.

Responsibilities

• Craft optimal end to end data processing architecture and systems for new data and ETL pipelines.
• Design, build, and maintain efficient and reliable data pipelines to move and transform big data.
• Implement and monitor quality control measures to ensure data accuracy, completeness, and consistency.
• Drive internal process improvements and automate manual processes for data quality and SLA management.
• Create and maintain technical documentation, such as data dictionaries, data flow diagrams, and system documentation, to ensure efficient and effective data management and analysis.
• Work closely with data engineering and various vertical teams within the Video Architecture platform and cross-functional teams within USDS.

Minimum Qualifications

• Bachelor's degree in Computer Science or a related technical background involving software/system engineering, or equivalent working experience.
• Good programming experience with SQL and at least one of the following languages: Java, Python, Go, or Scala.
• Experience in data analytics and data engineering ETL development, data pipeline design, implementation, and maintenance.
• Experience with big data ecosystem suing Spark, Flink, Hive, YARN, Kafka.

Preferred Qualifications

• Demonstrated independent thinking capabilities and troubleshooting skills in large scale distributed systems.
• Good communication and coordination skills.
• Experience in building data solutions with AWS, GCP, Azure and other cloud services is a plus.