Senior/Lead Data Engineer

We are looking for aSenior/Lead Data Engineerto build telemetry and cost-attribution data pipelines that map real-time accelerator utilization to workloads and teams.

Responsibilities

• Build BigQuery/Dataflow pipelines ingesting TensorCore utilization, FLOPs, and network saturation metrics
• Develop cost-attribution dashboards mapping accelerator spend to workloads/teams
• Design reliable, high-throughput data pipelines (Airflow, Cloud Composer, Dataflow)
• Manage complex data ingestion and preprocessing for telemetry data
• Advise on cloud storage architecture for large-scale telemetry data

Requirements

• Strong GCP data stack experience (BigQuery, Dataflow, Cloud Composer)
• Proficiency in SQL and pipeline orchestration tools (Airflow)
• Experience building distributed, high-throughput data pipelines
• Understanding of AI/ML data requirements
• Advanced knowledge of cloud storage systems

Nice to have

• Experience with real-time streaming telemetry systems
• FinOps/cost-attribution dashboard design experience