Senior Big Data Engineer with DevOps Experience

Position: Senior Big Data Engineer with DevOps Experience
Location: Pittsburgh, PA

Position Overview

We are looking for aSenior Big Data Engineer with strong DevOps expertise to join our team in Pittsburgh, PA. This role is ideal for professionals with8+ years of experience working with large-scale data platforms, automation frameworks, and production data operations.

The candidate should have hands-on expertise inHDFS, Hive, Impala, PySpark, Python, and DevOps tools likeJenkins anduDeploy. The role focuses on building scalable data pipelines, managing platform stability, and driving CI/CD practices across multiple environments.

Key Responsibilities

• Manage and maintain large-scaleBig Data environments, including HDFS table operations and data lifecycle governance.
• Develop, optimize, and monitorETL pipelines usingPySpark andPython.
• Ensure effectivecode governance and configuration management across development, testing, and production environments.
• Design and implementDevOps automation workflows using tools likeuDeploy andJenkins.
• Lead and coordinateplatform upgrades, patching, and performance tuning efforts.
• Provideproduction support, identify root causes, and resolve incidents to ensure high availability of data services.
• Collaborate with data engineers, analysts, and stakeholders to buildrobust, scalable data solutions.
• Champion best practices inCI/CD, monitoring, infrastructure automation, and data platform reliability.

Required Skills & Qualifications

• 8+ years of experience in Big Data Engineering and DevOps.
• Proficiency in:
  • HDFS, Hive, Impala
  • PySpark & Python
  • Jenkins, uDeploy
• Strong understanding ofdata platform governance, security, and performance optimization.
• Experience working inmulti-environment (dev/test/prod) data ecosystems.
• Excellent problem-solving and troubleshooting skills in a production support setting.