Data Operations Engineer / DataOps Specialist

Job Title: Data Operations Engineer / DataOps Specialist
Request-ID: 95578-1
Location: Philadelphia, PA
Duration: 6+ months
Pay Range: $45.00- 50.00 /Hour

We are seeking an experiencedSenior Data Operations Engineer (DataOps Specialist) with10+ years of experience in designing, building, and managing modern data platforms and AI-enabled data pipelines. The ideal candidate will have strong expertise inPython, LangChain, LangGraph, Vector Databases, LLM applications, ETL/ELT frameworks, cloud platforms, and DataOps best practices.

This role is responsible for building scalable data pipelines, ensuring data quality, automating workflows, monitoring production systems, and supporting AI/GenAI data platforms across enterprise environments.

Required Experience

• 10+ years of experience in Data Engineering, Data Operations, or DataOps.
• Strong experience building enterprise-scale ETL/ELT pipelines.
• Hands-on experience with Python-based data engineering and automation.
• Experience working in Agile and DevOps environments.

Primary Skills

• Python
• LangChain
• LangGraph
• Large Language Models (LLMs)
• Vector Databases (Pinecone, Milvus, Weaviate, ChromaDB, FAISS)
• Prompt Engineering
• AI Data Pipelines

Secondary Skills

• PL/SQL
• ETL Tools
• Apache Spark
• Hadoop Ecosystem
• AWS / Azure / GCP
• Data Modeling
• Data Architecture
• REST APIs
• Data Integration
• Git
• CI/CD Pipelines
• Docker
• Kubernetes
• Jenkins
• Terraform (preferred)

Key Responsibilities

Data Pipeline Development

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build automated workflows for data ingestion, transformation, and processing.
• Support both batch and real-time data processing frameworks.
• Develop reusable Python components for enterprise data operations.
• Integrate structured and unstructured data sources.

AI & LLM Data Engineering

• Build Retrieval-Augmented Generation (RAG) pipelines using LangChain and LangGraph.
• Develop AI workflows utilizing Vector Databases.
• Integrate enterprise applications with Large Language Models.
• Optimize prompt engineering strategies and LLM orchestration.
• Manage embeddings, document indexing, and semantic search pipelines.

Data Quality & Governance

• Implement automated data validation and reconciliation processes.
• Develop data quality monitoring frameworks.
• Ensure consistency, integrity, and reliability of enterprise datasets.
• Support metadata management and data lineage.
• Enforce enterprise data governance standards.

Monitoring & Incident Management

• Monitor production data pipelines and workflows.
• Detect and resolve pipeline failures.
• Perform root cause analysis for data issues.
• Implement alerting and observability solutions.
• Ensure SLA compliance and operational stability.