Data Engineer (Python, Pytorch)

Position Title:Data Engineer (Python, Pytorch)
Location:Dallas, TX. ONSITE
Duration:Long Term Contract
 
Role and Responsibilities:
Job Description:
We are seeking a highly skilled Data Engineer with strong expertise in Databricks, Python, and PyTorch to design, develop, and optimize scalable data and machine learning pipelines. The ideal candidate will be responsible for building cloud-native data platforms, enabling advanced analytics, and supporting AI/ML initiatives through robust data engineering practices. This role requires hands-on experience in large-scale data processing, feature engineering, model deployment support, and performance optimization across modern data ecosystems.

• Data Engineering & Platform Development
  • Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Python.
  • Build and optimize batch and real-time data ingestion frameworks from diverse data sources.
  • Develop ETL/ELT solutions to support enterprise data warehousing, analytics, and AI/ML workloads.
  • Implement data quality checks, validation frameworks, monitoring, and observability solutions.
  • Optimize Databricks jobs, notebooks, clusters, and workflows for performance and cost efficiency.
  • Design and maintain Delta Lake architectures, data models, and metadata management solutions.
• Machine Learning Data Enablement
  • Collaborate with Data Scientists and ML Engineers to prepare training datasets and feature stores.
  • Support development and deployment of deep learning solutions using PyTorch.
  • Develop scalable feature engineering pipelines and data transformation frameworks.
  • Enable model training, experimentation, and inference infrastructure on Databricks.
  • Integrate ML workflows with MLOps pipelines and CI/CD processes.
• Cloud & DevOps
  • Implement DataOps and MLOps best practices.
  • Automate deployments using CI/CD pipelines.
  • Work with cloud-native services for storage, security, monitoring, and orchestration.
  • Support infrastructure-as-code and environment automation initiatives.
• Collaboration & Governance
  • Partner with business, analytics, and data science teams to understand requirements and translate them into technical solutions.
  • Ensure compliance with data governance, privacy, and security standards.
  • Participate in architecture reviews and technology evaluations.

• Bachelor's or master’s degree in computer science, Data Engineering, Information Systems, or related field.
• 5 years of experience in Data Engineering.
• 3 years of experience with Databricks and cloud-based data platforms.
• Experience supporting machine learning and AI use cases using PyTorch.

• Strong experience with Databricks Platform
• Expertise in Python
• Strong hands-on experience with PySpark
• Experience with PyTorch for machine learning and deep learning workloads
• Knowledge of Delta Lake, Unity Catalog, and Databricks Workflows
• Experience with ETL/ELT pipeline development
• Strong SQL and data modeling skills
• Experience with distributed computing and big data architectures
• Knowledge of REST APIs and data integration patterns
• Git version control and CI/CD tools
• Experience with one or more cloud platforms: Microsoft Azure (preferred), AWS, Google Cloud Platform (GCP)

• Experience with GenAI and LLM data pipelines.
• Knowledge of MLflow and MLOps frameworks.
• Hands-on experience with Docker and Kubernetes.
• Experience with real-time streaming architectures.
• Familiarity with vector databases and AI platforms.
• Certifications in Databricks or Cloud platforms.
• Experience in Retail, QSR, Restaurant Technology, Digital Commerce, or Customer Analytics domains.
• Experience with large-scale enterprise data modernization initiatives.
• Experience with AI/ML and advanced analytics implementations.