A
Senior Data Engineer
Apolis
- 🇺🇸 United States
- On-site
- Senior
- 11 hours ago
- $60 – $65 / hour
- Power BI
- Python
- SQL
- AWS S3
- PySpark
- AWS Glue
- Delta Lake
- Parquet
- AWS Lambda
- AWS
- REST API
- Git
- Databricks
- Microsoft Fabric
- GMP
- Oracle Database
- Azure Data Factory
- Step Functions
11 hours ago
Location: Remote (Resource will need to work from Indianapolis office at least 50% of the capacity. Expenses will be covered.)
Duration: 6+ Months
Only W2
Senior Data Engineer (8+ years) will be responsible for designing, building, and maintaining data pipelines that extract data from source systems, transform it through the medallion
architecture layers, and prepare it for consumption by the analytics layer. This role will work closely with the Power BI Developer to ensure data is properly structured, documented, and accessible for reporting and analytics.
Technical Skills - Required
Skill Area
Python
SQL
AWS S3 / Data Lake Architecture
PySpark / AWS Glue
Delta Lake / Parquet
AWS Lambda
AWS DMS / CDC Patterns
REST API Integration
Git / Version Control
Databricks
Microsoft Fabric
Data Products
GMP experience is a preferred.
Location: Resource will need to work from Indianapolis office at least 50% of the capacity. Expenses will be covered.
Key Responsibilities
Phase 1 Deliverables
• Following EDB standard, design and establish AWS S3 bucket structure for data lake (Bronze/Silver/Gold zones) with Red CCI security controls
• Build new data pipelines to extract data from the selected SaaS-based HSE systems
• Enhance/extend data pipelines using the dataset from AWS EDB Asset related data products and digital solutions
• Build data ingestion pipeline for Manufacturing Execution System via REST API (Lambda-based extraction).
• Data pipeline to SaaS based tools to integrate data into the data warehouse
• Implement CDC pipeline for LabVantage LIMS using AWS DMS from Oracle database (or Azure Data Factory)
• Develop Bronze-to-Silver transformations using AWS Glue or Azure Data Factory depending on data domains
• Configure AWS Glue Data Catalog with appropriate metadata and Red CCI classification tags
Phase 2 Deliverables
• Connect to EDB marketplace for enterprise reference data
• Build Silver-to-Gold transformations creating batch-centric data products
• Implement data quality checks and monitoring dashboards
• Develop orchestration workflows using AWS Step Functions
Senior Data Engineer · Apolis