EI
Data Engineer
Expert In Recruitment Solutions
🇺🇸 United States
On-site
Senior
6 months ago
- Python
- SQL
- AWS
- AWS Cloud
- DynamoDB
- Athena
- ECS
- Amazon EventBridge
- Redshift
- Machine Learning
- OpenSearch
- RDS
- Swagger
- OpenAPI
- GraphQL
- Load Balancing
- Prometheus
- Grafana
- TypeScript
- ETL
- AWS S3
- AI/ML
- Large Language Models
- Microservices
6 months ago
Location – Malvern, PA – Hybrid
Duration – 6+ months
Interview – Video
Job Description –
Required Skills - Top 3-5 Key Words
Required Technical Skills: Python, SQL, AWS Web services (Glu, S3, Lambda)
Core Programming Skills:
- Expert proficiency in Python, with experience in building data pipelines and back-end
- systems.
- Advanced knowledge of SQL for querying and optimizing large datasets.
- AWS Cloud Services Expertise:
- DynamoDB, S3, Athena, GlueETL, Lambda, ECS, Glue Data Quality, EventBridge,
- Redshift Machine Learning, OpenSearch, and RDS.
- Proven expertise in designing fault-tolerant APIs using Swagger/OpenAPI, GraphQL,
- and RESTful standards.
- Robust understanding of distributed systems, load balancing, and failover strategies.
- Monitoring and Orchestration:
- Hands-on experience with Prometheus and Grafana for observability and monitoring.
- Senior Data Engineer – 7+ Years of Experience
- We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in
- designing, building, and optimizing robust data solutions. The ideal candidate must
- possess top-tier skills in Python, AWS services, API development, and TypeScript, and
- have significant hands-on experience with anomaly detection systems.
- The candidate should have a proven ability to work at both strategic and tactical levels,
- from designing data architectures to implementing them in the weeds.
Job Requirements
Key Responsibilities:
- Data Pipeline Development
- Independently design, build, and maintain complex ETL pipelines, ensuring scalability
- and efficiency for large-scale data processing needs.
- Manage pipeline complexity and orchestration, delivering high-performance data
- products accessible via APIs for business-critical applications.
- Archive processed data products into data lakes (e.g., AWS S3) for analytics and
- machine learning use cases.
- Anomaly Detection and Data Quality
- Implement advanced anomaly detection systems and data validation techniques, ensuring data integrity and quality.
- Leverage AI/ML methodologies, including Large Language Models (LLMs), to detect and address data inconsistencies.
- Develop and automate robust data quality and validation frameworks.
- Cloud and API Engineering
- Architect and manage resilient APIs using modern patterns, including microservices,
- RESTful design, and GraphQL.
- Configure API gateways, circuit breakers, and fault-tolerant mechanisms for distributed
Data Engineer · Expert In Recruitment Solutions