DS
US_East | IT Consultant - Cloud Specialist_L2
Datum Software, Inc
- 🇮🇱 Israel
- On-site
- Staff / Principal
- 1 day ago
- AWS
- AWS Glue
- AWS Lambda
- Apache Iceberg
- Step Functions
- CI/CD
- ETL
- ELT
- Microservices
- Regulatory Compliance
- Devops
- IaC
- CloudFormation
- Terraform
- AWS CDK
- GitHub Actions
- Jenkins
- CloudWatch
- Athena
- Redshift
- IAM
- Amazon EventBridge
- SNS
- SQS
- Apache Spark
- PySpark
- Python
- SQL
- Event-Driven Architecture
1 day ago
Location: Hanover
Job Description
We are seeking an experiencedAWS Engineering Lead to design, develop, and lead enterprise-scale, cloud-native data platforms on AWS.
The ideal candidate will have strong expertise inAWS Glue, AWS Lambda, Apache Iceberg, AWS Step Functions, and modern data engineering practices.
This role involves leading a team of engineers, architecting scalable data solutions, driving cloud engineering best practices, and collaborating with business and technology stakeholders to deliver high-quality data products.
Key Responsibilities:
1. Technical Leadership
- Lead the architecture, design, and implementation of scalable AWS-based data platforms.
- Provide technical direction, guidance, and mentorship to engineering teams.
- Establish engineering standards, coding best practices, and CI/CD processes.
- Conduct technical design reviews, code reviews, and architecture governance activities.
- Design and develop data ingestion, transformation, and orchestration pipelines using AWS services.
- Build, optimize, and maintain ETL/ELT workflows using AWS Glue.
- Develop scalable serverless solutions using AWS Lambda.
- Design and implement workflow orchestration using AWS Step Functions.
- Build and manage modern data lake solutions using Apache Iceberg on Amazon S3.
- Architect highly available, secure, scalable, and cost-efficient cloud solutions.
- Design event-driven and microservices-based architectures.
- Optimize the performance, reliability, scalability, and operational efficiency of AWS workloads.
- Implement metadata management, schema evolution, partitioning, and time-travel capabilities using Apache Iceberg.
- Support large-scale batch and near-real-time data processing requirements.
- Ensure data quality, governance, security, and regulatory compliance.
- Implement Infrastructure as Code (IaC) using AWS CloudFormation, Terraform, or AWS CDK.
- Build and maintain automated CI/CD deployment pipelines using GitHub Actions, Jenkins, or AWS CodePipeline.
- Monitor system performance and operational health using Amazon CloudWatch and monitoring dashboards.
- Drive automation, reliability, and continuous improvement across AWS environments.
- Collaborate with Product Owners, Architects, Business Teams, and Technology Stakeholders.
- Translate business requirements into scalable and reliable technology solutions.
- Drive project delivery while proactively managing technical risks, dependencies, and challenges.
Required Skills & Experience:
AWS Services
Strong hands-on expertise with:
- AWS Glue
- AWS Lambda
- AWS Step Functions
- Amazon S3
- Amazon Athena
- Amazon Redshift
- AWS Identity and Access Management (IAM)
- Amazon CloudWatch
- Amazon EventBridge
- Amazon SNS and Amazon SQS
- AWS Secrets Manager
- AWS Lake Formation
- Extensive experience designing and building enterprise-scale data lakes.
- Strong understanding of Apache Iceberg architecture and implementation.
- Expertise in ETL/ELT frameworks and data pipeline development.
- Strong knowledge of partitioning strategies, schema evolution, data compaction, and time-travel concepts.
- Experience with Apache Spark and PySpark is preferred.
- Python – Expert-level proficiency
- PySpark
- SQL
- Shell Scripting
- Serverless Architecture
- Event-Driven Architecture
- Microservices Architecture
- API Integration
- Data Lake Architecture
- Terraform, AWS CloudFormation, and AWS CDK
- GitHub Actions and Jenkins
- CI/CD Pipeline Development
- Automated Testing
“All qualified applicants will be considered without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.”
US_East | IT Consultant - Cloud Specialist_L2 · Datum Software, Inc