PL
Data Engineer
PeopleNTech LLC
🇺🇸 United States
On-site
6 months ago
- Python
- Java
- Scala
- Hive
- SQL
- AWS
- GCP
- Redshift
- Snowflake
- ETL
- AWS Glue
- EMR
- CI/CD
- Jenkins
- CloudFormation
- Docker
- Kubernetes
- BigQuery
- Dataflow
- Pub/Sub
- Composer
- GKE
- AWS Lambda
- Kinesis
- DynamoDB
- NoSQL
- VPC
- Devops
- Git
- dbt
- Disaster Recovery
- GitLab CI/CD
- Agile
6 months ago
Location : New York (100% onsite) (need local to NY)
Interview mode : ( 1 Video interview and CI will be in person)
Rate : $65/hr
Must have :
Languages & Scripting:Spark, Python, Java, Scala, Hive, Kafka, SQL
Cloud Platforms:AWS OR GCP
Data Warehousing & Analytics:Redshift or Snowflake or Big Query
Data Integration & ETL:AWS Glue, Aws EMR, Spark, Data Bricks
CI/CD:AWS Code Pipeline, Jenkins, CloudFormation, Docker, Kubernetes
JD :
- Results-driven Data Engineer with a decade of expertise in Data engineering across cloud platforms with a total of 12 years in IT.
- Extensive experience utilizing Google Cloud Platform (GCP) services, including BigQuery, Dataflow, Dataprep, and Pub/Sub, for data engineering solutions.
- Proficient in building and managing GCP data pipelines with tools like Cloud Composer and Cloud Dataflow.
- Proven ability in developing and deploying applications on Google Kubernetes Engine (GKE).
- Strong background in implementing security and compliance on GCP, ensuring data privacy and regulatory adherence.
- Track record of optimizing cost and resource usage within GCP environments.
- Skilled in AWS services such as Amazon EMR, Redshift, and Glue for efficient data processing.
- Expertise in architecting scalable, cost-effective solutions on AWS, with proficiency in configuring AWS Lambda for serverless computing.
- Adept at setting up AWS Kinesis streams to process real-time data, enhancing system responsiveness and data-driven decision-making.
- Proficient in leveraging AWS DynamoDB to create scalable, low-latency NoSQL databases for dynamic applications.
- Deep expertise in optimizing and managing Amazon Redshift data warehouses to deliver high-performance analytics and business insights.
- Experienced in integrating AWS services into CI/CD pipelines, streamlining automation for continuous integration, delivery, and deployment.
- Skilled in setting up and securing AWS Virtual Private Cloud (VPC) environments.
- Proficient in managing Azure virtual machines (VMs) for cloud infrastructure operations.
- Extensive experience managing on-premises data infrastructure, including data warehouses and databases.
- Familiar with AWS DevOps practices for continuous integration and deployment.
- Expertise in using Git for version control in DBT projects, ensuring proper tracking and documentation of data model changes.
- Skilled in performance optimization and tuning of on-premises data systems.
- Proficient in data migration strategies between on-premises and cloud environments.
- Strong troubleshooting skills in resolving issues within on-premises data systems.
- Proven ability to maintain high availability and disaster recovery solutions in on-premises environments.
- Experienced in implementing CI/CD pipelines using tools like Jenkins and GitLab CI/CD.
- Adept in automated testing processes, including unit, integration, and regression testing.
- Skilled in gathering and analyzing project requirements to ensure alignment with business goals.
- Experienced in Agile project management, contributing to successful outcomes through data-driven analytics and collaborative teamwork
Data Engineer · PeopleNTech LLC