AL
AWS PySpark Databricks Developer
Artech LLC
- 🌏 Worldwide
- Remote
- 1 day ago
- $40 – $47 / hour
- AWS
- PySpark
- Databricks
- Python
- Apache Spark
- AWS Cloud
- EMR
- Redshift
- CloudWatch
- IAM
- Scala
- Delta Lake
- SQL
- Airflow
- dbt
- CI/CD
- Jenkins
- GitHub Actions
- ETL
- ELT
- Step Functions
- Git
- Devops
1 day ago
Job Title: AWS PySpark Databricks Developer
Location: Remote
Duration: 6 months (Extendable)
Pay Rate: $40-$47/hr on W2
Skills:Python, Databricks, PySpark, Terradata to Databricks migration
Experience Required: 8 + Years
CANDIDATE MUST HAVE HANDSON EXP IN DATABRICKS, PYTHON AND PYSPARK
Technical/Functional Skills
- 5–6 years of total experience in data engineering or big data development.
- 2–3 years hands-on experience with Databricks and Apache Spark.
- Proficient in AWS cloud services (S3, Glue, Lambda, EMR, Redshift, CloudWatch, IAM).
- Strong programming skills in PySpark, Python, and optionally Scala.
- Solid understanding of data lakes, lakehouses, and Delta Lake concepts.
- Experience in SQL development and performance tuning.
- Familiarity with Airflow, dbt, or similar orchestration tools is a plus.
- Experience in CI/CD tools like Jenkins, GitHub Actions, or Code Pipeline.
- Knowledge of data security, governance, and compliance frameworks.
- Develop and maintain scalable data pipelines using Apache Spark on Databricks.
- Build end-to-end ETL/ELT pipelines on AWS using services like S3, Glue, Lambda, EMR, and Step Functions.
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions.
- Design and implement data models, schemas, and Lakehouse architecture in Databricks.
- Optimize and tune Spark jobs for performance and cost-efficiency.
- Integrate data from multiple structured and unstructured data sources.
- Monitor and manage data workflows, ensuring data quality, consistency, and security.
- Follow best practices in CI/CD, code versioning (Git), and DevOps practices for data applications.
- Write clean, reusable, well-documented code using Python / PySpark / Scala.
• Inclusive and diverse work environment
• Opportunities for professional growth and development
• Comprehensive health and wellness benefits
AWS PySpark Databricks Developer · Artech LLC