A
Data Engineer
Apolis
- 🇺🇸 United States
- On-site
- 16 hours ago
- $60 – $65 / hour
- PySpark
- Apache Spark
- Databricks
- SQL
- Data Architecture
- ETL
- Delta Lake
- Apache Iceberg
- Oracle
- Git
- CI/CD
- AWS
16 hours ago
Position Title:Data Engineer
Location:Round Rock, TX (REMOTE)
Duration:Long Term Contract
Role and responsibilities:
The Data Engineer will be responsible for designing, developing, and maintaining data processing systems and pipelines. The candidate will work with various data engineering tools and technologies to ensure efficient data flow and transformation. They will also be responsible for performance tuning and troubleshooting of data processes, ensuring data quality and integrity, and collaborating with other teams to meet business requirements.
Job description:
This role requires a highly skilled Data Engineer who can leverage their expertise in PySpark, Apache Spark, Databricks, and SQL to build and optimize data pipelines. The candidate should have a strong understanding of data architecture and be able to implement solutions that ensure data is processed efficiently and accurately. They will also need to demonstrate strong problem-solving abilities and excellent communication skills to effectively collaborate with team members and stakeholders.
Basic Qualifications:
- 5–8 years of experience in Data Engineering / ETL Development
- Strong analytical, problem-solving, and communication skills
- Strong hands-on experience with PySpark, Apache Spark, Databricks, and Spark SQL
- Experience developing large-scale ETL/data processing pipelines
- Strong knowledge of DataFrames, joins, transformations, partitioning, and window functions
- Experience in Spark performance tuning, including broadcast joins, caching, shuffle, and memory optimization
- Hands-on experience with Delta Lake and/or Apache Iceberg
- Experience with incremental loads, CDC, MERGE/Upsert, Insert, Update, and Delete
- Experience with Databricks Jobs, Workflows, clusters, monitoring, and troubleshooting
- Strong SQL skills and experience with databases such as Oracle
- Understanding of Bronze, Silver, and Gold (Medallion) architecture
- Experience with data quality, reconciliation, production support, and performance troubleshooting
- Knowledge of defect triaging
- Knowledge of Git, CI/CD, and cloud-based Data Engineering
- Knowledge of AWS, particularly Amazon S3, including buckets, objects, access/permissions, and integration with Databricks
- Exposure to other AWS data services is an added advantage
Data Engineer · Apolis