Data Engineer
- πΊπΈ United States
- On-site
- 9 hours ago
- $50 β $55 / hour
- Kinesis
- Oracle
- Parquet
- Apache Iceberg
- Python
- Java
- Apache Spark
- Flink
- REST API
Data Engineer
Location: Charlotte, NC
Experience: 10+ Years
Work Arrangement: Day 1 Onsite (Local)
Position IDs: 114068-20-1 and 114068-21-1
In-Person is mandatory
Key Responsibilities
-
Optimize data pipelines that process data from Amazon Kinesis to Oracle, improving exception handling for schema and data type mismatches and implementing data quality alerts.
-
Develop services to read and write data to NetApp Object Storage using S3 APIs.
-
Create analytical datasets in Parquet and Apache Iceberg formats.
-
Develop housekeeping scripts for log cleanup, archiving, compression, and object storage file management.
-
Required Skills
-
Strong expertise in Python, Java, and Data Engineering.
-
Hands-on experience building and deploying data pipelines using Apache Spark, Apache Flink, and distributed data processing frameworks.
-
Experience with real-time streaming architectures, REST APIs, and database connectors.
-
Proficiency in relational databases, particularly Oracle.
-
Experience with Amazon S3 APIs for data storage and integration.
-
Preferred Skills
-
Experience with Apache Iceberg.
Data Engineer Β· Apolis