TC
Senior Data Engineer
TechDigital Corporation
🇺🇸 United States
On-site
Senior
10 months ago
- Hadoop
- Databricks
- ETL
- Java
- Scala
- Python
- Data Modeling
- Agile
- Scrum
10 months ago
| Skill | Last Used | Experience in Years/month | Hands on Exp. Yes/No |
| Big Data | |||
| Hadoop/ Spark | |||
| Databricks/ Cloudera | |||
| ETL |
Job Description:
Senior Data Engineer who is passionate about data and analytics, highly motivated and can take technical ownership of data solutions with an emphasis on engineering efficiency and on-time delivery.
The ideal candidate is exposed to the fast-paced world of Big Data technology and has experience in building data solutions using new and emerging technologies while maintaining stability of the platform. The person will get a chance to work with diverse datasets and be on the cutting edge of transforming the way Client captures, processes, stores and visualizes data.
Role-
In this role, you will:
- Design & Develop batch and streaming solutions using various Big Data technologies
- Work closely with architect to design secure and scalable
- Be responsible for assessing technologies and approaches for ingestion, transformation and storage
- Develop software utilizing open source technologies to interface with distributed and relational data sources
- Work closely with team members from across Client to identify functional and system requirements
The ideal candidate for this position should:
- Hands-on experience with Big Data and analytics technologies including Hadoop, Spark, and related frameworks.
- Programming expertise in languages such as Java, Scala, or Python, with an emphasis on developing data processing and transformation logic.
- Real-time and batch data processing skills with experience in building robust data pipelines for real-time or near-real-time ingestion.
- Proficiency in data modeling and schema design for relational and non-relational databases, supporting high data volume, and optimized query performance.
- Exposure to cloud technologies with hands-on experience in Databricks and Cloudera, including knowledge of cloud architecture, data security, and scalability best practices.
- Experience in optimizing and managing ETL workflows using scheduling, monitoring, and debugging tools.
- Strong problem-solving abilities with a mindset for automation and efficiency improvements.
- Excellent communication and collaboration skills to work effectively across technical and business teams, facilitating knowledge sharing and alignment on data engineering best practices.
- Proven track record of delivering software projects and willingness to roll up sleeves to get the job done,
- Have excellent oral and written communication
- Be familiar with Agile/Scrum methodologies
- Have the ability to manage multiple priorities simultaneously
- Have a BS/BA degree in Computer Science, Information Systems or related field
Senior Data Engineer · TechDigital Corporation