Regular Data Engineer

The role involves sourcing data from multiple systems, optimizing workflows and collaborating with architects and analysts to deliver clean, well-structured datasets.

Required Skills & Experience:

• 3+ years of experience in data engineering across hybrid environments (on-premise and cloud).

• Proficiency in SQL and Python or Java/Scala.

• Hands-on experience with ETL/ELT tools and frameworks.

• Good understanding of GCP data services: BigQuery, Dataproc, Dataflow, Cloud Storage.

• Familiarity with data modeling, schema design, and metadata management.

• Knowledge of data governance, security, and compliance best practices.

Nice To Have:

• GCP certification (e.g., Professional Data Engineer) is a major plus.

• Experience with Big Data technologies.

Key Responsibilities:

• Solution Design: Architect data pipelines down to the low-level elements, ensuring clarity and precision in implementation.

• Data Sourcing: Extract data from diverse repositories including relational
databases (Oracle, PostgreSQL), NoSQL stores, file systems, and other
structured/unstructured sources.

• Data Transformation: Design and implement ETL/ELT workflows to standardize and cleanse data using best practices in data engineering.

• Pipeline Development: Build scalable, fault-tolerant data pipelines that support batch and streaming use cases.

• Cloud data processing: Load transformed data into GCP destinations such as BigQuery or Cloud Storage using tools like Dataproc, Dataflow, and other GCPnative services.

• Workflow Orchestration: Design and manage workflows using orchestration tools such as Apache Airflow or Cloud Composer.

• Data Format Expertise: Work with various data formats including JSON, AVRO, Parquet, CSV, and others.

• Optimization & Monitoring: Ensure performance, reliability, and cost-efficiency of data pipelines through continuous monitoring and tuning.

• Collaboration: Work closely with data architects, analysts, and business
stakeholders to understand data requirements and deliver high-quality solutions.