DL
GCP Data Engineer
Diverse Lynx India
🇮🇳 India
On-site
1 month ago
- ETL
- ELT
- BigQuery
- GCP
- Dataflow
- Composer
- SQL
- Hive
- Python
- Pandas
- NumPy
- SciPy
- Matplotlib
- PySpark
- Apache Airflow
- Pub/Sub
- Airflow
1 month ago
| Data Pipeline Development: Build and maintain scalable, secure ETL/ELT data workflows using BigQuery and other GCP services (e.g., Dataflow, Cloud Storage). [1, 2] |
| Extensive Experience in IT data analytics projects, Hands on experience in migrating on premise ETLs to Google Cloud Platform (GCP) using cloud native tools such as BIG query, Cloud Data,Proc, Google Cloud Storage, Composer |
| SQL concepts, Presto SQL, Hive SQL, Python (Pandas, Numpy, SciPy, Matplotlib) and Pyspark |
| Design and implement scalable data pipelines using Google Cloud Dataflow |
| Develop and optimize BigQuery datasets for efficient data analysis and reporting |
| Collaborate with cross-functional teams to integrate data solutions with business processes |
| Automate data workflows using Cloud Composer and Apache Airflow |
| Ensure data security and compliance with GCP Identity and Access Management |
| Query & Storage Optimization: Develop advanced SQL queries and design data models using partitioning, clustering, and materialized views to minimize costs and maximize performance |
| Implement real-time data processing with Google Cloud Pub/Sub and Dataflow |
| Continuously evaluate and adopt new GCP tools and technologies |
| GCP Biq query, Python , SQL , Airflow |
| Composer |
GCP Data Engineer · Diverse Lynx India