TC
GCP Data Platform
TechDigital Corporation
- Location not stated
- Remote
- 18 hours ago
- GCP
- Composer
- Apache Airflow
- IaC
- Terraform
- Dataflow
- BigQuery
- Cloud Functions
- Cloud Run
- Pub/Sub
- Cloud SQL
- CI/CD
- Jenkins
- IAM
- Python
- Java
- ServiceNow
- FinOps
- VPC
18 hours ago
- Administer, configure, maintain and upgrade GCP Composer (Apache Airflow) environment ( Mandatory )
- Design and implement infrastructure as code using Terraform for provisioning and managing GCP resources. (Mandatory)
- Able to create Terraform modules and publish same in TFE (Terraform Enterprise )
- DAG deployment, Troubleshooting, performance tuning, and operational support.
- Support and optimize Dataflow and Dataproc workloads, including batch and streaming data processing pipelines.
- Maintain GCP infrastructure including services like:
- GCP Composer (Apache Airflow)
- Google Cloud Storage (GCS)
- BigQuery (BQ)
- Cloud Dataflow /Apache Beam
- Cloud Dataproc
- Cloud Functions --- (Mandatory)
- Cloud Run
- Pub/Sub
- Cloud SQL
- Secret Manager
- Logging & Monitoring
- Monitor GCP usage and quotas, and proactively prevent quota-related issues.
- Set up and manage alerting by Creating Alert policies in GCP Monitoring . (Mandatory)
- Manage CI/CD pipelines using Jenkins and/or using other CI/CD tools.(Mandatory)
- Able to request Identity and Access Management (IAM) permissions, service accounts, and security policies following least-privilege principles.
- Ensure security, compliance, and cost-efficiency across the cloud environment.
- Monitor cloud infrastructure health, performance, and resource utilization using Cloud Monitoring, Cloud Logging, Error Reporting, and Cloud Trace.
- Develop automation scripts and operational tools using Python and/or Java to improve platform efficiency and reduce manual effort.
- Create and maintain operational documentation, runbooks, architecture diagrams, standard operating procedures (SOPs), and platform best practices.
- Excellent problem-solving and troubleshooting skills.
- Experience with CDC platforms such as Striim
- Experience with ServiceNow integrations and operational workflows
- Knowledge of FinOps, cloud cost optimization, and governance frameworks
- Networking components understanding including:
- VPCs and Shared VPCs
- VPC Connectors
- Firewall Rules
- CIDR Ranges
GCP Data Platform · TechDigital Corporation