L
Associate Principal - Cloud Engineering
LTM
🇺🇸 United States
On-site
Staff / Principal
2 months ago
- Databricks
- AWS
- AWS Cloud
- PySpark
- SQL
- Delta Lake
- Redshift
- Amazon EventBridge
- IAM
- CI/CD
- IaC
- Terraform
- Unity Catalog
- AWS Glue
- AWS Lambda
- AWS IAM
- CloudWatch
- SNS
- SQS
- KMS
- Apache Spark
- Python
- Data Modeling
- Snowflake
- Microsoft Fabric
- GCP
- BigQuery
- Airflow
- dbt
- Docker
- Kubernetes
- OpenShift
- Informatica
- Collibra
- Vault
2 months ago
Job Title:DATABRICKS Architect
Work Location
Irving TX
Job Description:
Role Overview
- We are seeking a highly skilled Databricks Engineer with AWS expertise to support Data Services portfolio
- The role will be responsible for designing developing optimizing and supporting enterprise scale data platforms data pipelines and analytics solutions leveraging Databricks Lakehouse AWS Cloud and modern data engineering practices
- The candidate will work closely with data architects’ application teams business stakeholders and platform engineering teams to deliver scalable secure and high-performance data solutions
Key Responsibilities
- Design develop and maintain scalable data pipelines using Databricks PySpark Spark SQL and Delta Lake
- Build and support enterprise data lake and Lakehouse solutions on AWS
- Develop batch and Realtime ingestion frameworks from multiple source systems
- Implement data transformation enrichment cleansing and validation processes
- Optimize Spark workloads for performance scalability and cost efficiency
- Design and implement data models supporting reporting analytics and downstream applications
- Integrate Databricks with AWS services including S3 Glue Lambda Redshift EventBridge and IAM
- Support data migration and modernization initiatives from traditional data warehouses to Databricks Lakehouse
- Implement CICD pipelines and Infrastructure as Code IaC using Terraform and Gitbased repositories
- Collaborate with security teams to ensure compliance with client governance risk and data security standards
- Troubleshoot production incidents and provide L3 operational support
- Participate in architecture reviews code reviews and technical governance activities
Mandatory Technical Skills
- Databricks
- Databricks Lakehouse Architecture
- Databricks Workflows
- Delta Lake
- Unity Catalog
- Databricks SQL
- Auto Loader
- Structured Streaming
- Databricks Performance Tuning
- Medallion Architecture
- AWS
- Amazon S3
- AWS Glue
- AWS Lambda
- AWS IAM
- Amazon Redshift
- AWS CloudWatch
- SNS SQS
- EventBridge
- KMS
- Secrets Manager
- Data Engineering
- PySpark
- Apache Spark
- Python
- SQL
- Data Warehousing Concepts
- ETLELT Design
- Data Modeling
- Data Quality Frameworks
- Batch RealTime Processing
Preferred Skills
- Snowflake
- Microsoft Fabric
- GCP BigQuery
- Kafka
- Airflow
- dbt
- Terraform
- Docker
- Kubernetes OpenShift
- Informatica
- Collibra
- Data Vault Modeling
- Dimensional Data Modeling
- Banking Domain Experience
- Strong experience within Banking and Financial Services
- Understanding of enterprise data governance regulatory reporting risk compliance and audit requirements
- Experience working in large global organizations with stringent security and operational standards
Qualifications
- Bachelors or Masters degree in Computer Science Engineering Information Systems or related discipline
- Databricks Certified Data Engineer Preferred
- AWS Certified Data Analytics or AWS Solution Architect Preferred
Associate Principal - Cloud Engineering · LTM