Databricks Architect

Job Description: Databricks Architect

Job Details

• Location: Louisville, KY

• Work Location: 101 E Main Street, Louisville, KY 40202 / Remote

• Duration: 6 months

• Experience Required: 8–10 years

• Base Salary Range: Minimum and maximum salary to be specified

• Role: Google Gen AI Cloud | Databricks Architect

Must-Have Technical/Functional Skills

• Expertise in designing, delivering, and implementing Google Cloud Platform (GCP) infrastructure and solutions.

• Experience architecting and implementing large-scale migration pipelines from on-premises systems to Azure Data Lake Storage (ADLS).

• Strong experience in designing and developing data pipelines using Azure Data Factory (ADF).

• Strong knowledge of GCP AI/ML services and infrastructure.

• Proficiency in AI/ML frameworks such as TensorFlow or PyTorch.

• Strong technical skills in Python, data modeling, SQL, and APIs.

• Hands-on experience with Terraform for Infrastructure as Code (IaC) and automation.

• Proven experience delivering AI/ML solutions in cloud environments.

• Experience performing data analytics and generating insights for stakeholders.

• Experience ingesting data from multiple sources using Azure Data Factory.

• Ability to handle high-volume unstructured data with reliability, performance, and cost efficiency.

• Experience implementing data quality checks, logging, and traceability to meet audit and compliance requirements.

• Hands-on experience transforming data using Azure Databricks and Delta Lake.

Roles and Responsibilities

- Architecture and Design:

• Design enterprise-grade AI platforms, including Retrieval-Augmented Generation (RAG) and agentic AI solutions.

• Develop scalable, secure, and high-performance cloud architectures.

- GCP Implementation:

• Develop and deploy AI models using Vertex AI and Gemini.

• Ensure model security, scalability, and performance.

- Pipeline Development:

• Build and manage data pipelines and model lifecycle workflows.

• Implement CI/CD, MLOps, and LLMOps practices.

• Utilize Cloud Build and Vertex AI Pipelines for automation and deployment.

- Cloud Infrastructure:

• Leverage Google Kubernetes Engine (GKE), Cloud Run, and BigQuery for robust application deployment and data processing.

• Design and manage scalable cloud infrastructure.

- AI Strategy and Governance:

• Implement AI safety features and governance frameworks.

• Monitor model performance and reliability.

• Optimize infrastructure and model costs.

Generic / Managerial Skills

• Experience working in Agile development projects.

• Strong collaboration and communication skills.

• Ability to work with cross-functional teams and business stakeholders.

Essential Skills

• Google Gen AI Cloud

• Google Cloud Platform (GCP)

• Databricks Architecture

• Generative AI and Agentic AI

• Vertex AI and Gemini

• Azure Data Factory (ADF)

• Azure Data Lake Storage (ADLS)

• Azure Databricks and Delta Lake

• AI/ML and MLOps/LLMOps

Desirable Skills

• Additional cloud architecture and AI platform implementation experience.

• Experience with enterprise-scale data migration and governance.

• Knowledge of AI security, compliance, and cost optimization.

Keywords

• Google Gen AI Cloud

• GCP

• Databricks Architect

• Generative AI

• Agentic AI

• RAG

• Vertex AI

• Gemini

• TensorFlow

• PyTorch

• Python

• Terraform

• Azure Data Factory

• Azure Data Lake Storage

• Azure Databricks

• Delta Lake

• BigQuery

• GKE

• Cloud Run

• MLOps

• LLMOps

• CI/CD

• Data Engineering

• AI/ML Solutions