Databricks Engineer
- Databricks
- CI/CD
- Unity Catalog
- RBAC
- AWS
- Azure
- GCP
- Power BI
- Delta Lake
- Devops
- IaC
- Terraform
JOB_DESCRIPTION
Client : Infosys
Job Title : Databricks Engineer
Location : East Coast, NJ (Onsite)
We are seeking a highly skilled Databricks Platform Engineer to join existing engineering team. This role is focused on designing, optimizing, and maintaining the Databricks platform and architecture, enabling development teams to efficiently build and deploy data solutions.
The ideal candidate will act as a platform expert, ensuring scalability, reliability, governance, and performance of the Databricks ecosystem while supporting developers with best practices, tools, and frameworks.
Key Responsibilities
Platform Engineering & Architecture
• Design, implement, and maintain Databricks platform architecture to support enterprise-scale data workloads
• Establish best practices for workspace setup, cluster management, and job orchestration
• Optimize platform performance, scalability, and cost efficiency
• Define and enforce standards for data engineering and analytics workloads within Databricks
Developer Enablement
• Support development teams by providing guidance, documentation, and reusable frameworks
• Enable developers with CI/CD pipelines, automation, and deployment standards
• Troubleshoot and resolve platform-related issues impacting development teams
• Provide technical mentorship and onboarding support to engineers using Databricks
Governance, Security & Compliance
• Implement and manage data governance, access controls, and security models (Unity Catalog, RBAC, etc.)
• Ensure compliance with enterprise data policies and regulatory requirements
• Monitor and audit platform usage, access patterns, and data lineage
Integration & Ecosystem Management
• Integrate Databricks with cloud platforms (AWS/Azure/GCP), data lakes, and enterprise systems
• Manage connectivity with tools such as Power BI, APIs, data ingestion frameworks, and orchestration tools
• Support ingestion and processing frameworks (e.g., batch, streaming, Delta Lake)
Operational Excellence
• Monitor platform health, performance, and usage metrics
• Implement automation for provisioning, monitoring, and scaling
• Drive continuous improvements in platform reliability and developer experience
Required Qualifications- Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
- 5+ years of experience indata engineering or platform engineering roles
- 3+ years of hands-on experience withDatabricks (architecture, administration, or platform engineering)
- Strong understanding ofdistributed data processing (Spark)
- Experience withcloud platforms (Azure preferred, or AWS/GCP)
- Knowledge ofdata lake architecture, Delta Lake, and modern data platforms
- Experience implementingsecurity, governance, and access control frameworks
- Familiarity withCI/CD pipelines, DevOps practices, and infrastructure as code (Terraform, etc.)
Databricks Engineer · Info Way Solutions LLC