Senior Databricks Engineer (Platform & Data Engineering)

Location:

Work from home (Pennsylvania)

Shift:

Days (United States of America)

Scheduled Weekly Hours:

40

Worker Type:

Regular

Exemption Status:

Yes

Job Summary:

We are seeking a Databricks Platform Administrator / Data Engineer who enjoys development, application configuration, and building innovative solutions. This role is ideal for someone who thrives on possibilities, sees the bigger picture, and consistently delivers the best version of any task. You will architect solutions from conversational ideas, break work into actionable tasks in Microsoft Planner or Azure DevOps, and collaborate across a dynamic team to drive data platform excellence.

You'll be responsible for the performance of tasks related to application implementation, adoption, enhancement, and support. This includes but is not limited to analysis, development, training, testing, documentation, go-live and on-going support, problem resolution, and maintenance.

Job Duties:

Key Responsibilities

• Databricks Administration: Manage workspaces, clusters, jobs, security configurations, libraries, and overall platform health.
• Data Engineering: Build and optimize data pipelines and transformations usingPySpark andT‑SQL.
• Automation & Development: Create automation and tooling usingPython,Bash, andPowerShell.
•  Governance & Security: Implement and automate data governance, access controls, auditability, and compliance best practices.
• Note: A significant portion of our development focuses on governance and security automation.
• Solution Architecture: Translate conversational ideas into clear solution architectures; break work into tasks inPlanner orAzure DevOps for team execution.
• Effort Estimation: Estimate level of effort and forecast delivery dates with reasonable accuracy.
• Documentation: Produce high-quality documentation including workflows, gap analyses, design specs, build docs, test plans, and transition documentation.
• Testing Oversight: Oversee unit, scenario-based, integrated, capacity, and parallel testing.
• On-Call Support: Participate in a 24×7×365 on-call rotation to supportDatabricks,SQL Server, and other relevant applications supported by the team.
• Upgrades & Monitoring: Support system upgrades, patches, performance tuning, observability, and proactive monitoring.
• Data Integrity: Maintain accountability for data integrity across applications and pipelines.
• Change Management: Complete system build and documentation in adherence to departmental guidelines and change management policy.
• Peer Review: Review teammates’ work and provide constructive feedback.
• Communication & Escalation: Communicate effectively with peers, stakeholders, and leadership; provide regular status updates and escalate issues appropriately.
• Facilitation & Leadership: Drive discussions, mentor accountable leaders, and coordinate resources across initiatives.
• Training & Mentorship: Provide feedback on training materials; conduct training sessions; develop and mentor other team member.
• Policies & Vendor Engagement: Help develop department guidelines, standards, procedures, and policies; facilitate vendor reviews and selection processes.
• Integrations (as applicable): Demonstrate understanding of integration and technical requirements for peripherals/devices that interface with assigned applications.
• Support Coordination: Coordinate remote and/or on-site support as needed for end-users.

Optional Contributions

• Create complexAI/BI dashboards.
• Contribute toCI/CD pipelines and integrations usingAzure DevOps orGitHub.
• ApplyTerraform for infrastructure as code.
• ProvideMS SQL Server administrative support.

Work is typically performed in an on-campus or home office environment. Accountable for satisfying all job specific obligations and complying with all organization policies and procedures. The specific statements in this profile are not intended to be all-inclusive. They represent typical elements considered necessary to successfully perform the job.

*Relevant experience may be a combination of related work experience and degree obtained (Associate’s Degree = 2 years; Bachelor’s Degree = 4 years).

Position Details:

Required Skills & Qualifications

• Solid experience withDatabricks administration, including workspace management, cluster configuration, security, jobs, and libraries.
• Strong proficiency inPython and scripting withBash andPowerShell.
• Experience withPySpark and/orT‑SQL for data processing and transformation.
• Excellent communication skills with the ability to articulate technical concepts clearly to varied audiences.
• Demonstratedcritical thinking,problem-solving, andtroubleshooting skills.
• Ability toestimate effort anddeliver on time.
• Strong understanding ofdata governance principles and best practices.
• Commitment toinformation security best practices.

Preferred Qualifications

• Familiarity withTerraform for infrastructure as code.
• Experience creatingAI/BI dashboards.
• Knowledge ofCI/CD pipelines and integration usingAzure DevOps orGitHub.
• Hands-on experience withAWS data ingestion tools (e.g.,Glue,Lambda).
• Experience withMS SQL Server administration.
• Knowledge ofcloud architecture andmulti-cloud environments.

Education:

High School Diploma or Equivalent (GED)- (Required)

Experience:

Minimum of 3 years-Managing people, processes, or projects (Required), Minimum of 12 years-Relevant experience* (Required)

Certification(s) and License(s):

Skills:

Communication, Critical Thinking, Interpersonal Communication

OUR PURPOSE & VALUES:Everything we do is about caring for our patients, our members, our students, our Geisinger family and our communities.

• KINDNESS:We strive to treat everyone as we would hope to be treated ourselves.
• EXCELLENCE:We treasure colleagues who humbly strive for excellence.
• LEARNING:We share our knowledge with the best and brightest to better prepare the caregivers for tomorrow.
• INNOVATION: We constantly seek new and better ways to care for our patients, our members, our community, and the nation.
• SAFETY: We provide a safe environment for our patients and members and the Geisinger family. 

We offer healthcare benefits for full time and part time positions from day one, including vision, dental and domestic partners. Perhaps just as important, we encourage an atmosphere of collaboration, cooperation and collegiality.

We know that a diverse workforce with unique experiences and backgrounds makes our team stronger. Our patients, members and community come from a wide variety of backgrounds, and it takes a diverse workforce to make better health easier for all.  We are proud to be an affirmative action, equal opportunity employer and all qualified applicants will receive consideration for employment regardless to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or status as a protected veteran.