Databricks Engineer

Job Title: Databricks Engineer

Duration: Long Term (Hybrid)
Location: Austin, TX (Locals Only)
Interview: In-Person
Visa: No OPT/CPT

Note:Please submit only recent State experience resumes with 3 professional references.

Responsibilities

• Understand business objectives and problems, identify alternative solutions, and perform studies and cost/benefit analysis of alternatives.
• Analyze user requirements, procedures, and problems to automate processing or improve existing computer systems.
• Confer with personnel of organizational units involved to analyze current operational procedures, identify problems, and learn specific input and output requirements, such as forms of data input, how data is to be summarized, and formats for reports.
• Write detailed descriptions of user needs, program functions, and steps required to develop or modify computer programs.
• Review computer system capabilities, specifications, and scheduling limitations to determine if requested programs or program changes are possible within the existing system.
• Design, develop, and optimize scalable data solutions on Databricks, leveraging PySpark or Scala for large-scale data processing.
• Build and maintain ingestion pipelines, Declarative Pipelines (DLT), and Medallion Architecture (Bronze, Silver, Gold) to support enterprise analytics and reporting.
• Develop robust data models and implement data quality, validation, and governance frameworks.
• Create dynamic dashboards, Databricks Apps, and analytical solutions to deliver actionable business insights.
• Optimize workloads, monitoring, and operational processes to ensure scalability, security, and cost efficiency.

Required Skills

• 8+ years of IT experience supporting the design, development, deployment, or delivery of technology solutions.
• 8+ years of experience with Databricks, including building and optimizing ETL/ELT data pipelines using Apache Spark.
• 8+ years of experience in data warehousing and dimensional data modeling (Star Schema and Snowflake Schema).
• 8+ years of proficiency in SQL and Python (or Scala) for large-scale data processing.
• 8+ years of experience designing and developing dashboards and applications natively within Databricks (Databricks SQL Dashboards, Databricks Apps).
• 8+ years of experience implementing data governance, data quality, and data security best practices.
• 8+ years of experience implementing Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT) for building and managing production data pipelines.
• 8+ years of experience with Delta Lake, Medallion Architecture (Bronze/Silver/Gold), Data Lakehouse design, and creating/scheduling offline jobs using Lakeflow Jobs (formerly Databricks Workflows) or orchestration tools such as Apache Airflow.
• 8+ years of excellent verbal and written communication skills, including presenting technical solutions and business insights to stakeholders.

Preferred Skills

• 1+ year of experience working in public sector or state government environments.
• 1+ year of experience with CI/CD practices for data pipelines, including DevOps and Git-based workflows.
• Databricks Certification such as Databricks Certified Data Engineer Associate or Databricks Certified Data Engineer Professional.