Sr. Data Engineer

Job Title:Senior Data Engineer (Databricks)
Primary Location:Chicago, IL (Hybrid – 3 days onsite / 2 days remote)
Position Type:Full-Time, Direct Hire

• Bachelor's degree in Computer Science, Engineering, Data Science, or a related field
• 6+ years of hands-on data engineering experience, with a track record of owning pipelines end to end: ingestion, validation, transformation, and governance
• Real, hands-on Databricks experience (Azure preferred; AWS or GCP acceptable), including Delta Lake, Unity Catalog, Databricks SQL, Spark clusters, and Workflows/Jobs
• Strong coding skills in Python, SQL, and Apache Spark that go well beyond basic scripting
• Proven experience designing and building scalable ETL/ELT pipelines, including reusable, metadata-driven pipelines
• Experience with object storage, lakehouse engines, orchestration tools, and streaming or CDC pipelines
• Solid data modeling, schema design, and performance tuning experience
• Working knowledge of Git-based workflows, CI/CD, test automation, and monitoring and alerting
• Strong troubleshooting skills and a solid grasp of engineering fundamentals: scalability, reliability, and maintainability
• The ability to clearly explain what you built, why it mattered, and how it solved a business or platform problem

• Experience building or migrating a data platform from the ground up
• Infrastructure as Code (Terraform, Bicep, ARM) and Governance as Code experience
• Multi-cloud background with depth in Azure
• Legacy data warehousing and/or Hadoop experience
• Deep Databricks exposure beyond basic pipelines, such as enterprise-scale work and optimization
• A track record of mentoring and upskilling peers
• Agile delivery experience

• Lead the design and build of scalable data pipelines and models in Azure Databricks on a brand-new platform with no production code yet
• Own pipelines end to end, from ingesting source systems to transforming and modeling data to producing outputs for downstream analytics, BI, and AI/ML use cases
• Refactor and migrate multiple legacy data warehouses into the centralized Databricks lakehouse
• Work at the code level every day in Python, SQL, and Spark, building pipelines with Databricks Asset Bundles, Spark workflows, and API-driven jobs
• Establish engineering standards for CI/CD (GitHub/Azure DevOps), automated data testing, monitoring, and alerting
• Help shape and enforce standards for data modeling and governance
• Troubleshoot and optimize performance across the platform
• Mentor and coach junior engineers, conduct code reviews, and lead by example in design, coding, and testing practices
• Partner with Data Engineering, BI, AI, Finance, and product owners, and help translate technical work into business impact

The expected salary range for this position i s $150,000-$164,000 per year, depending on experience and qualifications. This role also qualifies for comprehensive benefits such as health insurance, 401(k), and paid time off. TalentFish is committed to pay transparency and equal opportunity. The salary range provided is in compliance with applicable state and federal regulations.
This role requires authorization to work in the U.S. without current or future visa sponsorship.
All offers are contingent upon the completion of a background check, which may include but is not limited to reference checks, education verification, employment verification, drug testing, criminal records checks, and any required certifications or compliance requirements based on the end client's background check policies and applicable laws.
TalentFish is an employee-owned company pioneering a new realm in talent acquisition. We are redefining IT staffing by evolving AI, video screening, and our unique platform. TalentFish focuses on providing the best employee, consultant, and client experience possible. At TalentFish we are an Equal Opportunity Employer; we embrace and encourage diversity!