Senior Databricks Engineer – Teradata Discovery & Assessment

Senior Databricks Engineer – Teradata Discovery & Assessment

Location: Las Vegas, NV, USA – Remote

Role Summary

We are seeking an experiencedSenior Databricks Engineer to lead technical discovery and assessment for Teradata modernization initiatives. The ideal candidate will analyze existing data warehouse assets, ETL processes, dependencies, and migration complexity to support modernization and migration to Databricks.

Key Responsibilities

• Analyze Teradata databases, tables, views, stored procedures, macros, and BTEQ scripts.

• Reverse engineer IBM DataStage jobs, workflows, and ETL dependencies.

• Perform data profiling, data quality assessments, and migration readiness analysis.

• Conduct technical impact analysis, data lineage mapping, and dependency assessments.

• Develop source-to-target mappings and detailed migration specifications.

• Identify transformation logic, integration patterns, technical risks, and migration challenges.

• Collaborate with data architects and stakeholders to define Databricks migration strategies.

• Design and develop Databricks notebooks and integrate them with the Gold framework.

• Understand existing frameworks quickly and support their implementation and enhancement.

• Contribute to large-scale data warehouse modernization and migration initiatives.

Required Skills and Qualifications

• Strong hands-on experience withDatabricks and PySpark.

• Extensive SQL experience and strong knowledge ofTeradata.

• Experience withIBM DataStage, including ETL jobs, workflows, and dependency analysis.

• Experience analyzing existing data warehouse environments and assessing migration complexity.

• Knowledge of data profiling, data quality, data lineage, and source-to-target mapping.

• Experience with technical discovery, impact analysis, and migration readiness assessments.

• Ability to understand existing frameworks and collaborate effectively with architects and stakeholders.

• Strong analytical, troubleshooting, and documentation skills.

Preferred Qualifications

• Experience with large-scale Teradata-to-Databricks modernization initiatives.

• Familiarity with Control-M scheduling and job dependencies.

• Experience designing and developing Databricks notebooks within established data frameworks.