VI
Sr Data Engineer
VARITE INDIA PRIVATE LIMITED
🇮🇳 India
On-site
Senior
7 months ago
- IoT
- SQL
- Python
- triage
- dbt
- AWS
- Databricks
- Great Expectations
- AWS Glue
- PySpark
- Athena
- IAM
- CI/CD
7 months ago
About The Client:
An Indian multinational information technology (IT) consulting company headquartered in Noida, The company has offices in 52 countries and over 225,944 employees. The Client is a global IT services and consulting company that offers a wide range of services and products across various industries like IT Infrastructure Services, Cybersecurity Services, Cloud Services, Big Data and Analytics, Internet of Things (IoT) Solutions, Semiconductor Services and Enterprise Software Products.
About The Job:
- We are looking for a hands-on engineering heavy-hitter to join our Customer Data & Intelligence (CDI) function immediately. We have a rich dataset covering multiple global markets, but our legacy codebase (monolithic SQL and Python scripts) is fragile.
- Your immediate mission is to triage, refactor, and stabilize our critical data pipelines.
- You will take "God Queries" and break them down into modular, testable, and performant dbt models
Immediate Deliverables
- The "Code Rescue": Audit and patch critical queries currently causing data corruptions. Fix logic errors.
- Modularization Pilot: Implement dbt (Data Build Tool) within our AWS/Databricks environment. Migrate the most critical reporting tables from stored procedures/scripts into dbt models.
- Automated Quality Gates: Deploy automated tests (using dbt tests or Great Expectations) to check for identity uniqueness and any data errors on critical columns. Stop bad data before it hits the dashboard.
- Refactoring: Rewrite inefficient legacy SQL to improve performance and readability.
- Pipeline Repair: Fix error handling in existing AWS Glue/PySpark jobs .
- Standardization: Establish the "Gold Standard" for what good code looks like. Create the Pull Request template and SQL linting rules that the rest of the team must follow.
- Mentorship: Act as the "Bar Raiser" in code reviews, establishing standards and teaching the existing team how to write modular, defensive code.
- Exp: 5 to 9 years
- Expert SQL: You can read a multi thousand-line spaghetti query, understand the intent, and rewrite it into 5 clean, modular steps without losing logic.
- dbt (Data Build Tool): Proven experience setting up dbt from scratch. You know how to structure a project (Staging -> Intermediate -> Marts).
- Python & Spark: Ability to read and fix PySpark syntax errors and optimize Spark execution plans (Databricks/AWS Glue).
- AWS Ecosystem: Comfortable with S3, Athena, and IAM permissions.
- CI/CD: Experience setting up and running tests automatically on commit.
Unlock Rewards: Refer Candidates and Earn.
If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.
Experience Level Bonus Referral:
| 0-2 years | INR 5,000 |
| 2-6 years | INR 7,500 |
| 6+ years | INR 10,000 |
About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.
Equal Opportunity Employer:
VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.
Sr Data Engineer · VARITE INDIA PRIVATE LIMITED