Data Engineer

Required Skills & Experience
Around 5 years of professional experience in Data Engineering, Python development, or a related role.
Strong hands-on expertise in Python, particularly for:
Data wrangling and transformation
ETL/ELT development
File and data processing
Automation and scripting
Data validation and cleansing
Strong knowledge of at least one relational SQL database such as:
PostgreSQL
MySQL
Oracle
SQL Server
Strong understanding of SQL, including joins, subqueries, CTEs, window functions, aggregations, and query optimization.
Practical experience with AWS cloud services used in data engineering.
Experience developing RESTful APIs using FastAPI.
Good understanding of API concepts including HTTP methods, request/response handling, authentication, status codes, error handling, and API integration.
Experience with Git/version control and standard software development practices.
Good understanding of data engineering concepts, including data pipelines, ETL architecture, data quality, and data integration.

Good to Have
Hands-on experience with PySpark and distributed data processing.
Experience with NoSQL databases such as DynamoDB, or Cassandra.
Experience with AWS services such as S3, Lambda, Glue, Athena, Redshift, EMR, RDS, Step Functions, EventBridge, and CloudWatch.
Experience with workflow orchestration tools such as Apache Airflow or AWS Step Functions.
Experience with Docker and containerized applications.
Familiarity with CI/CD pipelines and DevOps practices.
Knowledge of data warehousing and dimensional data modeling.
Experience working with large-scale datasets and performance optimization.
Familiarity with cloud-based data lake and data warehouse architectures.

Technical Skills
Skill Area Expected Knowledge
Programming Python – Advanced
Data Engineering ETL/ELT, data wrangling, transformation, validation
Database SQL, Relational Databases
Cloud AWS
API Development FastAPI, REST APIs
Big Data PySpark – Good to Have
NoSQL MongoDB / DynamoDB / Cassandra – Good to Have
Data Formats JSON, CSV, XML
Version Control Git
DevOps CI/CD, Docker – Good to Have