DL
Software Engineer-
Diverse Lynx India
Location not stated
Hybrid
Senior
2 months ago
- AWS Cloud
- Machine Learning
- RAG
- Python
- SQL
- AWS
- ETL
- ELT
- Airflow
- Step Functions
- AI
- AWS Glue
- AWS Lambda
- EMR
- Redshift
- Kinesis
- AWS IAM
- Apache Airflow
- Vector Search
- OpenSearch
- Pinecone
- FAISS
- Chroma
- LangChain
- LlamaIndex
- Amazon Bedrock
- OpenAI
- Docker
- Kubernetes
- Terraform
- CloudFormation
- CI/CD
- MLOps
2 months ago
This role requires strong hands-on experience in Python, SQL, distributed data systems, AWS services, and enterprise-grade GenAI architectures with governed data access.
Key Responsibilities
- Design, build, and maintain scalable enterprise data platforms and data pipelines.
- Develop robust ETL/ELT pipelines for structured and unstructured data.
- Build secure, production-grade Retrieval-Augmented Generation (RAG) applications.
- Design document ingestion, chunking, embedding, indexing, and retrieval pipelines.
- Implement governed data access, metadata management, and security controls for GenAI applications.
- Develop scalable data orchestration workflows using Airflow or AWS Step Functions.
- Optimize data models and distributed processing frameworks for performance and scalability.
- Collaborate with AI, Data Science, Platform Engineering, Security, and Business teams to deliver enterprise AI solutions.
- Ensure compliance with security, governance, and regulatory requirements.
Required Skills
Core Technical Skills
- 10–15 years of Data Engineering / Platform Engineering experience.
- 3+ years of hands-on experience with GenAI, LLMs, and Retrieval-Augmented Generation (RAG).
- Strong programming skills in Python and SQL.
- Expertise in data modelling, ETL/ELT, and distributed data systems.
- Experience designing scalable enterprise data architectures.
AWS Expertise
Hands-on experience with:
- Amazon S3
- AWS Glue
- AWS Lambda
- Amazon EMR
- Amazon Redshift
- Amazon Kinesis
- AWS IAM
Data Orchestration
Experience with one or more:
- Apache Airflow
- AWS Step Functions
GenAI / RAG
Hands-on experience with:
- Enterprise RAG architecture
- LLM application development
- Embedding models
- Prompt engineering
- Vector search and semantic retrieval
- Document processing and indexing
- Secure and governed enterprise data access
Vector Databases
Experience with one or more:
- OpenSearch Vector Engine
- Pinecone
- FAISS
- Chroma
Preferred Skills
- LangChain
- LlamaIndex
- Amazon Bedrock
- OpenAI APIs
- Docker
- Kubernetes
- Terraform or CloudFormation
- CI/CD pipelines
- Data governance and security best practices
Domain Experience
Candidates with experience in Banking, Financial Services, Insurance, or other highly regulated industries will be preferred.
Nice to Have
- Knowledge of AI governance and Responsible AI principles.
- Experience with enterprise-scale data platforms.
- Exposure to MLOps and LLMOps practices.
- Strong stakeholder communication and solution design skills.
Software Engineer- · Diverse Lynx India