DL
AI Engineer
Diverse Lynx India
Location not stated
1 month ago
- AI
- Machine Learning
- EKS
- AWS SageMaker
- Kubernetes
- AWS
- API Gateway
- RAG
- Vector Search
- Azure OpenAI
- AWS Bedrock
- Snowflake
- Cortex
- LangChain
- LangGraph
- LlamaIndex
- Hugging Face
- FAISS
- OpenSearch
- Pinecone
- Chroma
- PyTorch
- TensorFlow
- scikit-learn
- MLOps
- CI/CD
- Python
1 month ago
About the Role
Hands-onArchitect to design and scale enterprise insight platforms powered by modern data, AI, andGenerative AI ecosystems. This role focuses on enabling self-service analytics and intelligent decision-making using platforms such asDataiku (on EKS), Posit, and AWS SageMaker, while embeddingGenAI-driven insight capabilities across the data lifecycle.
You will bridge data engineering, data science, and business domains to deliver scalable, governed, and AI-powered insight solutions.
Experience – should be 5+ years below 10
Key Responsibilities
Platform & Architecture
- Architect and designAI &ML ecosystems leveragingDataiku (on Kubernetes / EKS), Posit, and SageMaker.
- Definecloud-native architectures on AWS integrating EKS, S3, Lambda, API Gateway, and SageMaker for analytics and AI workloads.
- Designdata-to-insight pipelines including ingestion, transformation, feature engineering, and consumption layers.
GenAI Enablement & Patterns
- Design and implementGenAI solution patterns, including:
- RAG (Retrieval-Augmented Generation) pipelines (chunking, embeddings, vector search, retrieval optimization)
- Agentic workflows (tool/function calling, multi-step reasoning, memory)
- Prompt engineering frameworks (template management, evaluation, guardrails)
- Semantic search and knowledge augmentation systems
- Integrate LLMs usingAzure OpenAI,AWS Bedrock, Snowflake Cortex
- Designenterprise-grade GenAI guardrails, including:
- Hallucination mitigation
- Response grounding and traceability
- Safety, bias, and compliance controls
- Establishevaluation frameworks for GenAI systems, including:
- Retrieval quality
- Response accuracy and relevance
- Latency and cost optimization
Tools, Libraries & Frameworks
- Enable and standardize usage of:
- LangChain, LangGraph, LlamaIndex for orchestration and agent design
- Hugging Face Transformers for model integration and experimentation
- Vector databases (FAISS, OpenSearch, Pinecone, Chroma)
- ML/AI frameworks (PyTorch, TensorFlow, scikit-learn)
- Integrate GenAI workflows into:
- Dataiku pipelines (recipes, plugins, scenarios)
- Posit notebooks and analytical workflows
- SageMaker pipelines and endpoints
Self-Service & Insight Delivery
- Enableself-service analytics and AI workflows through governed datasets, reusable features, and standardized templates.
- Designsemantic layers and data models aligned with business domains for consistent insight consumption.
- Deliver insights viadashboards, APIs, notebooks, and GenAI-powered conversational interfaces.
Governance, MLOps & Observability
- Define and implementDataOps/MLOps practices:
- CI/CD pipelines for data and ML workflows
- Model versioning, monitoring, and reproducibility
- Experiment tracking and lineage
- Establishgovernance frameworks across:
- Data quality and lineage
- Model performance and drift
- GenAI safety and compliance
- Monitor and optimizeplatform performance, cost, and usage adoption.
Collaboration & Enablement
- Collaborate with business, engineering, and data science teams totranslate requirements into scalable insight and AI solutions.
- Build reusableaccelerators, frameworks, and best practices for analytics and GenAI adoption.
- Mentor teams and drivedesign-led, self-service-first culture in insight delivery.
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or related field
- Strong experience indata/AI architecture, insight enablement, or analytics platforms
- Hands-on expertise with:
- Dataiku (including EKS deployment)
- Posit (RStudio/Workbench)
- AWS SageMaker (training, deployment, pipelines)
- Strong understanding of:
- Modern data architectures (Datamesh, lakehouse, semantic layers, data products)
- GenAI patterns (RAG, agents, prompt engineering)
- MLOps/DataOps practices
- Proficiency inPython and/or R
- Experience withvector databases and LLM integration frameworks
Preferred Qualifications
- Experience withLangChain, LangGraph, LlamaIndex, Hugging Face ecosystem
- Knowledge ofAWS Bedrock and enterprise LLM deployment patterns
- Familiarity withKubernetes/EKS and containerized analytics platforms
- Experience withSnowflake or similar cloud data warehouses
- Exposure toindustry domains (Insurance, Finance, Healthcare)
Key Traits
- Strongarchitecture and design mindset with focus on scalability and usability
- Passion forGenAI innovation and real-world application of LLMs
- Ability to work acrossstrategy, design, and hands-on implementation
- Advocate forself-service, reusable, and governed analytics ecosystems
AI Engineer · Diverse Lynx India