AI Software Engineer

Vichara is a Financial Services focused products and services firm headquartered in NY and building systems for some of the largest i-banks and hedge funds in the world.

Key Responsibilities

🔹Architecture & System Design

• Architect, design, and leadmulti-agent LLM systems usingLangGraph, LangChain, and Promptfoo for prompt lifecycle management and benchmarking.

• BuildRetrieval-Augmented Generation (RAG) pipelines leveraginghybrid vector search (dense + keyword) usingLanceDB, Pinecone, or Elasticsearch.

• Define system workflows for summarization, query routing, retrieval, and response generation, ensuring minimal latency and high precision.

• DevelopRAG evaluation frameworks combining retrieval precision/recall, hallucination detection, and latency metrics — aligned with analyst and business use cases.

🔹AI Model Integration & Fine-Tuning

• IntegrateGPT-4o, PaLM 2, and open-weight models (LLaMA, Mistral) for task-specific contextual Q&A.

• Fine-tune transformer models (BERT, SentenceTransformers) for document classification, summarization, and sentiment analysis.

• Manage prompt routing and variant testing usingPromptfoo or equivalent tools.

🔹Agentic AI & Orchestration

• Implementmulti-agent architectures with modular flows — enabling task-specific agents for summarization, retrieval, classification, and reasoning.

• Designfallback and recovery behaviors to ensure robustness in production.

• EmployLangGraph for parallel and stateful agent orchestration, error recovery, and deterministic flow control.

🔹Data Engineering & RAG Infrastructure

• Architect ingestion pipelines for structured and unstructured data — including financial statements, filings, and PDF documents.

• LeverageMongoDB for metadata storage andRedis Streams for async task execution and caching.

• Implement vector-based search and retrieval layers for high-throughput and low-latency AI systems.

🔹Observability & Production Deployment

• Deploy end-to-end AI systems onAWS EKS / Azure Kubernetes Service, integrated withCI/CD pipelines (Azure DevOps).

• Build comprehensivemonitoring dashboards usingOpenTelemetry andSignoz, tracking latency, retrieval precision, and application health.

• Enforce testing and regression validation using golden datasets and structured assertion checks for all LLM responses.

🔹Cross-functional Collaboration

• Collaborate with DevOps, MLOps, and application development teams to integrate AI APIs withReact / FastAPI-based user interfaces.

• Work with business analysts to translate credit, compliance, and customer-support requirements into actionable AI agent workflows.

• Mentor a small team of GenAI developers and data engineers in RAG, embeddings, and orchestration techniques.

• Experience:
  • 5+ years as an AI or ML Engineer

• Required Skills & Experience

• LLMs & GenAI: GPT-4o, PaLM 2, LangGraph, LangChain, Promptfoo, SentenceTransformers

• RAG Frameworks: LanceDB, Pinecone, ElasticSearch, FAISS, MongoDB

• Agentic AI: LangGraph multi-agent orchestration, routing logic, task decomposition

• Fine-Tuning: BERT / domain-specific transformer tuning, evaluation framework design

• Infra & MLOps: FastAPI, Docker, Kubernetes (EKS/AKS), Redis Streams, Azure DevOps CI/CD

• Monitoring: OpenTelemetry, Signoz, Prometheus

• Languages & Tools: Python, SQL, REST APIs, Git, Pandas, NumPy

• 🧠Nice-to-Have Skills

• Knowledge ofReranker-based retrieval (MiniLM / CrossEncoder)

• Familiarity withPrompt evaluation and scoring (BLEU, ROUGE, Faithfulness)

• Domain exposure toCredit Risk, Banking, and Investment Analytics

• Experience withRAG benchmark automation andmodel evaluation dashboards