As a Senior Backend Software Engineer on our LLM Services team, you’ll be at the forefront of integrating AI into healthcare delivery. Your tasks will include designing and implementing Python-based services that leverage Large Language Models to enhance our core healthcare platform. Working closely with our distributed engineering and product teams, you'll:

Architect and build scalable micro-services that power our LLM-enabled features
Collaborate with other Software, ML and Infrastructure engineers to optimize LLM integration and deployment
Design and implement LLM orchestration services, including prompt management, model switching, and response streaming
Build robust evaluation pipelines to measure and improve LLM output quality and consistency
Develop scalable APIs for LLM-powered features, including context injection and retrieval augmentation
Implement efficient caching and optimization strategies for LLM inference
Create monitoring systems for tracking token usage, latency, and other LLM-specific metrics
Work on prompt engineering and chain-of-thought implementations
Ensure compliance with healthcare regulations while working with LLM outputs
Participate in technical design discussions and code reviews
Monitor and maintain services in production, including LLM Observability using tools like LangFuse

Requirements

Strong proficiency in Python and experience building production-grade backend services
Understanding of prompt engineering principles and LLM evaluation metrics
Experience with vector databases (like Pinecone, Weaviate, or similar) for semantic search
Familiarity with streaming architectures for real-time LLM responses
Experience implementing rate limiting and failover strategies for API services
Strong grasp of software engineering best practices, including testing, documentation, and version control
Excellent problem-solving skills and attention to detail
Strong written and verbal communication skills

You'd be a great fit if you have

Experience fine-tuning or implementing RAG (Retrieval Augmented Generation) systems

Experience with LLM frameworks such as LangGraph, LangChain, LlamaIndex, or similar orchestration tools
Familiarity with different LLM providers (OpenAI, Anthropic, etc.) and their APIs
Familiarity with the use of Open LLMs, either self-hosted or on through a third party provider, e.g. Amazon Bedrock
Knowledge of LLM output validation and safety measures
Experience with embeddings and semantic search implementations
Background in prompt engineering or LLM evaluation metrics
DevOps experience, particularly with containerised service deployment, GPU-enabled compute environments and scaling
Knowledge of healthcare data privacy regulations and security best practices
Experience working in distributed teams across multiple time zones

Location & Work Style

All-remote position
Flexible working hours
Occasional late meetings to collaborate with US-based team members

HealthbookPlus remote jobs

Lead LLM Engineer

Senior Backend Software Engineer (Python, LLM)

Lead LLM Engineer