Lead Software Engineer - Performance

Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!

Description: 

• We are seeking a talented LeadSoftwareEngineer –Performanceto deliver roadmap features of EnterpriseTruRiskPlatform which would help customers to Measure, Communicate and Eliminate Cyber Risks.  

• You will lead the performance engineering efforts across Spark, Kafka, Elasticsearch, and Middleware APIs, ensuring that our real-time data pipelines and services meet enterprise-grade SLAs. 

• As part of our high-performing engineering team, you will design and execute performance testing strategies,identify system bottlenecks, and work with development teams to implement performance improvements that support billions ofcyber security events processinga day across our data platform. 

Responsibilities: 

• Own the performance strategy across distributed systemswhichincludes Hadoop, Spark, Kafka, Elasticsearch/OpenSearch,Big Data Componentsand APIs for each release. 

• Define, develop, and execute performance test plans, load tests, stress tests, and soak tests. 

• Create realistic performance test scenarios for data pipelines and microservices based on production-like workloads. 

• Proactivelyidentify bottlenecks, resource contention, and latency issues using tools such as JMeter, Spark UI, Kafka Manager, Elastic Monitoring and App Dynamics. 

• Provide deep-dive analysis and recommendations on tuning and scaling Spark jobs, Kafka topics/partitions, ES queries, and API endpoints. 

• Collaborate with developers, architects, and infrastructure teams to integrate performance feedback into design and implementation. 

• Simulate and benchmark real-time and batch dataflow at scale using synthetic and production-like datasets and own this framework end to end for synthetic data generator. 

• Lead the initiative to build a performance testing framework that integrates with CI/CD pipelines. 

• Establish and track SLAs for throughput, latency, CPU/memoryutilization and Garbage collection. 

• Create performance dashboards and visualization using Prometheus/Grafana, Kibana, or equivalent. 

• Document performance test findings and create technical reports for leadership and engineering teams. 

• Recommend performance optimization to Dev and Platform groups. 

• Responsible foroptimizing theoverall cost. 

• Contribute to feature development and fixes apart from performance benchmarking. 

Qualifications: 

• Bachelor's degree in computer science, Engineering, or related field. 

• 8+ years of overall experience in distributed systems and backend performance engineering. 

• 4+ years of JAVA development experience with Microservices architecture.   

• Proficient inscripting (Python, Bash) for automation and test data generation. 

• 4+ years of hands-on experience withApache Spark – performance tuning, memory management, and DAG optimization. 

• 3+ years of experience withKafka – topic optimization, producer/consumer tuning, and lag monitoring. 

• 3+ years of experience withElasticsearch/OpenSearch – query profiling, indexing strategies, and cluster optimization. 

• 3+ years of experience withperformance testing tools such as JMeter or similar. 

• Excellent programming and designing skills and Hands-on experience on Spring, Hibernate. 

• Deep understanding ofmiddleware and microservices performance including REST APIs. 

• Strong knowledge ofprofiling, debugging, and observability tools (e.g., Spark UI, Athena, Grafana, ELK). 

• Experience designing and running benchmarks at scale for high-throughput environments in PBs. 

• Experience with containerized workloads and performance testing inKubernetes/Docker environments. 

• Solid understanding ofcloud-native architecture (OCI) and distributed systems design. 

• Strong knowledge of Linux operating systems and performance related improvements. 

• Familiarity withCI/CD integration for performance testing (e.g., Jenkins, GitHub). 

• Knowledge ofdata lake architecture, caching solutions, and message queues. 

• Strong communication skills and experience influencing cross-functional engineering teams. 

Additional Plus Competencies:

• Prior experience in anyanalytics platform on Big Data would be a huge plus.