Software Engineer (Backend & LLM)

About NewsCatcher

Turning the web into a structured database of real world events.

Tech

Large-scale web crawling and web scraping. Our tech stack: - Docker & K8s - Jenkins - Python (including asynchronous workloads) - RabbitMQ - Elasticsearch - MongoDB/Dynamo - PostgreSQL, MySQL - Gitlab - AWS, GCP, DO

The role

### **Functions:*** Design and implement new data pipelines tailored to a specific project or new product.* Continuously improve data pipelines to make them more generic and modular, reducing the effort required for subsequent integrations.* Write, optimise and refactor prompts and prompt-related pipelines.* Conduct thorough testing and validation of data pipelines to ensure accuracy and consistency of data.* Ensure pipelines are optimized for performance, scalability, and reusability to facilitate future projects.### **Examples of day-to-day tasks**1. We want to support people’s enrichment within the CatchAll Tool. Do research on what is available on the market that can help us enrich data, Double-check open databases, and verify whether there is already an existing solution available within the NewsCatcher’s code base. Plan and design the pipeline. Build a prototype and demo it to the whole team. Proceed to, test on Dev, write test cases around. Deploy on Prod, get feedback and improve.2. One of the users complained about a job being stuck. Debug the pipeline, try to understand where the problem comes from: promts, code, database. Find the issue and prepare a fix. Test it within the whole pipeline on Dev. Break some pipeline, fix and test again. Deploy on Prod, make clients happy.3. Prompt responsible for validating results is giving only 50% precision. Review the prompt, review the model. Do some prompt engineering. Realize that “with each new model I feel like LLMs become stupier”. Try another LLM provider, adapt the prompt. Test results, improve the accuracy and create a dataset to prove it. Get 70% precision, wait for compliments.### **Experience:*** 3+ years of experience in B2B SaaS as Backend / Software / Data Engineer* Strong systems thinker with attention to performance and scalability* Comfortable working with both SQL and NoSQL databases* Experience shipping LLM-based functionality into production* Able to move from prototype → stable, maintainable architecture### Must Have* Strong Python (including async workloads)* Docker & Kubernetes* RabbitMQ (or Kafka / PubSub / SNS / ActiveMQ)* MongoDB / DynamoDB / Redis* Hands-on experience with LLM frameworks### We Also Use* Elasticsearch/OpenSearch* PostgreSQL / MySQL* GitLab* AWS / GCP / DigitalOcean* Jenkins### Nice to Have* Experience building production AI agents* Frontend experience (useful for product UI)* Experience in API-first or DaaS companies### **Compensation and Perks**:* Competitive salary and equity* Up to 24 days of vacation & 16 days of sick leave/holidays (all fully paid)* One meeting-free day per week* Co-working Budget* Training Budget* We provide all the necessary equipment to work comfortably and efficiently from home.* Yearly company retreats (2025 — Portugal, 2024 — Canary Islands, 2023 — French Alpes)

Skills

• Kubernetes
• Python
• Docker