Data Engineer - Warsaw

About the Role

We are looking for aData Engineer to design and maintain scalable data solutions that power advanced analytics and AI-driven insights. This role combines expertise inbig data engineering andweb scraping, enabling you to work on high-impact projects involving large, complex, and unstructured datasets.

You will architect data pipelines, enforce governance standards, and build tools for extracting and processing data from diverse sources, including websites and external vendors. If you thrive in solving complex data challenges and want to work with cutting-edge technologies, this is the role for you.

What You’ll Do

• Architect, develop, and maintainhigh-throughput data pipelines inDatabricks andAWS (Glue, EMR, Fargate, Step Functions).
• Ingest, normalize, and enrich large volumes of structured and unstructured data, including internal data, market data, vendor feeds, and alternative sources.
• Collaborate withAI Engineers,ML scientists, and software teams to translate requirements into scalable data architectures, schemas, and APIs.
• Optimize pipeline performance and cost using distributed processing techniques (Spark, Delta, Arrow) and AWS best practices (spot fleets, autoscaling).
• Enforcedata governance, privacy, and lineage standards, cataloguing assets in Unity Catalog and managing PII/PCI classification.
• Build automatedvalidation, testing, and monitoring frameworks to ensure data quality and freshness for both offline and online workloads.
• Support onboarding and integration of new external data vendors, ensuring compliance and rapid time-to-value.
• Continuously evaluate emergingGenAI tooling (vector stores, LLMOps platforms, synthetic-data generators) and drive proof-of-concepts.
• Web Scraping Focus:
  • Own the creation of tools and workflows for web crawling and scraping using compliance-approved technologies.
  • Test and validate scraped data for accuracy, quality, and compliance.
  • Identify and resolve issues with scrapes and scale processes as needed.

What’s Required

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering or a related role.
• Strong experience inPython andSQL.
• Experience withSpark orScala and distributed data processing.
• Proficiency in building scalable, distributed data pipelines in acloud environment.
• Familiarity withLinux/UNIX, HTTP, HTML, JavaScript, and networking concepts.
• Knowledge of web scraping tools and libraries (e.g.,Requests, BeautifulSoup, Scrapy, Pandas, Selenium, Spark).
• Working knowledge of version control systems and open-source practices.
• Solid understanding ofdata architecture principles, data modeling, and data warehousing.
• Excellent analytical and problem-solving skills.
• Strong communication skills in English (written and spoken).
• Commitment to the highest ethical standards.

Preferred Skills

• Experience extracting text from PDFs, images, and applications.
• Familiarity with system monitoring/administration tools.
• Knowledge of graph databases.
• Prior experience analysing big data sets.

Vantage Point Global is fully committed to being an Equal Opportunities, inclusive employer.We are passionate about attracting diverse talent, and welcome applications regardless of ethnicity, culture, age, gender, nationality, religion, disability, or sexual orientation.

Things you need to know:

• To apply, you’ll need to provide us with a CV and answer a few initial questions.

• We’d like to make you aware that if you have not heard back from us within three weeks of the date of application that we will not be progressing your application.