PG
Data Engineer (fabric)
Perfict Global, Inc.
Location not stated
2 months ago
- ETL
- ELT
- Fabric
- Azure Data Factory
- RBAC
- Key Vault
- Python
- SQL
- Azure
- CI/CD
- Azure DevOps
- GitHub
- AI
- Agile
- Microsoft Fabric
- Databricks
- Azure SQL
- Power BI
- Polars
- DuckDB
- Pandas
- NumPy
- Git
2 months ago
Location: Woburn MA. (Remote or Onsite)
Interview Process: 2 ROUNDS – 1st virtually and 2nd will be onsite
Job Details:
- Develop and maintain scalable ETL (Extract, Transform, Load) processes to efficiently extract data from diverse sources, transform it as required and load it into data warehouses or analytical systems.
- Design and optimize database architectures and data pipelines to ensure high performance, availability and security while supporting structured and unstructured data.
- Build and maintain robust ETL/ELT pipelines using Fabric Pipelines, Azure Data Factory, and Synapse.
- Implement secure data architectures using Roles-based Access Controls (RBAC), Key Vault, Private Endpoints
- Integrate data from APIs, third-party platforms, and hybrid (on-prem/cloud) systems.
- Develop data workflows using Python, SQL, and Spark, selecting appropriate frameworks based on workload characteristics.
- Drive cost-efficient, low-latency analytics through partition-aligned Delta storage, columnstore-optimized warehouse tables, DirectLake semantic access, and SCD-managed dimensional models, ensuring predicate pushdown, partition elimination, and minimal data movement across the query execution lifecycle.
- Implement secure data architectures using:
- RBAC and row/column/object-level security (RLS/CLS/OLS)
- Azure Key Vault, Private Endpoints, Managed Identities.
- Implement CI/CD pipelines for data platforms using Azure DevOps or GitHub
- Establish automated deployment, environment promotion, and testing strategies.
- Support downstream semantic models and reporting layers by delivering well-modeled, performant, and governed data structures (no report/dashboard development responsibilities).
- Partner with analytics and AI teams to deliver trusted, production-ready data products.
- Ensure platform reliability through constraint-driven data validation, fault-tolerant pipeline orchestration, and telemetry-backed observability, enabling anomaly detection, automated alerting, and lineage-driven root cause analysis across distributed data workloads.
- Lead design decisions and influence enterprise data architecture standards
- Collaborate with engineers, analysts, and business stakeholders to translate requirements into scalable solutions.
- Mentor junior engineers and contribute to engineering excellence and knowledge sharing.
- Operate effectively in Agile delivery environments.
- 8+ years of experience in data, software, or platform engineering.
- 5+ years building production data solutions on the Microsoft/Azure stack.
- Strong experience with Microsoft Fabric and at least two of:
- Azure Data Factory, Synapse, Databricks, Azure SQL, Power BI
- Advanced proficiency in SQL and Python (delta-rs, PyArrow, Polars, DuckDB, Pandas, and NumPy).
- Proven experience designing modern data architectures (lakehouse, medallion, etc.).
- Hands-on experience with CI/CD, Git workflows, and environment promotion.
- Experience implementing enterprise security, identity, and access controls.
- Strong troubleshooting, performance tuning, and root cause analysis skills.
- Experience working in regulated or high-scale environments.
- DP-700 (Fabric Data Engineer) – required or within 6 months.
- DP-600,DP-203– preferred.
Data Engineer (fabric) · Perfict Global, Inc.