Principal Data Engineer-PySpark

Job Summary
We are seeking a skilled Data Engineer with extensive experience in designing and implementing scalable, high-performance data warehousing solutions using Azure Synapse Analytics. The ideal candidate will play a pivotal role in migrating existing data systems to the Azure Cloud, optimizing data workflows, and ensuring data accuracy and security. This position requires a strong technical background combined with excellent stakeholder management and consulting skills to deliver robust data solutions aligned with business objectives.

Key Responsibilities
- Design and deploy highly scalable and efficient data warehousing solutions within Azure Synapse, ensuring optimal performance and reliability.
- Analyze current data systems and lead migration efforts to Azure Cloud, enhancing data flow and collection processes to improve overall data quality and usability.
- Participate actively in data architecture design and modeling initiatives, aligning technical solutions with business requirements and adhering to data governance policies.
- Collaborate closely with stakeholders to define data requirements, project deliverables, and timelines, providing expert technical leadership throughout the project lifecycle.
- Implement and enforce data security and privacy measures consistent with industry best practices and organizational standards.
- Optimize data integration pipelines to maximize performance and scalability, ensuring efficient data processing and transformation.
- Develop and maintain comprehensive documentation covering data architectures, operational procedures, and implemented solutions to support ongoing maintenance and knowledge sharing.

Required Qualifications
- Proven expertise in Azure Synapse Analytics, including data integration, management, and security capabilities.
- Proficiency in SQL, Python, PySpark, and other scripting languages commonly used in data engineering tasks.
- Experience with Azure Infrastructure as Code (IaC) and CI/CD automation tools such as ARM templates, Bicep, or Terraform.
- Strong understanding of ELT and ETL processes, with hands-on experience using Azure Data Factory for data orchestration.
- Solid knowledge of data modeling techniques, data warehousing concepts, and data lakes; experience or certification in Data Vault 2.0 is highly preferred.
- Demonstrated consulting or advisory experience in designing and deploying large-scale data solutions within Azure environments.
- Ability to engage directly with clients or stakeholders, translating business needs into clear technical specifications and actionable project plans.
- Proven track record of contributing to project leadership, coordinating across multiple client functional groups, and managing client relationships effectively.

Preferred Qualifications and Benefits
While not explicitly stated, candidates with certifications in Azure Synapse, Data Vault 2.0, or related cloud and data engineering technologies will be highly regarded. The role offers the opportunity to work on cutting-edge cloud data platforms, collaborate with diverse teams, and influence strategic data initiatives within a dynamic environment.

If you are passionate about leveraging Azure technologies to build scalable data solutions and enjoy working closely with business stakeholders to drive impactful data strategies, we encourage you to apply.