Lead Azure Data Engineer

Qualifications:Proven experience in a similar role, with a strong track record as Sr. Data Engineer and has helped/lead development teams in delivering high-quality data orchestration solutions with min 7+ years’ experience.Nice to haves 1.Python Unit Testing2.Unity Catalog3.Databricks Asset Bundles4.Unit Testing/MockingKey Responsibilities• Development: - Working with varied data file formats (Avro, json, csv) using PySpark for ingesting and transformation, Terraform scripting and DevOps process.• Lead Development Initiatives: Develop code with FTR and of quality. Also to take ownership of different development initiatives, ensuring timely delivery while meeting or exceeding quality standards.• Code Quality and Maintenance: Oversee the maintenance of code quality and enforce best practices, including code reviews, unit testing, CICD, and adherence to coding standards.• Reduce Technical Debt: Identify, prioritize, and implement strategies to reduce technical debt and address vulnerabilities in the codebase.• Team Leadership: Provide mentorship and guidance to team members, fostering a collaborative, innovative, and high-performance environment.• Collaboration: Work closely with cross-functional teams, including product management, Cloud Security, LOB, Network security, and DevOps, to integrate different features into broader solutions.• Problem Solving: Address and troubleshoot complex technical issues, providing solutions that enhance system performance and user experience.• Documentation: Ensure comprehensive documentation of systems, processes, and code to facilitate knowledge sharing and maintenance.• Stakeholder Communication: Communicate progress, challenges, and solutions to stakeholders, ensuring transparency and alignment with business objectives.