Azure Data Engineer w/ Python/C#

Azure Data Engineer W/ Python/C#Location: 100% Remote

Interview Mode: Video Need 12-13+ years candidate PySpark or ScalaSpark SQLDelta lakes * Python/C#PRIMARY DUTIES AND ACCOUNTABILITIESCreate and maintain optimal data pipeline architecture 20%Assemble large, complex data sets that meet functional / non-functional business requirements. 20%Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. 20%Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Big Data technologies. 20%Deliver automation & lean processes to ensure high quality throughput & performance of the entire data & analytics platform. 10%Work with data and analytics experts to strive for greater functionality in our analytics platforms. 10%POSITION SPECIFICATIONSMinimum: Experience in building/operating/maintaining fault tolerant and scalable data processing integrations using AzureStrong problem-solving skills with emphasis on optimization data pipelinesExcellent written and verbal communication skills for coordinating across teams A drive to learn and master new technologies and techniquesExperienced in DevOps and AgileExperience using Docker or Kubernetes is a plusDemonstrated capabilities with cloud infrastructures and multi-cloud environments such as Azure, AWS, IBM cloud environments and using CI/CD pipelines. Experience with Delta LakesExperienced using Databricks & Apache SparkExperienced using Azure Data Factory or Synapse Analytics