Data Engineer SQL/Python/automation

Analytics EngineeringBuild data pipelines and partner with data scientists on ML/MLOps. Personalization for retail pharmacy (text/calls/SMS/emails) using business rules, AB testing, etc.Databricks Spark Python NoSQL DBs Kubernetes SQLTranslating business requirements into clinical rules enginesFramework Development/Design à CapabilitiesTransitioning from batch to real-time à batch to real time via microservices & Kafka.Basic scripting commands – someone on the infra side will do thisNo reporting for this teamSnowflake à nice to have (moving source data)Snowpark as a replacement of DatabricksKafka basics – just need to know basic services i.e. producer, consumer, streams etc.You will collaborate with business partners to identify opportunities to leverage big data technologies in support of pharmacy personalization with a common set of tools and infrastructure to make analytics faster, more insightful, and more efficient. You will design highly saleable and extensible batch and real-time big data and cloud platforms which enables collection, storage, modeling, and analysis of massive data sets from numerous channels. You will define and maintain data architecture, focusing on applying technology to enable business solutions. You will assess and provide recommendations on business relevance, with appropriate timing and deployment. You will perform architecture design, data modeling, and implement CVS big data platforms and analytic applications. You will bring a DevOps mindset to enable big data and batch/real-time analytical solutions that leverage emerging technologies. You will develop prototypes and proof of concepts for the selected solutions, and implement complex big data projects. You will apply a creative mindset to a focus on collecting, parsing, managing, and automating data feedback loops in support of business innovation.Required skills: • Strong in SQL and Python, with 3+ years hands-on coding experience with both • Experience building automated big data pipelines • Experience performing data analysis and data exploration • Experience working in an agile delivery environment • Strong critical thinking, communication, and problem solving skills • Experience with big data frameworks (i.e. Hadoop and Spark) • Experience with cloud-based platforms (i.e. Azure, GPC, AWS) • Experience with Snowflake and hands-on query tuning/optimization. • Experience working in multi-developer environment, using version control (i.e. Git) • Experience with orchestrating pipelines using tools (i.e. Airflow, Azure Data Factory) • Experience with real-time and streaming technology (i.e. Azure Event Hubs, Azure Functions Kafka, Spark Streaming) • Experience with REST API/Microservice development using Python • Experience with deployment/scaling of apps on containerized environment (i.e. Kubernetes, AKS) • Experience with technical solutioning and system architecture design • Experience partnering cross-functionally with other technical teams (i.e.