Big Data Dev
Job Title100% Remote. 6 month contract. 2 round interview to hire.Project Scope / Components:RPS (Results Prioritization Service) – this is the last application in the data pipeline.Optum receives claims data from various clients, runs fraud analytics, and sends the client a response to accept or decline the claim to result in $$ savings for clientsThere are 2 parts to this application – One side is Java/Azure based, and the other is Spark based using ADF pipelines – this resource will work on the Spark/ADF sideHigh-Level Individual DutiesPrimary responsibility will be building the streaming apps with Spark and Scala for AzureMust-Haves (Concepts & Tools)Spark – need experience building streaming applicationsAzure – Functions, Data Lakes, Databricks, Data Factory, Event Hub, Event GridScala or Java – Scala is preferred, Java is second optionExperience building Terraform scriptsDevOps experience – Docker, KubernetesPrimary ResponsibilitiesDevelops services, controls, and reusable patterns that enable the team to deliver value safely, quickly, and sustainably in the public cloud.Develop highly-scalable applications for the Azure cloud by building microservices, Azure functions, and patterns for public cloud enabling security and privacy at scale.Define and implement technical solutions to meet business needs through Agile process.Understand new product capabilities and decompose the implementation into specific functional changes for verification.Foster high-performance, collaborative technical work resulting in high-quality output.Gather and analyze data to aid in informed decision-making while providing detailed, realistic estimates.Interact skillfully with business stakeholders and third-party technical organizations.Proactively automate infrastructure, application, and services to enable an automated delivery through the CICD pipelines to the cloud.Review code and provide feedback relative to best practices and improving performance.Mentor and guide other engineers within the team.Perform comprehensive testing of product distributions prior to release.Troubleshoot production support issues post-deployment and come up with solutions as required.Required QualificationsExperience in programming with Java/ScalaExperience with distributed processing of Big Data workloads using Spark FrameworkExperience building RESTful web servicesProficiency with Azure, Docker, Kubernetes, Linux, PowerShellExperience with DevOps, Continuous Integration and Continuous Delivery (Jenkins, Azure DevOps, Fortify, Sonar)Experience with migrating applications to the Azure cloudExperience building and deploying applications to the Azure cloud using Infrastructure as Code tools such as TerraformExperience working in an Agile environmentExpertise in performance and scalability optimizationKnowledge of software and infrastructure securityStrong communication and problem solving skillsKnowledge of Data Pipeline/workflow management tools such as Azure data factory.Knowledge of Azure Synapse and/or Snowflake Cloud Datawarehouse platformsExtensive knowledge of data architecture principles (e.g., Data Lake, Databricks Delta Lake, Data Warehousing, etc.)