Data Engineer

*Due to the nature of this role, only Green Card Holders and United States Citizens will be considered.*Optomi, in partnership with a leading entertainment industry giant, is looking for a Senior Data Engineer to join their team in Glendale, CA.Position Summary: As a Senior Data Engineer, you will play a pivotal role in transforming data into actionable insights. You will collaborate with a dynamic team of technologists to develop cutting-edge data solutions that drive innovation and fuel business growth. Your responsibilities will include managing complex data structures, delivering scalable and efficient data solutions, and optimizing data-driven decision-making processes. If you're passionate about leveraging data to make a tangible impact, we welcome you to join us in shaping the future of our organization.What the right candidate will enjoy:Working onsite 2-4 days per week in Glendale, CA, with a highly collaborative and innovative team.Involvement in agile/scrum ceremonies to improve team processes.Opportunities to engage with and understand customer needs for innovative offerings.What type of experience does the right candidate have:5-7+ years of data engineering experience developing large data pipelines.Proficiency in at least one major programming language such as Python, Java, or Scala.Hands-on production experience with distributed processing systems like Spark and data pipeline orchestration systems like Airflow.What the responsibilities are of the right candidate:Contribute to maintaining, updating, and expanding existing Core Data platform data pipelinesBuild tools and services to support data discovery, lineage, governance, and privacyCollaborate with other software/data engineers and cross-functional teamsTech stack includes Airflow, Spark, Databricks, Delta Lake, Kubernetes and AWSCollaborate with product managers, architects, and other engineers to drive the success of the Core Data platformContribute to developing and documenting both internal and external standards and best practices for pipeline configurations, naming conventions, and moreEnsure high operational efficiency and quality of the Core Data platform datasets to ensure our solutions meet SLAs and project reliability and accuracy to all our stakeholders (Engineering, Data Science, Operations, and Analytics teams)Be an active participant and advocate of agile/scrum ceremonies to collaborate and improve processes for our teamEngage with and understand our customers, forming relationships that allow us to understand and prioritize both innovative new offerings and incremental platform improvementsMaintain detailed documentation of your work and changes to support data quality and data governance requirements