JOBSEARCHER

Senior Big Data Engineer

Sr. Big Data Engineer This role will be responsible for transforming extensive and complex data into consumable business capabilities. Create system architecture, design, and specification using in-depth engineering skills and knowledge to solve complex development problems and achieve engineering goals. Determine and source appropriate data for a given analysis. Work with data modelers/analysts to understand the business problems they are trying to solve, then create or augment data assets to feed their analysis. Acts as a resource and mentor for colleagues with less experience.
Core responsibilities include:
  • Leading a team of data engineers to develop data products and tools, explore new technologies, and continually improve tools and processes
  • Hands-on experience working with Data Analysis and Architecture, Spark, PySpark, Python, Sqoop, Pig, Hive, No SQL Data Stores, Object Store, Design and Development of APIs, Kafka
  • Creating business value by leading the design, writing code, and deploying big data and machine learning capabilities
  • Possessing expert knowledge in performance, large-scale data distributed system scalability, system architecture, and data engineering best practices
  • Prioritizing operational excellence, evaluating system performance, security, designing system metrics, and driving quality improvements
  • Providing leadership, working collaboratively, and being a mentor in a fantastic team
  • Ability to produce both detailed technical work and high-level architectural designs
Requirements include:
  • 7+ years of recent hands-on in an object-oriented language (Java, Scala, Python)5+ years of experience designing and building data pipelines and data-intensive applications
  • Experience using Big Data frameworks (e.g., Hadoop, Spark), databases for complex data assembly and transformation
  • Experience working with Healthcare data is a plus