Data Engineer

Data engineers play a vital role in supporting ingestion and transformation pipelines that manage data for both analytical and operational purposes across various business areas and enterprise data domains. As a key member of support teams, the data engineer focuses on ensuring production stability for data processing workflows utilized by analytics teams and data scientists engaged in predictive analytics, machine learning, and data mining.Duties:Provide expertise in data engineering to support ingestion and transformation pipelines, addressing a wide range of business needs and enterprise data domains.Ensure stability and reliability of production data processing workflows used by analytics teams and data scientists for predictive analytics, machine learning, and data mining.Define, integrate, govern, store, describe, model, and maintain enterprise data for accuracy and usage, while preserving the current state.Uphold best practices in data architecture, including accountability, governance, and requirements.Perform additional duties as assigned.Qualifications:Minimum 5 years of software engineering experience with a Bachelor’s degree in Computer Science or a related field.Experience with a broad range of modern data science and analytics tools (e.g., SQL, Hadoop, Spark, Python, R).3+ years of experience in large-scale Big Data development and deployment automation in private/public cloud environments, preferably on AWS.Hands-on experience with big data environments such as Cloudera or Hortonworks.Experience with DevOps, Continuous Integration, and Continuous Delivery (e.g., Maven, Jenkins, Stash, Ansible, Docker).Proficiency in programming languages such as Scala, Spark, Python, JavaScript, and Java, along with Unix shell scripting skills.Expertise in defining, integrating, governing, storing, describing, modeling, and maintaining enterprise data for accuracy and usage.Familiarity with policies and procedures enforced by data governance committees to ensure adherence to best practices in data architecture.Experience working with XML.Experience building data ingestion pipelines on cloud platforms using tools like Glue.Understanding of the principles, best practices, and trade-offs of schema design for both relational and NoSQL database systems.Strong understanding of Big Data NoSQL databases/technologies (e.g., MarkLogic, HBase, Hive, Spark, MongoDB).Excellent written and verbal communication skills.Willingness to travel as required.Desired (but not required):Knowledge and experience in chemistry, drug discovery/development, or the medical industry.