Senior Python Big Data Engineer
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Senior Data DeveloperSeeking a Senior Data Developer to implement data processing and ingestion of structured and semi-structured data as a member of the Innovation in Data Engineering and Analytics (IDEA) team.Responsibilities include:Cleanse, manipulate and analyze large datasets (Semi-Structured and Unstructured data – XMLs, JSONs, CSVs, PDFs) using python and Snowflake database.Develop Python scripts to filter/cleanse/map/aggregate data.Manage and implement data processes (Data Quality reports)Develop data profiling, deduping logic, matching logic for analysisProgramming Languages experience in Python, PySpark and SQL for data ingestionPresent ideas and recommendations on data handling and data parsing technologies to managementQualifications:5+ years of experience in processing large volumes and variety of data (Structured and semi-structured data, writing code for parallel processing, shredding XMLS, JSONs and reading PDFs) - Mandatory3+ years of programming experience in Python for data processing and analysis – Mandatory2+ years of experience with Snowflake, preferable parsing JSON and XML files- DesirableStrong SQL experience is a must - Mandatory3+ years of experience – using Hadoop platform and performing analysis. Familiarity with Hadoop cluster environment and configurations for resource management for analysis work - Optional2+ years of programming experience in PySpark for data processing and analysis - OptionalDetail oriented.