JOBSEARCHER

Data Engineer (Python)

Seasoned Data engineer need to migrate data pipelines to modernize pipelines the enterprise team has implemented - Once the process is run and produced, it has to close the enterprise pipeline , Already an established pattern that needs to be followedInvolve dealing with a lot of data, close to 5-7 TB of large data , flowing through data streamClose to 6 datasets that they are working with - They use snowflake as the warehouseDay to day - working with registration of data and every data set has to be registered correctlyExchange team that the contractors will work onAWS - Must have worked with EMR specificallyPython and framework like spark is helpfulDebugging skillsSQL is nice to have