Data Engineer x 2
Data Engineer - 100% Remote - Contract thru EOYInterviews Set: 5/18, 5/19, 5/20
Interview Process:
30 minute phone screen1 hour technical interview
Top 3 Requirements:
- Ability to work independently and figure things out on their ownSQL – Senior level
- Apache Spark (with Scala) – Could look at Pyspark, but need to be senior level in either one
- Snowflake – Just need knowledge, don't need to be an expert
Nice To Have: Snowflake Rally/Agile experience
Team: 2 Data Engineers, 2 BA's, 1 Architect
Project:
Transformation of Data / Ingestion. Taking old system that is decommissioned, and moving data to a new system (new schema, new design, etc.). Will ingest data, transform it, and make it match the reports. 3 options that they are considering: Flat file (from new system), direct connect to new system, or Kafka streaming.
Responsibilities:
- Research, proof of concepts, architecture, design, development, test, deployment and maintenance
- Does not have a lot of documentation for a path forward, so there will be a lot of discovery work
- Walk back data and figure it (may not get requirements)
- Writing logic for data ingestion and ETLRally for Agile – Know how to do user stories