Lead Data Engineer
Interview : VideoVisa : USC, GC, GC EAD, H4, L2This is onsite from day-1Description :Client needs a forward-thinking Lead Data Engineer willing to work on-site) in New York NY. This opportunity is in the Asset Management/ Financial Services / Investment Management / Hedge Funds industry and requires a proven track record of flawless execution of the below skills:ETL/ELT process and frameworks.Expertise in SQL, Oracle, MySql, PostgreSQLProficient in Python and Scala programmingStrong in Git repo and CI/CD deployment processExperience with Big Data - Hadoop, Spark.Job Description:Job Duties: • At least 8+ years of relevant experience in design, development, complete end-end design of enterprise-wide big data solution.• Strong Application development experience in Scala/Python.• Experience in designing & developing a big data solution using Spark, Scala, AWS Glue, Lambda, SNS/SQS, Cloudwatch• Strong Database SQL experience, preferably Redshift.• Strong in Git repo and CI/CD deployment process.• Experience in Snowflake .• Experience with ETL/ELT process and frameworks is a must.• Create integration and application technical design documentation.• Conduct peer-reviews of functional design documentation.• Complete development, configuration, test cases and unit testing• Perform code reviews and ensure standards are applied to each solution component.• Resolve complex defects during testing phases and identify root causes.• Drive performance testing.• Production Support: Assist in troubleshooting production environment and tuning environments.• Ensure best practices are followed from a technical perspective during all phases of the project.Domain Experience :• Asset Management• Financial Services• Investment Management• Hedge FundsMinimum Skills Required: Experience with ETL/ELT process and frameworks is a mustExpertise in SQL and experience with relational databases like Oracle, MySql, PostgreSQLProficient in Python and Scala programming for data engineering tasks, automationsStrong in Git repo and CI/CD deployment processExperience with Big Data - Hadoop, SparkDomain Experience :• Asset Management• Financial Services• Investment Management• Hedge Funds