W2 Contract- Long term Remote Position- Sr. Data Engineer
Data EngineerRemote PositionDuration: Long TermRequired Skills:Python & SparkBigQuery - Google CloudSQLBasic Qualifications:KafkaData ProctorCI/CD, Test Automation and cloud deploymentsResponsible for:Design infrastructure that delivers needed solution performance and enables processes and/or data storageDrives development, testing, deployments, and iterative improvement of product capabilities and features in collaboration with designers, product managers, and other engineers on the product teamCollaboration through frequent pair programmingPair with Data scientist to share relevant data source, ingest or derive new data elements and triage issue in data pipelinePair with Data scientist to create workflow in scheduler, setup build pipeline and identify issues in infrastructure / environment to work to fix the issuesTest Driven DevelopmentBuilding development practices like CI/CD. Test Automation and cloud deployments.Growing your skills, tackling interesting work and complex problemsBuilding leadership skills - coach and mentor other engineersThe ideal candidate will have:Strong knowledge building development practices like CI/CD, Test Automation and cloud deploymentsKnowledge of build management tools such as Jenkins or MavenStrong Java/J2EE knowledge (spring boot preferable)Demonstrated understanding of source control systems such as GITDatabase Design experience including either SQL, PL/SQLImplementing ETL process with Big Data TechnologiesRequired: Google Dataproc/Dataflow, Kafka, Spark, Python, Scala and AirflowGitlab pipeline creationPreferred: MapReduce, Pig, Hive, Kafka, Sqoop, and FlumeExperience in designing and creating automation workflows and executionExperience working with major cloud platforms (GCP, AWS, or Azure)Experience and/or interest in Test Driven Development (TDD) and agile methodologiesGood understanding of MPP databases such as Teradata and NetezzaStrong communication skills and interest in a pair-programming environmentPassion for growing your skills, tackling interesting work and complex problemsExperience deploying to cloud environments a plusGood knowledge of Airflow and working/theoretical knowledge of Azkaban5+ years of relevant experienceThanks and Regards Poornachandra Rao Software Technology Inc. Email: poornab@stiorg.com