GCP Data Engineer

GCP Data EngineerOnsite – Phoenix, AZ6 months$50.00 - $52.00/Hour on W2 (Without Benefits). Applicants must be willing to work on W2Role DescriptionIn-depth knowledge of the Hadoop ecosystem, including HDFS, MapReduce, Spark, Scala, Hive. Expertise in Hive for data warehousing and querying large experience with Apache Spark for distributed data processing and real-time analytics. Advanced proficiency in Scala programming for big data application development.Familiarity with data modeling, ETL processes, and data integration techniques.Experience with cloud platforms and big data tools (e.g., Google Big Query).Strong analytical and problem-solving skills to address complex data challenges.Basic knowledge on JAVA, Spring boot. Strong SQL skill.Roles ResponsibilitiesArchitect and design scalable big data solutions leveraging technologies like Hadoop, Hive, Spark, and Scala.Develop and optimize data pipelines and workflows for processing and analyzing large-scale and fine-tune distributed computing frameworks to ensure high performance and reliability.Collaborate with cross-functional teams, including data engineers, analysts, and business stakeholders, to gather and define data requirements.Implement data security, governance, and compliance measures in big data environments.Diagnose and resolve technical issues in big data systems to maintain operational efficiency.Skills Required5+ years of hands-on software development experience with Big Data & Analytics solutionsGCP Cloud - Big Query, Airflow, DataProc, PubSub, Hadoop Hive, Spark, Python, shell scriptingStrong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.Experience of building event processing pipelines with Kafka or GCP PubSub.Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.Knowledge of distributed (multi-tiered) systems, algorithms & relational Object-Oriented Programming skills and design with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).Good knowledge and experience with configuration management tools like GitHubAbility to analyze complex data engineering problems, propose effective solutions, and implement them effectively.Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big DataLooks proactively beyond the obvious for continuous improvement opportunities.Communicate effectively with product and cross functional team.Willingness to learn new technologies and leverage them to their optimal potential.Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.Certifications in cloud platform (GCP Professional Data Engineer) is a plus.Overall Experience Required is 5 - 6 Years: Essential SkillsIn-depth knowledge of the Hadoop ecosystem, including HDFS, MapReduce, Spark, Scala, HiveExpertise in Hive for data warehousing and querying large experience with Apache Spark for distributed data processing and real-time analytics.Advanced proficiency in Scala programming for big data application development.Familiarity with data modeling, ETL processes, and data integration techniques.Experience with cloud platforms and big data tools (e.g., Google Big Query).Strong analytical and problem-solving skills to address complex data challenges.Basic knowledge on JAVA, Spring bootStrong SQL skill.4 to 7 years in Bigdata and Hadoop Ecosystems