Azure Data Engineer

Job Title: Azure Data EngineerSkills: Azure, Python, Java, Scala, PySpark, SQL, Hive, Pig, Hadoop, Cloud, AWS, Databricks, Datalake, and Big data will always be there on-prem.Experience: 7+ yearsJob Location: Dallas, TXFTE/ OnsiteWe at Coforge are hiring Azure Data Engineer with the following skillset:Over 7-10 years of experience as a Data EngineerHe has hands on experience with Python, Java, Scala, PySpark, SQL, Hive, Pig, Hadoop Ecosystem on both on-premises and cloud (AWS) Infrastructure.In-depth understanding of Spark Architecture including Spark Core, PySpark, Spark SQL, Data Frames, Spark StreamingWorked on Dimensional Data modelling in Star and Snowflake schemas and Slowly Changing Dimensions (SCD).Experience on Azure Blob Storage and Cosmos Database for data transformation in Azure DatabricksExperience in performing ETL on top of streaming data from IOT Machines using Spark Streaming.Worked in ETL, Provided technical consulting and guidance to development teams for the design and development of highly complex or critical ETL architecture.Experience on building the reporting subscription application in AWS cloud ecosystem using Amazon S3, EMR, Glue, Dynamo DB, Lambda and Redshift DatabaseHands on experience in importing and exporting data from databases like MySQL, MongoDB, Oracle, Teradata into HDFS and vice-versa using Sqoop.Experience in performing analytics on structured data in Hive with Hive queries, Views, Partitioning, and Bucketing using HiveQLExperience in writing the SQL and PL/SQL for creating the views and table for reporting and analytics.Experience in usage of Hadoop distribution like Cloudera5.3 (CDH5, CDH3) and Hortonworks distribution.