Hadoop Developer/Tester

Location: Tempe, AZ Duration: Long term Job Description : Experience in validating the different source file formats to standard messaging format (ex. JSON). Understanding of Kafka streaming process and spark framework. Experience in Validating Hive database tables using mapping/transformation logic with source files (ex. Avro, parquet, etc..) or staging tables. Experience in Validating HBase database tables or similar column-oriented storage/NoSQL databases. Experience in writing Python/Scala or other scripts to automate test execution. Tokenization or Data masking knowledge and testing the same. Test data setup/mockup knowledge in file systems and database tables. Knowledge in analyzing the log files, error files for any data ingestion failures. Knowledge or hands-on experience in Nifi, Kafka, Atlas, Schema Registry, Ranger, Oozie, GitLab, CI/CD, PyCharm, HDFS Programming language knowledge in Python/Scala/ JAVA, HiveQL/ NoSQL/Spark SQL.