Upvote
Downvote
Big Data Engineer
Share Job
- Suggest Revision
- Working as part of the Big Data Engineering team which is responsible for transforming data into useful information for the data science team and product team.
- Working with Linux systems and Hadoop databases to extract data from Hadoop database and ingested using Scoop.
- Staging the real time data from gateway into AWS S3 or Azure Blob storage.
- Implementing Spark using Scala and utilizing Data frames and Spark SQL API for faster processing of data.
- Responsible to Use partitions in Spark session to improve the performance of the load time.
Active Job
Updated 2 days agoSimilar Job
Relevance
Active