- UpvoteDownvoteShare Job
- Suggest Revision
Experience with Event driven Big Data streaming infrastructure and ETL/ELT frameworks (e.g., Spark Streaming, Flink, Kafka, Hive, Hadoop, Airflow, etc.) Utilize programming languages like Python, SQL, and NoSQL databases, Container Orchestration services including Terraform, Docker and Kubernetes, and a variety of Azure tools and services to build an Event Driven Big Data Streaming platform for an ELT data pipeline.
RemoteExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Kafka, HDFS, Hive, Cloud computing, machine learning, LLMs, NLP & Web development experience is a plus. Experience and expertise in Databricks, Spark, Hadoop etc. 10+ years experience in data engineering with data warehouse technologies.
ExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Optimize jobs to utilize Kafka, Hadoop, Presto, Spark, and Kubernetes resources in the most efficient way. Create different consumers for data in Kafka using Spark Streaming for near time aggregation.
RemoteExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Knowledge of big data technologies such as Apache Spark, Hadoop, or Kafka. Design and implement scalable data pipelines using AWS services such as Glue, Redshift, S3, Lambda, EMR, Athena.
ExpandApply NowActive JobUpdated 2 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Experience with Big Data technologies such as AWS, Hadoop, Spark, Pig, Hive, Pig, Hive, Presto, HBase, etc. Coursework or thesis in machine learning, data mining, information retrieval, statistics or natural language processing.
$151,300 - $261,500 a yearFull-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Experience with cloud platforms (e.g., AWS, Azure, GCP) and big data technologies (e.g., Spark, Hadoop) is a plus. Your expertise in Airflow, DBT, and Databricks will be essential in optimizing our data workflows and enabling advanced analytics and insights.
$180,000 - $210,000 a yearFull-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Bonus if those were open-sourced big data processing frameworks using technologies like Spark, Airflow, Kafka, Flink, Iceberg, Deltalake. Our team leverages cutting-edge open-source technologies such as Apache Spark, Flink, Kafka, Airflow, Delta Lake, and Iceberg to build and maintain a scalable, high-quality data ingestion framework.
Full-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Prior technical experience in Big Data solutions - you've built meaningful pieces of data infrastructure. Cutting-Edge Technology: Work with advanced open-source technologies such as Apache Spark, Flink, Kafka, Airflow, Delta Lake, and Iceberg.
$140,100 - $210,100 a yearFull-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
We process billions of mail messages using cutting edge algorithms in areas including but are not limited to: Natural language processing, GenAI, Large Language Models, Machine Learning techniques, big data processing in order of petabytes to: Extract information, build mail content and user knowledge, and interconnect different sources to identify, highlight and amplify what matters.
ExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Would be preferred to have executed end to end implementation/migration of a Data Warehouse and Big Data (Hadoop) project from on prem to GCP (using Google BigQuery, DataFlow, DataProc etc.
Full-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Big Data & Streaming: 2+ years using big data and/or streaming technologies (e.g. Apache Spark, Apache Kafka, Apache Flink) Data: Good understanding of data & data processing tools (e.g. Spark, Kafka, SQL), of relational database technologies and of analytics databases (e.g. Redshift, Vertica, Snowflake.
Full-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
4+ years of enterprise data warehouse development with SQL, Python, Azure Data Factory (ADF), Azure Databricks, Kafka, and ideally Snowflake experience. Looking for a Data Engineer with SQL, Python, ADF, Databricks, Kafka and ideally Snowflake experience to work 100% remote.
RemoteExpandApply NowActive JobUpdated 5 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Design, develop and deliver scalable, robust and highly re-usable components using technologies such as Python, Java, AWS serverless (Lambda, Glue), Apache Spark, Apache Kafka and REST. Collaborate with other peer organizations (e.g., Business Analyst, Data Modeler, QA, technical support, etc.
Full-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
Sound experience in Google Cloud Data Services: Big Query, Data Proc, PubSub, Cloud Functions, Cloud Storage, Dataflow, Composer. Strong knowledge of large-scale search applications and building high volume data pipelines, preferably using PySpark on GCP and It’s native tools such as BigQuery, Airflow, Composer, DataProc, PUB/SUB, DataFlow and Vertex AI.
Full-timeExpandApply NowActive JobUpdated Today - UpvoteDownvoteShare Job
- Suggest Revision
This Data Mining and Analysis Lead Engineer role is centered around supporting the Vehicle, Hardware-in-the-Loop (HIL), Software-in-the-Loop (SIL), and Virtual Simulation test teams with robust and accurate scripts, both for automating tests and for analyzing the resulting test data for Autonomous Driving OEM projects.
Full-timeExpandApply NowActive JobUpdated Today
hadoop kafka big data mining jobs Company: Smx Corporation
FEATURED BLOG POSTS
How to Get a Job in a New City - Best Tips to Follow
When your dream job posts, will you be ready? What if that dream job was in a faraway city or a different continent…would you move for it? Most of us have dreamed about what it would be like to accept a job in a new place at one time or another. But when it comes to actually making a move, it’s usually the logistical details that give us pause. The key to job searching in a different location is to have a plan.
What is a Policy of Non-Retaliation & How to Create One
Your company probably has many policies in place – anti-discrimination policies, equal opportunity policies, ethics policies, and so on. However, it is important to re-evaluate whether you have a policy that covers every situation. Say for instance one of your employees submit a complaint in good faith to their supervisor. To their surprise, they are met with retaliation, such as demotion, defamation, or even termination.
When to Tell Your Boss You're Job Searching
The Bureau of Labor Statistics has found that individuals, on average, have 12.4 jobs throughout their working life. So regardless of where you are on the career ladder, it’s inevitable that at some point, you’ll begin looking for another job. That said, you might be wondering,
What Questions Can You Not Ask in an Interview?
It can be stressful to interview candidates to fill an open role at your company. Maybe your boss is on you to hire someone who’s absolutely perfect as soon as possible. Maybe you don’t have a ton of experience yet in conducting interviews. Whatever the case, there’s a lot to take under consideration when you’re the one in charge of interviewing. For instance, “What questions can you not ask in an interview?” may very well be running through your mind given its legal ramifications. Knowing exactly which interview questions are off-limits (and which are fine to ask) will boost your confidence as you continue to search for the ideal new hire.
How to Prepare to Be Fired - What You Need to Do
If you’re reading this, let me be the first to tell you how sorry I am. Getting fired feels crappy, disheartening, hurtful, and all the other bad, sad words. But here’s what I want you to do. First, let yourself fumble for a minute. Then, pick your head up — sometimes getting fired is a blessing in disguise. If you think termination is around the corner, we’ll teach you how to prepare to be fired and what to do next so you land somewhere even better.
How to Find a Job That Makes You Happy - 11 Concerning Facts
Do you ever feel like your life is like one of those rom-com movie scene openers? You know, the ones where the main character rolls out of bed, awakened by a casually upbeat theme song, sulks their way to the coffee pot, and then trudges toward their computer to begin yet another boring day at work?
How to Decline a Job Offer You Already Accepted
When you think about it, turning down a job offer is not the worst position you could be in. If you’ve been lucky enough to consider multiple job offers, well, then you’re lucky enough.