- UpvoteDownvoteShare Job
- Suggest Revision
You will primarily be working on Apache Spark/DataLake/Airflow (in Python) and other cloud-based technologies. Experience in Apache Spark, Databricks or similar distributed processing systems.
Full-timeExpandApply NowActive JobUpdated 5 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Python, MWAA (Airflow), Spark Optimization, AWS Glue, AWS EC2, Apache Spark. · Develop test maintain and document data pipelines using DBT Spark AWS Glue and Airflow.
ExpandApply NowActive JobUpdated 3 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Direct hands on experience with highly scalable data pipelines using BigData technologies (Spark, Hive, Airflow, DBT, Parquet / ORC, Kafka / Streaming, etc) Distributed Compute: Spark/Databricks.
Full-timeExpandApply NowActive JobUpdated 5 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Expertise in Spark Streaming, Apache Airflow, Cloud (Azure preferred), and on-premises open-source data systems. Design and implement high-performance data processing pipelines using Spark Streaming, Databricks, Snowflake and other technologies to ensure data security and privacy.
$215,000 - $322,250 a yearFull-timeExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure DevOps. In-depth knowledge of spark internals, catalyst optimization, and Databricks runtime environment. Technologies Databricks, Delta Lake storage, Spark (PySpark, Spark SQL.
ExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Experience with distributed data/computing tools, such as: Spark, Airflow, dbt. You have worked with Dagster/Airflow, BigQuery, GCP, Terraform, Kubernetes, sklearn, keras/TensorFlow/pytorch, dbt, data modeling, Python/Pandas data frameworks, and scalable technical concepts/solutions.
ExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Desired experience in: Looker / Google Data Studio, BigQuery, Apache Airflow / Google Cloud Composer. Experienced with Google Cloud Composer / Apache Airflow or similar data orchestration services.
RemoteExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
O Programming Language: Python, R, SQL, Java, Scala, Pyspark/Apache Spark, Shell scripting. Use programming languages including but not limited to Python, R, SQL, Java, Scala, Pyspark/Apache Spark, Shell scripting.
ExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Experience with streaming / data processing technologies such as Beam, Spark, Kafka, Airflow, HBase, Presto. Experience in stream / data processing technologies like Kafka, Spark, Google BigQuery, Google Dataflow, Presto, HBase.
ExpandApply NowActive JobUpdated 6 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Experience with data processing technologies and tools, such as Spark, Hadoop(HDFS, Hive, MapReduce), Athena, Airflow, Luigi. Build big data processing pipelines using Spark and Airflow.
ExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Experience working with Airflow as workflow management tools and build operators to connect, extract and ingest data as needed. Experience in Real Time Data Ingestion using GCP PubSub, Kafka, Spark or similar.
ExpandApply NowActive JobUpdated 8 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Includes Snowflake, Apache Airflow, AWS cloud infrastructure (e.g., Kinesis, Kubernetes/EKS, Lambda, Aurora RDS PostgreSQL), Python and Typescript. Experience building and managing production-grade data pipelines using tools such as Airflow, dbt, DataHub, MLFlow.
ExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Java, Spring, Spring Boot, AWS, Kafka, Kubernetes, Oracle, Postgres, DynamoDB, MongoDB, RESTful, Airflow, Redis, Hive, Presto etc. Full-stack experience in one or more of the following: Java, Spring, JavaScript, SQL, ReactJS, Redux, Spark, Kafka, Kubernetes, AWS, and the confidence to easily pick up others.
ExpandApply NowActive JobUpdated 4 days ago - UpvoteDownvoteShare Job
- Suggest Revision
Extensive experience using Snowflake, AWS SageMaker, GCP/GCP BigQuery, DBT, and Airflow for AI/ML model deployment and operationalization. Strong programming skills with one or more languages such as Python, PySpark, Apache Spark, Java, JavaSript, C, C.
Full-timeExpandApply NowActive JobUpdated 6 days ago - UpvoteDownvoteShare Job
- Suggest Revision
This includes, but is not limited to, setting up SageMaker instances, creating Docker images, configuring lambda functions, setting up orchestration layer via Airflow, deploying endpoints, and creating API gateways.
Full-timeExpandApply NowActive JobUpdated 6 days ago
airflow spark jobs
FEATURED BLOG POSTS
Talent Mapping for the Rest of the Year
As you enter the next quarter of 2023, it's important to reflect on how well your talent strategy is aligning with your business goals. This is an opportune time to design or reassess your talent mapping approach, so your recruiting and hiring scheme going forward stays in line with this year's business goals.
How To Decorate a Cubicle for Inspiration & Productivity
You know the negative reputation cubicles have – dull, dreary, gray. How can you possibly be expected to sit in one every day and be your naturally productive, enthusiastic, and upbeat self? The answer lies in decorating your cubicle so it sparks creativity and feels welcoming. While this might seem impossible given the bland canvas you have to work with, take heart because it’s totally doable!
Virtual Reality Job Interviews
With the advent of desktop computers, the arduous task of scouring through weekly job classifieds became a thing of the past. The mid-1990s brought about a new era where job seekers could easily search and apply for jobs online. The introduction of AOL's Instant Messaging feature provided an even faster means for employers and candidates to communicate and schedule interviews. As smartphones became more pervasive in the early 2000s, hiring managers increasingly used phone calls for screening and interviewing candidates. Despite this trend, over 80% of interviews still took place in person.
A Potential TikTok Ban?!
As you may already know, there has been a lot of talk lately about the possibility of a TikTok ban. While this has not yet come to fruition, it's important to consider the implications this could have for businesses and recruiters who rely on TikTok as a platform to market their brand, recruit new talent, and connect with their audience.
The Effects of Workplace Racism and Sexism
One day it's a covert statement to a mother returning to work after maternity leave. Another day it's a lingering gaze at an employee enjoying a culturally rich meal. These microaggressions (or sometimes macroaggressions) can take an employee from a confident, high-performer to one that feels insecure being themselves at work. Your employees engage with people with different ideas and feel most comfortable and valued when they can work without losing their cultural, racial, and gender identity. While most employers know this, why have workplace racism and sexism often been neglected?
When Rage Applying Strikes: How to Identify Unserious Candidates
As the job market remains highly competitive, we have seen a surge in "rage applying." This is when candidates apply to multiple jobs, often without considering whether they are truly interested in the role. Rage applying goes hand-in-hand with quiet quitting. Often, employees want to entertain the thoughts and feelings of leaving their job, but they aren't necessarily serious about leaving yet. Meanwhile, other employees engaging in this trend are actually trying to find a better role. As a recruiter, it can be hard to identify who are the real applicants in a sea full of quiet quitters, but understanding rage applying and identifying red flags will certainly help.