GCP Data Engineer
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Data Engineer With Spark & ScalaWe are seeking a Data Engineer with Spark & SCALA; Streaming skills builds real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML.
Responsibilities:
As a Senior Data Engineer, you will design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink).Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming Tuning: Optimize Spark jobs, SQL queries, and data processing workflows for speed, efficiency, and cost-effectiveness Data Quality: Implement data quality checks, monitoring, and alerting systems to ensure data accuracy and consistency.
Required Skills & Qualifications:
Programming: Strong proficiency in Python, SQL, and potentially Scala/Java.
Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming).Streaming: Experience with messaging queues like Apache Kafka, or Pub/Sub.
Cloud: Familiarity with GCP, Azure data services.
Databases: Knowledge of data warehousing (Snowflake, Redshift) and NoSQL : Experience with Airflow, Databricks, Docker, Kubernetes is a plus.