JOBSEARCHER

Senior Data Engineer

ARCHIVED

We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.

Senior Data Engineer – Data Platforms Charlotte, NC36-month contract with potential for extension or full-time conversion. Work authorization required now and in future. No visa sponsorship available. No Corp-to-Corp or 1099 arrangements. W-2 only. Brooksource is searching for a Senior Data Engineer to join our Fortune 500 Energy & Utilities Client. In this role you’ll design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely with product managers, data analysts, BI developers, and other engineers to deliver reliable, high-quality data pipelines across both batch and real-time streaming architectures. The ideal candidate has deep experience building cloud-native data solutions on AWS, strong fundamentals in distributed systems, and hands-on experience with stream processing frameworks such as Apache Flink, using Python and Java. This position follows a hybrid schedule, 3 days a week on site and 2 days remote.
Responsibilities will include:
  • Data Engineering & Platform Development
  • Design, develop, and maintain scalable data pipelines supporting batch and real-time streaming workloads
  • Build and optimize data processing jobs using AWS Glue, PySpark, Python, and Java
  • Develop and maintain stream processing applications using Apache Flink
  • Implement reliable data ingestion solutions using AWS DMS, Kafka, and AWS LambdaDesign and manage data persistence layers using Amazon Aurora (PostgreSQL) and related AWS services
  • Streaming & Batch Processing
  • Design and support real-time and near-real-time streaming pipelines using Kafka and Apache FlinkBuild and maintain stateful stream processing jobs, including windowing, aggregation, and event-time processing
  • Develop efficient batch processing workflows for large-scale data transformation and enrichment
  • Ensure data consistency, latency, fault tolerance, and reliability across streaming and batch systems
  • Data Quality, Reliability & Performance
  • Implement monitoring, logging, and alerting for batch and streaming data pipelines
  • Diagnose and resolve data quality, performance, and scalability issues
  • Apply best practices for schema evolution, checkpointing, fault tolerance, and back-pressure handling
  • Optimize pipelines for performance and cost efficiency
  • Analytics & Consumption
  • Enable downstream analytics and reporting through well-modeled, well-documented datasets
  • Partner with analytics and BI teams supporting tools such as Qlik
  • Support data consumers by improving data discoverability, usability, and trust
  • Collaboration & Engineering Excellence
  • Collaborate with product, analytics, and platform teams to translate requirements into technical solutions
  • Participate in architecture discussions, design reviews, and code reviews
  • Contribute to data engineering standards, reusable frameworks, and documentation
  • Mentor junior engineers and promote best practices across the team
  • Core Technologies This role will work extensively with the following technologies:
  • AWS GluePySparkPythonJavaApache Flink (Stream Processing)
  • KafkaAWS DMSAWS LambdaAmazon Aurora (PostgreSQL)
  • Streaming and Batch Data Processing
  • Streaming and Batch Data Processing
  • Qlik (analytics/BI consumption)
  • Preferred/Nice-to-Have Experience Amazon Redshift
  • Data warehouse concepts (dimensional modeling, star/snowflake schemas)
  • Experience building or supporting enterprise data warehouses or data lakes
  • Familiarity with event-driven architectures and real-time analytics use cases
  • Experience with data governance, metadata management, or lineage tools
Required Qualifications:
  • 5+ years of experience in data engineering or backend engineering
  • Strong hands-on experience with Python and Java in distributed systems
  • Experience building streaming data applications using Apache Flink and KafkaSolid experience with AWS data services
  • Strong SQL skills and understanding of relational data modeling
  • Experience working with large-scale, distributed data systems
What Success Looks Like:
  • Streaming and batch pipelines and reliable, scalable, and observable
  • Real-time data is processed with low latency and high correctness
  • Data is trusted and easily consumable by analytics and downstream systems
  • Systems are designed with fault tolerance, performance and maintainability in mind
  • Engineering best practices are consistently applied and shared
Eight Eleven Group provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, national origin, age, sex, citizenship, disability, genetic information, gender, sexual orientation, gender identity, marital status, amnesty, or status as a covered veteran in accordance with applicable federal, state, and local laws.