Principal Data Engineer
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Principal Data Engineer
Job Description: Principal Data Engineer
Location: Johnston, RI (One Citizens Way) or Charlotte, NC or Boston, MARole Overview
Principal-level Java engineer to design and build enterprise-grade, real-time and batch data processing systems using Java, Spark, Kafka, and Microservices architecture. Strong focus on event-driven pipelines, API development (build + consume), and high-volume streaming platforms.
Key Responsibilities:
- Architect, design, and implement enterprise-grade Java-based data platforms and distributed processing systems
- Build and maintain production-ready Spark applications (Java) for batch and real-time processing
- Design and evolve Kafka-based event streaming and ingestion pipelines
- Develop and consume REST APIs within microservices architecture
- Lead architecture ensuring scalability, reliability, and regulatory compliance
- Apply strong object-oriented design and engineering practices
- Mentor engineers on performance tuning and production readiness
- Design and implement MDM solutions (match, merge, survivorship logic)
- Ensure data quality, observability, and system stability
- Support production deployments and operational handoffs
Required Skills & Experience:
- 10–12+ years experience in Java/backend or data engineering
- Hands-on experience building real-time data pipelines (Kafka, Spark Streaming/Flink)
- Solid knowledge of relational databases (Redshift, PostgreSQL, Snowflake) and NoSQL databases (MongoDB or similar)
- Strong Kafka and event-driven architecture experience
- Strong Microservices experience (Spring Boot, REST APIs)
- Experience in API development and API consumption
- Hands-on Spark experience (batch and streaming)
- Strong SQL and data modeling skills
- AWS experience (S3, Glue, EMR, Redshift)
- Experience in regulated/data governance environments
- CI/CD, Git, Docker/Kubernetes familiarity Preferred
- Scala or Python experience
- Talend/DataStage exposure
- Data lake experience (Iceberg/Parquet)
- Frontend/API integration exposure
- Experience supporting large-scale production systems
Mandatory Screening Criteria
Candidates must have hands-on experience building real-time/event-driven data pipelines using Kafka and Spark/Flink, along with strong microservices and API development experience. Data engineer with Spark + SQL + Redshift/ Snowflake + ETL datalake experience
Job Description:
- Design, develop, and maintain scalable ETL/ELT pipelines using Apache Spark (PySpark/Scala).
- Build and manage data lake solutions on Amazon S3, ensuring efficient storage and retrieval using Parquet and Iceberg formats.
- Implement and maintain Change Data Capture (CDC) pipelines for real-time and near real-time data ingestion.
- Develop batch and streaming data pipelines using Kafka and Spark Structured Streaming.
- Optimize data processing jobs for performance, scalability, and cost efficiency (partitioning, parallelism, memory tuning).
- Design and maintain data models and schemas for analytics and reporting, including Redshift data warehouse integration.
- Work with AWS EMR / EMR Serverless for distributed data processing.
- Ensure data quality, reliability, and consistency across pipelines.
- Implement job scheduling, monitoring, and alerting using AutoSys or similar schedulers.
- Troubleshoot and resolve pipeline failures, performance bottlenecks, and data inconsistencies.
- Collaborate with cross-functional teams (data analysts, data scientists, and application teams) to deliver data solutions. SNOW- Service Now – (Not sure whether we can source from our practice or not )
Job Description:
- -Reviews and guides solution design to ensure best practices are followed
- Translates business needs into effective ServiceNow solutions
- Establishes and enforces platform governance, standards, and guardrails
- Oversees complex development work, integrations, and enterprise workflows
- Supports solution architecture across the ServiceNow platform
- Designs end-to-end solutions and ensures alignment with enterprise architecture
- Drives integration strategy across systems (e.g., Azure, Jira, Teams, etc.)Provides architectural oversight for platform changes and enhancements
- Leads governance of scalable, secure, and high-performing solutions
- Platform Operations & Execution
- Supports incident resolution, troubleshooting, and root-cause analysis
- Stakeholder & Strategic WorkActs as a liaison between business stakeholders and technical teams
- Advises leadership and contributes to roadmap and priorities
- Drives platform improvements, innovation, and automation ideas
- Represents the ServiceNow platform across the organization
- Operational / Administrative Contributions
- Assists with reporting and analysis when needed