Senior Data Engineer
*Job Title:* Senior Data Engineer
*Job Location:* Richmond, VA 23219
*Job Duration:* 12 Months (Possibility of Extension)
*Shift:* Day Shift
*Work Mode:* Hybrid – Alternating weeks (5 days onsite, 5 days remote).
*Pay Rate:* $59.86/Hour on W2
*Position Summary:*
- Design, build, and optimize scalable, reliable, enterprise-level data pipelines and advanced ETL/ELT solutions.
- Provide hands-on technical leadership by implementing data engineering best practices, troubleshooting complex data challenges, and guiding architectural decisions.
- Develop and optimize batch, incremental, and real-time data processing workflows for analytics, reporting, and AI/ML use cases.
- Ensure data pipelines meet enterprise standards for performance, reliability, security, quality, and governance.
- Collaborate with architecture, infrastructure, platform, and application teams to improve the overall data ecosystem.
*Responsibilities:*
- Build end-to-end data pipelines and ETL/ELT solutions to support analytics, reporting, and AI/ML use cases, ensuring solutions are robust and production-ready through practical implementation.
- Apply scalable patterns for batch and incremental processing by developing, testing, and deploying data workflows, focusing on hands-on coding and troubleshooting.
- Review and implement data modeling, transformation logic, and performance strategies using deep technical expertise to optimize and validate solutions.
- Evaluate, select, and integrate tooling, frameworks, and platform capabilities by actively prototyping and configuring systems to meet project requirements.
- Build complex, high-volume data pipelines using SQL-centric ETL/ELT patterns.
- Design and implement scalable streaming pipelines to process real-time data, ensuring low latency and reliable delivery for analytics and operational use cases.
- Lead performance tuning efforts across pipelines, warehouses, and workloads.
- Ensure data pipelines are resilient, observable, and production ready.
- Implement enterprise-grade error handling, restartability, and monitoring.
- Build and maintain scalable, low-latency streaming data pipelines using technologies such as Kafka, Kinesis, or Spark Streaming.
- Perform on-the-fly data cleaning, validation, and enrichment before data reaches its destination.
- Use strategies such as indexing and partitioning to fine-tune data warehouses and big data environments, improving query response times and scalability.
- Implement standards for data quality checks, validation, and reconciliation.
- Ensure pipelines meet security, access control, and governance requirements.
- Partner with governance and DataOps teams on metadata, lineage, and auditability.
- Apply consistent naming conventions, documentation, and coding standards.
- Improve operational monitoring, alerting, and incident response processes.
- Proactively identify reliability, performance, and cost optimization opportunities.
- Support and guide production troubleshooting and root cause analysis.
- Investigate data quality incidents and identify design and coding gaps.
- Participate in design and code reviews to enforce quality and best practices.
- Partner with infrastructure teams, application teams, and architects to generate process designs and complex transformations across various data elements to provide business insights into business processes.
- Translate ambiguous requirements into well-designed technical solutions.
- Work in complex, multi-platform environments across multiple project assignments.
*Required Qualifications:*
- Must have 5–7+ years of Data Engineering experience.
- Advanced expertise in SQL, ELT patterns, and performance tuning.
- Strong experience with Oracle Exadata, Snowflake, or similar cloud/on-premises data warehouses.
- Hands-on experience with enterprise ETL/ELT platforms such as Talend, dbt, or Informatica.
- Deep understanding of data warehousing architecture and dimensional modeling.
- Experience designing and supporting large-scale, production data pipelines.
- Strong scripting experience, particularly with Python.
- Experience with data virtualization tools such as Denodo, Composite, Dremio, or Starburst.
- Experience with DataOps practices, CI/CD, and observability.
- Experience in ETL development and process support, including the ability to work weekends or outside regular business hours when required.
- Bachelor's degree or higher in Computer Science, Information Systems, Mathematics, or a related discipline.
*Preferred / Nice to Have:*
- Experience supporting AI/ML or advanced analytics pipelines.
- Cloud platform experience with AWS, Azure, or GCP.
- Prior experience influencing enterprise data standards or reference architecture.
- Experience optimizing cost and performance in cloud data warehouses.
- Hands-on experience with Cribl, Apache Kafka, Kafka Connect, Spark Streaming, or Apache Flink.
*Key Behavioral Expectations ("How We Work"):*
- Delivering at Pace: Acts with urgency to meet deadlines while maintaining quality and prioritizing workload effectively under pressure.
- Collaborative Teamwork: Actively supports colleagues, shares knowledge freely, and treats others with dignity and respect.
- Effective Communication: Tailors communication style to the audience, ensuring clarity and transparency in both successes and challenges.
- Ownership and Adaptability: Takes responsibility for outcomes and welcomes feedback for improvement.
- Ability to work independently.
- Achievement-oriented and self-starter mindset.
- Concern for quality and flexibility.
#IRI-IT
Pay: $59.86 per hour
Benefits:
- 401(k)
- Dental insurance
- Health insurance
Work Location: In person