Sr. Data Engineer

Halvik Corp delivers a wide range of services to 13 executive agencies and 15 independent agencies. Halvik is a highly successful WOB business with more than 50 prime contracts and 500+ professionals delivering Digital Services, Advanced Analytics, Artificial Intelligence/Machine Learning, Cyber Security and Cutting-Edge Technology across the US Government. Be a part of something special!Role SummaryThe Senior Data Engineer will be a key contributor within the Data Products Delivery Area, responsible for designing, building, and operating scalable, secure, and high-quality data pipelines that power Business Intelligence & Analytics and AIML products. This role supports a Databricks Data Lakehouse using Medallion Architecture, built on Data Fabric principles, and operates within a release-based Agile framework.This role requires strong hands-on development using Python, PySpark, SQL, and APIs, and experience integrating diverse enterprise and external data sources.Key ResponsibilitiesData Engineering & ArchitectureDesign, develop, and optimize end-to-end data pipelines using Python and PySpark.Build and maintain data ingestion frameworks leveraging REST APIs, streaming APIs, and batch interfaces.Implement Medallion Architecture (Bronze, Silver, Gold) within a Databricks Lakehouse.Integrate data from structured, semi-structured, and unstructured sources, including API-based and event-driven sources.Apply Data Fabric principles including metadata-driven ingestion, lineage, observability, and reusability.Collaborate with Senior Data Architects on logical and physical data models.Data Products & DeliveryDeliver data assets as releasable productsSupport downstream consumers including: BI&A dashboards and visualizations, AIML feature engineering, training, and inference pipelines, Data services and curated datasets exposed via APIsDecompose requirements into well-defined user stories aligned with MoSCoW prioritization.Data Quality, Governance & SecurityImplement data validation, reconciliation, and quality checks within PySpark pipelines.Ensure secure handling of PII and sensitive data, including encryption, masking, and access controls.Partner with CloudOps (DevSecOps, FinOps, InfraOps) to ensure:Secure API access and secrets managementCI/CD automation for Python and PySpark workloadsCost-optimized compute and storage usageProduction Support & OptimizationSupport O&M for production data pipelines, APIs, and analytics products.Tune Spark jobs, SQL queries, and API integrations for performance and reliability.Implement monitoring, logging, and alerting for data workflows.Drive automation and refactoring to improve resiliency and scalability.Collaboration & LeadershipMentor junior and mid-level data engineers on Python, PySpark, and API integration best practices.Work closely with Business Analysts & Data Architects, Data Scientists & ML Engineers, BI Developers, Automation Test EngineersContribute to reusable frameworks, coding standards, and engineering best practices.Required Skills & ExperienceCore Technical SkillsStrong hands-on experience with Python for data engineering and API integration, PySpark / Apache Spark for large-scale data processing, SQL (advanced querying and performance tuning)Proven experience building API-driven data ingestion pipelines (REST, JSON, OAuth, pagination, throttling).Strong experience with Databricks, Delta Lake, and Lakehouse Architecture.Experience implementing Medallion Architecture in production environments.Cloud & DevOpsExperience working in cloud platforms (AWS, Azure, or GCP).Familiarity with CI/CD pipelines for Python and Spark workloads, Infrastructure as Code concepts, Secure API authentication and secrets managementCost-conscious engineering mindset (FinOps exposure is a plus).Data Management & AnalyticsExperience supporting BI&A workloads at scale.Understanding of data modeling for analytics and reporting.Experience enabling data consumption for AI/ML use cases.Agile & DeliveryProven experience working in Agile Scrum / Scrum of Scrums environments.Comfortable working with release-based delivery and production support.Nice to HaveExperience exposing data via internal or external APIs.Familiarity with streaming technologies (Kinesis, Firehose, Kafka, Event Hubs, etc.).Experience with data governance, cataloging, and lineage tools.Exposure to feature stores or ML data pipelines.Soft SkillsStrong analytical and problem-solving skills.Excellent communication and collaboration abilities.Ownership mindset with a focus on reliability and quality.Halvik Offers a Competitive Full Benefits Package IncludingCompany-supported medical, dental, vision, life, STD, and LTD insuranceBenefits include 11 federal holidays and PTOEligible employees may receive performance-based incentives in recognition of individual and/or team achievements.401(k) with company matchingFlexible Spending Accounts for commuter, medical, and dependent care expensesTuition AssistanceCharitable Contribution matchingHalvik Corp is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status.Halvik's pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.