{"schemaVersion":"jobsearcher.job.v1","id":"ed2ebac5be3d35bd62eb6c12","url":"https://jobsearcher.com/jobs/ed2ebac5be3d35bd62eb6c12","canonicalUrl":"https://jobsearcher.com/jobs/ed2ebac5be3d35bd62eb6c12","title":"Senior Databricks Data Engineer","description":"Senior Databricks Data Engineer\n\nOur Company\n\nN2IA Technologies is a consulting company specializing in acquisition/contracting support, cost/FinOps, and technology optimization for federal clients. We deliver tailored strategies, robust software solutions, and streamlined operations to help organizations achieve their goals. At N2IA, we are committed to developing innovative financial and compliance-based solutions that address evolving business and regulatory needs. If you are passionate about accounting, financial accuracy, and supporting government programs, N2IA is the place for you.\n\nOverview\n\nN2IA is seeking a Senior Databricks Data Engineer to build, refine, and maintain the core data pipelines on the Databricks Lakehouse Platform that enable enterprise AI, analytics, and mission systems. This role is essential for making data accessible, reliable, and usable across the agency. You will work with cloud engineers, data architects, and AI developers to ensure data is clean, connected, secure, and optimized for advanced analytics and AI/ML workflows. This role is remote supporting a Quantico, VA customer and onsite work is required at the client site full-time.\n\nKey Responsibilities:\n\nDesign, build, and maintain scalable data pipelines and ETL/ELT workflows on Databricks using Delta Lake, Delta Live Tables (DLT), and the medallion (bronze/silver/gold) architecture across cloud and on-prem systems.\nDevelop and orchestrate workflows using Databricks Workflows/Jobs.\nImplement data quality checks, validation, lineage tracking, and metadata management using Unity Catalog and Lakehouse Monitoring / DLT expectations.\nCollaborate with other teams to ensure datasets meet requirements and are discoverable through Unity Catalog.\nAutomate ingestion and transformation processes using Databricks-native patterns such as Auto Loader, Structured Streaming, and the COPY INTO command.\nSupport API strategy, data modeling, and domain-driven data structures within the Lakehouse.\nTroubleshoot pipeline issues and optimize Spark jobs and clusters for performance and cost (e.g., Photon, cluster sizing, Z-ordering, liquid clustering, and workload tuning).\n\nRequired Qualifications:\n\n3+ years of hands-on data engineering experience building production pipelines on Databricks.\n10+ years of technical professional experience.\nStrong proficiency with Apache Spark (PySpark and Spark SQL), Delta Lake, and the Databricks Lakehouse Platform.\nHands-on experience with Databricks on Azure and/or AWS (e.g., Azure Databricks, Azure Data Factory, AWS Glue, or similar orchestration tooling).\nExperience with Unity Catalog for data governance, access control, and lineage.\nExperience integrating structured and unstructured data sources.\nKnowledge of SQL, schema design, data modeling, and cloud-based storage patterns (e.g., Delta tables, cloud object storage).\nSecret clearance.\n\nPreferred Qualifications:\n\nDatabricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).\nExperience supporting AI/ML pipelines, vectorization, or embedding generation, ideally using MLflow, Databricks Mosaic AI, Model Serving, or Vector Search.\nExperience with Delta Live Tables and streaming pipelines (Structured Streaming / Auto Loader).\nAzure, AWS, or multi-cloud data ecosystem familiarity.\nExperience with data governance, access controls, or sensitive data protection (Unity Catalog, row/column-level security).\nExposure to DevOps/CI-CD patterns for data workflows (e.g., Databricks Asset Bundles, Repos, Terraform).\nMuleSoft implementation experience.\n\nN2IA is committed to fostering a diverse and inclusive work environment. We are an Equal Employment Opportunity Employer and encourage applications from all qualified individuals, regardless of gender, race, ethnicity, sexual orientation, disability, or veteran status.","company":"N2ia Technologies","rawCompany":"n2ia technologies","city":"Washington","state":"DC","isRemote":false,"isActive":false,"createdAt":"2026-11-18T11:47:18.884Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Senior Databricks Data Engineer","description":"Senior Databricks Data Engineer\n\nOur Company\n\nN2IA Technologies is a consulting company specializing in acquisition/contracting support, cost/FinOps, and technology optimization for federal clients. We deliver tailored strategies, robust software solutions, and streamlined operations to help organizations achieve their goals. At N2IA, we are committed to developing innovative financial and compliance-based solutions that address evolving business and regulatory needs. If you are passionate about accounting, financial accuracy, and supporting government programs, N2IA is the place for you.\n\nOverview\n\nN2IA is seeking a Senior Databricks Data Engineer to build, refine, and maintain the core data pipelines on the Databricks Lakehouse Platform that enable enterprise AI, analytics, and mission systems. This role is essential for making data accessible, reliable, and usable across the agency. You will work with cloud engineers, data architects, and AI developers to ensure data is clean, connected, secure, and optimized for advanced analytics and AI/ML workflows. This role is remote supporting a Quantico, VA customer and onsite work is required at the client site full-time.\n\nKey Responsibilities:\n\nDesign, build, and maintain scalable data pipelines and ETL/ELT workflows on Databricks using Delta Lake, Delta Live Tables (DLT), and the medallion (bronze/silver/gold) architecture across cloud and on-prem systems.\nDevelop and orchestrate workflows using Databricks Workflows/Jobs.\nImplement data quality checks, validation, lineage tracking, and metadata management using Unity Catalog and Lakehouse Monitoring / DLT expectations.\nCollaborate with other teams to ensure datasets meet requirements and are discoverable through Unity Catalog.\nAutomate ingestion and transformation processes using Databricks-native patterns such as Auto Loader, Structured Streaming, and the COPY INTO command.\nSupport API strategy, data modeling, and domain-driven data structures within the Lakehouse.\nTroubleshoot pipeline issues and optimize Spark jobs and clusters for performance and cost (e.g., Photon, cluster sizing, Z-ordering, liquid clustering, and workload tuning).\n\nRequired Qualifications:\n\n3+ years of hands-on data engineering experience building production pipelines on Databricks.\n10+ years of technical professional experience.\nStrong proficiency with Apache Spark (PySpark and Spark SQL), Delta Lake, and the Databricks Lakehouse Platform.\nHands-on experience with Databricks on Azure and/or AWS (e.g., Azure Databricks, Azure Data Factory, AWS Glue, or similar orchestration tooling).\nExperience with Unity Catalog for data governance, access control, and lineage.\nExperience integrating structured and unstructured data sources.\nKnowledge of SQL, schema design, data modeling, and cloud-based storage patterns (e.g., Delta tables, cloud object storage).\nSecret clearance.\n\nPreferred Qualifications:\n\nDatabricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).\nExperience supporting AI/ML pipelines, vectorization, or embedding generation, ideally using MLflow, Databricks Mosaic AI, Model Serving, or Vector Search.\nExperience with Delta Live Tables and streaming pipelines (Structured Streaming / Auto Loader).\nAzure, AWS, or multi-cloud data ecosystem familiarity.\nExperience with data governance, access controls, or sensitive data protection (Unity Catalog, row/column-level security).\nExposure to DevOps/CI-CD patterns for data workflows (e.g., Databricks Asset Bundles, Repos, Terraform).\nMuleSoft implementation experience.\n\nN2IA is committed to fostering a diverse and inclusive work environment. We are an Equal Employment Opportunity Employer and encourage applications from all qualified individuals, regardless of gender, race, ethnicity, sexual orientation, disability, or veteran status.","datePosted":"2026-11-18T11:47:18.884Z","dateModified":"2026-11-18T11:47:18.884Z","hiringOrganization":{"@type":"Organization","name":"N2ia Technologies","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Washington","addressRegion":"DC","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"ed2ebac5be3d35bd62eb6c12"},"url":"https://jobsearcher.com/jobs/ed2ebac5be3d35bd62eb6c12"}}