{"schemaVersion":"jobsearcher.job.v1","id":"eecaf49531b0f6eae5e12dec","url":"https://jobsearcher.com/jobs/eecaf49531b0f6eae5e12dec","canonicalUrl":"https://jobsearcher.com/jobs/eecaf49531b0f6eae5e12dec","title":"Data Platform Engineer","description":"Position Description: Senior Cloud Systems EngineerStart Date – End Date: ASAP– 12/31/2026Note: Period of performance may extend beyond 12/31/2026.Hands-on experience administering Databricks (workspace administration,clusters/compute policies, jobs, SQL warehouses, repos, runtime management) andexpertise using Databricks CLI.• Strong Unity Catalog administration: metastores; catalogs/schemas; grants; serviceprincipals; external locations; storage credentials; governed storage access.• Identity & Access Management proficiency: SSO concepts, SCIM provisioning, group-based RBAC, service principals, least-privilege patterns.• Security fundamentals: secrets management, secure connectivity, audit logging, accessmonitoring, and evidence-ready operations.• Cloud platform expertise (AWS ): IAM roles/policies, object storage security patterns,networking basics (VPC concepts), logging/monitoring integration.• Automation skills: scripting and/or IaC using Terraform/CLI/REST APIs for repeatableconfiguration and environment promotion.• Experience implementing data governance controls (classification/tagging,lineage/metadata integrations) in partnership with governance teams.• CI/CD practices for jobs/notebooks/config promotion across SDLC environments.• Understanding of lakehouse concepts (e.g., Delta, table lifecycle management, separationof storage/compute).• SQL proficiency and data engineering fundamentals for troubleshooting queryperformance issues, understanding ETL/ELT workflow patterns, and debugging datapipeline failures; basic Python/Scala familiarity for notebook/code issue diagnosis.• Experience with compliance and regulatory frameworks (FedRAMP, HIPAA, SOC2, orsimilar) including implementation of data residency requirements, retention policies, andaudit-ready evidence collection.• Hands-on experience with AWS security and networking services including PrivateLink,Secrets Manager/Systems Manager integration, CloudWatch/CloudTrail integration, S3bucket policies, cross-account access patterns, and KMS encryption key management.• Experience administering Databricks serverless compute, Workspace Git integrations(GitLab), Databricks Asset Bundles (DABs) for deployment automation, and modernworkspace features supporting DevOps workflows.• SLA/SLO management and stakeholder communication skills; ability to define platformservice levels, produce operational reports, translate technical issues to businessstakeholders, and manage vendor relationships (Databricks account teams).Education / Experience/Certifications/Accreditations• Bachelor’s degree in a related field or equivalent practical experience.• 7+ years in cloud/data platform administration and operations, including 4+ yearssupporting Databricks or similar platforms.• Databricks Platform Administrator/Databricks AWS Platform Architect• Databricks Certified Data Engineer Associate/Professional• AWS Certified Solutions Architect Associate or ProfessionalThe Contractor shall deliver, but not limited to, the following:• Administer Databricks account and workspaces across SDLC environments; standardizeconfiguration, naming, and operational patterns.• Configure and maintain clusters/compute, job compute, SQL warehouses, runtimeversions, libraries, repos, and workspace settings.• Implement platform monitoring/alerting, operational dashboards, and health checks;maintain runbooks and operational procedures.• Provide Tier 2/3 operational support: troubleshoot incidents, perform root-cause analysis,and drive remediation and preventive actions.• Manage change control for upgrades, feature rollouts, configuration changes, andintegration changes; document impacts and rollback plans.• Enforce least privilege across platform resources (workspaces, jobs, clusters, SQLwarehouses, repos, secrets) using role/group-based access patterns.• Configure and manage secrets and secure credential handling (secret scopes / keymanagement integrations) for platform and data connectivity.• Enable and maintain audit logging and access/event visibility; support security reviewsand evidence requests.• Administer Unity Catalog governance: metastores, catalogs/schemas/tables, ownership,grants, and environment/domain patterns.• Configure and manage external locations, storage credentials, and governed access tocloud object storage.• Partner with governance stakeholders to support metadata/lineage integration,classification/tagging, and retention controls where applicable.• Coordinate secure connectivity and guardrails with cloud/network teams: privateconnectivity patterns, egress controls, firewall/proxy needs.• Configure cloud integrations required for governed data access and service connectivity(roles/permissions, endpoints, storage integrations).• Implement cost guardrails: cluster policies, auto-termination, scheduling, workload sizingstandards, and capacity planning.• Produce usage/cost insights and optimization recommendations; address waste drivers(idle compute, oversized clusters, inefficient jobs).• Automate administration and configuration using APIs/CLI/IaC (e.g., Terraform) toreduce manual drift and improve repeatability.• Maintain platform documentation: configuration baselines, security/governancestandards, onboarding guides, and troubleshooting references.• Design and implement backup and disaster recovery procedures for workspaceconfigurations, notebooks, Unity Catalog metadata, and job definitions; maintainrecovery runbooks and perform periodic DR testing aligned to RTO/RPO objectives.• Monitor and optimize platform performance, including SQL warehouse query tuning,cluster autoscaling configuration, Photon enablement, and Delta Lake optimizationguidance (OPTIMIZE, VACUUM, Z-ordering strategies).• Administer Delta Live Tables (DLT) pipelines and coordinate with data engineeringteams on pipeline health, data quality monitoring, failed job remediation, and pipelineconfiguration best practices.• Manage third-party integrations and ecosystem connectivity, including BI toolintegrations (e.g., Power BI), and external metadata catalog integrations.• Implement Databricks Asset Bundles (DABs) for standardized deployment patterns;automate workspace resource deployment (jobs, pipelines, dashboards) across SDLCenvironments using bundle-based CI/CD workflows.• Conduct capacity planning and scalability analysis, including forecasting concurrentuser/workload growth, platform scaling strategies, and proactive resource allocationduring peak usage periods.• Facilitate user onboarding and enablement, including new user/team onboarding procedures, training coordination, workspace access provisioning, and creation of self-service documentation/guides","company":"Providge Consulting","rawCompany":"providge consulting","city":"Washington","state":"DC","isRemote":false,"isActive":false,"createdAt":"2026-05-23T18:16:32.061Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Platform Engineer","description":"Position Description: Senior Cloud Systems EngineerStart Date – End Date: ASAP– 12/31/2026Note: Period of performance may extend beyond 12/31/2026.Hands-on experience administering Databricks (workspace administration,clusters/compute policies, jobs, SQL warehouses, repos, runtime management) andexpertise using Databricks CLI.• Strong Unity Catalog administration: metastores; catalogs/schemas; grants; serviceprincipals; external locations; storage credentials; governed storage access.• Identity & Access Management proficiency: SSO concepts, SCIM provisioning, group-based RBAC, service principals, least-privilege patterns.• Security fundamentals: secrets management, secure connectivity, audit logging, accessmonitoring, and evidence-ready operations.• Cloud platform expertise (AWS ): IAM roles/policies, object storage security patterns,networking basics (VPC concepts), logging/monitoring integration.• Automation skills: scripting and/or IaC using Terraform/CLI/REST APIs for repeatableconfiguration and environment promotion.• Experience implementing data governance controls (classification/tagging,lineage/metadata integrations) in partnership with governance teams.• CI/CD practices for jobs/notebooks/config promotion across SDLC environments.• Understanding of lakehouse concepts (e.g., Delta, table lifecycle management, separationof storage/compute).• SQL proficiency and data engineering fundamentals for troubleshooting queryperformance issues, understanding ETL/ELT workflow patterns, and debugging datapipeline failures; basic Python/Scala familiarity for notebook/code issue diagnosis.• Experience with compliance and regulatory frameworks (FedRAMP, HIPAA, SOC2, orsimilar) including implementation of data residency requirements, retention policies, andaudit-ready evidence collection.• Hands-on experience with AWS security and networking services including PrivateLink,Secrets Manager/Systems Manager integration, CloudWatch/CloudTrail integration, S3bucket policies, cross-account access patterns, and KMS encryption key management.• Experience administering Databricks serverless compute, Workspace Git integrations(GitLab), Databricks Asset Bundles (DABs) for deployment automation, and modernworkspace features supporting DevOps workflows.• SLA/SLO management and stakeholder communication skills; ability to define platformservice levels, produce operational reports, translate technical issues to businessstakeholders, and manage vendor relationships (Databricks account teams).Education / Experience/Certifications/Accreditations• Bachelor’s degree in a related field or equivalent practical experience.• 7+ years in cloud/data platform administration and operations, including 4+ yearssupporting Databricks or similar platforms.• Databricks Platform Administrator/Databricks AWS Platform Architect• Databricks Certified Data Engineer Associate/Professional• AWS Certified Solutions Architect Associate or ProfessionalThe Contractor shall deliver, but not limited to, the following:• Administer Databricks account and workspaces across SDLC environments; standardizeconfiguration, naming, and operational patterns.• Configure and maintain clusters/compute, job compute, SQL warehouses, runtimeversions, libraries, repos, and workspace settings.• Implement platform monitoring/alerting, operational dashboards, and health checks;maintain runbooks and operational procedures.• Provide Tier 2/3 operational support: troubleshoot incidents, perform root-cause analysis,and drive remediation and preventive actions.• Manage change control for upgrades, feature rollouts, configuration changes, andintegration changes; document impacts and rollback plans.• Enforce least privilege across platform resources (workspaces, jobs, clusters, SQLwarehouses, repos, secrets) using role/group-based access patterns.• Configure and manage secrets and secure credential handling (secret scopes / keymanagement integrations) for platform and data connectivity.• Enable and maintain audit logging and access/event visibility; support security reviewsand evidence requests.• Administer Unity Catalog governance: metastores, catalogs/schemas/tables, ownership,grants, and environment/domain patterns.• Configure and manage external locations, storage credentials, and governed access tocloud object storage.• Partner with governance stakeholders to support metadata/lineage integration,classification/tagging, and retention controls where applicable.• Coordinate secure connectivity and guardrails with cloud/network teams: privateconnectivity patterns, egress controls, firewall/proxy needs.• Configure cloud integrations required for governed data access and service connectivity(roles/permissions, endpoints, storage integrations).• Implement cost guardrails: cluster policies, auto-termination, scheduling, workload sizingstandards, and capacity planning.• Produce usage/cost insights and optimization recommendations; address waste drivers(idle compute, oversized clusters, inefficient jobs).• Automate administration and configuration using APIs/CLI/IaC (e.g., Terraform) toreduce manual drift and improve repeatability.• Maintain platform documentation: configuration baselines, security/governancestandards, onboarding guides, and troubleshooting references.• Design and implement backup and disaster recovery procedures for workspaceconfigurations, notebooks, Unity Catalog metadata, and job definitions; maintainrecovery runbooks and perform periodic DR testing aligned to RTO/RPO objectives.• Monitor and optimize platform performance, including SQL warehouse query tuning,cluster autoscaling configuration, Photon enablement, and Delta Lake optimizationguidance (OPTIMIZE, VACUUM, Z-ordering strategies).• Administer Delta Live Tables (DLT) pipelines and coordinate with data engineeringteams on pipeline health, data quality monitoring, failed job remediation, and pipelineconfiguration best practices.• Manage third-party integrations and ecosystem connectivity, including BI toolintegrations (e.g., Power BI), and external metadata catalog integrations.• Implement Databricks Asset Bundles (DABs) for standardized deployment patterns;automate workspace resource deployment (jobs, pipelines, dashboards) across SDLCenvironments using bundle-based CI/CD workflows.• Conduct capacity planning and scalability analysis, including forecasting concurrentuser/workload growth, platform scaling strategies, and proactive resource allocationduring peak usage periods.• Facilitate user onboarding and enablement, including new user/team onboarding procedures, training coordination, workspace access provisioning, and creation of self-service documentation/guides","datePosted":"2026-05-23T18:16:32.061Z","dateModified":"2026-05-23T18:16:32.061Z","hiringOrganization":{"@type":"Organization","name":"Providge Consulting","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Washington","addressRegion":"DC","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"eecaf49531b0f6eae5e12dec"},"url":"https://jobsearcher.com/jobs/eecaf49531b0f6eae5e12dec"}}