Databricks Integration Architect

Job Title: Databricks Integration ArchitectLocation: Remote - Washington DCDuration: 16+ MonthsClient: Amtrak (Railways)Job ID: C.IT.100678.SADAI.07/1Note:We have shifted to adopting SAFe and encourage Contractors trained in SAFeRequest that contractors have camera available and on majority of the timeSet expectations that they are part of Agile team on which we need them to embrace self-management and self-organization in terms of their iteration commitments, capacity planning, capacity allocation, etc.As long as they can come DC on need basis and attend the important SAFe ceremonies like PI onsiteJob and Role Overview:We are seeking a Senior IT Applications Solution Architect to lead the strategic design, integration, and architectural governance of our enterprise Databricks platform and connected application ecosystem. In this role, you will not just execute blueprints—you will create them. You will design high-performance data architecture frameworks, establish enterprise-wide integration standards, and serve as the technical authority ensuring our Medallion Lakehouse seamlessly powers downstream business applications, APIs, and core IT systems. You will collaborate closely with platform infrastructure teams, business stakeholders, and engineering squads to turn strategic business objectives into scalable, secure technological realities.Key Responsibilities:Solution Architecture & Enterprise IntegrationArchitectural Blueprinting: Design and document comprehensive, end-to-end integration patterns utilizing Databricks Delta Live Tables (DLT), structured streaming, and advanced orchestration to link data assets with core enterprise applications.Application Ecosystem Alignment: Lead the integration strategy between the Databricks Lakehouse platform and downstream operational applications, microservices, business intelligence tools, and external APIs.Data Governance & Quality Frameworks: Architect and enforce data quality and compliance frameworks (including automated DLT "Expectations" templates, Unity Catalog governance, and data lineage tracking) across the entire application portfolio.Guardrails, Standards & Technical LeadershipEstablish Enterprise Standards: Define, maintain, and evangelize coding standards, naming conventions, optimization guardrails, and reusable design patterns for engineering teams to follow.Framework Development: Partner with Core Platform and Cloud Infrastructure teams to build reusable, secure platform libraries and abstraction layers that accelerate delivery.Architectural Review & Mentorship: Lead architectural review boards, oversee peer code/design reviews, and mentor mid-to-senior engineers to ensure alignment with enterprise-wide design principles.Platform Optimization & DevOps ArchitecturePerformance & Cost Strategy (FinOps): Architect global optimization and performance tuning strategies (e.g., Z-Ordering, Liquid Clustering, Liquid Partitioning, and Photon compute strategies) to balance high throughput with cloud compute costs.CI/CD & Lifecycle Governance: Design and govern robust, multi-tier CI/CD pipelines and deployment topologies (Dev, Test, Stage, Prod) to safely manage and promote Databricks assets, infrastructure as code (IaC), and data configurations.Required Qualifications:Professional Experience: 8+ years of experience in IT solution architecture, application design, or senior-level data engineering, with a proven track record of architecting cloud data platform ecosystems.Databricks Architectural Mastery: Authoritative, hands-on experience designing and implementing production-grade architectures exclusively leveraging the Databricks platform, PySpark, SQL, and Delta Lake core concepts.Advanced DLT & Streaming Expertise: Deep technical proficiency blueprinting complex Delta Live Tables (DLT) architectures and real-time structured streaming solutions.Enterprise Data Modeling: Expert-level understanding of designing relational, dimensional, and Medallion (Bronze/Silver/Gold) architectures optimized for multi-application consumption.DevOps & Automation Architecture: Strong experience designing enterprise deployment strategies and automated infrastructure provisioning using Git, Azure DevOps, GitHub Actions, or Terraform.Collaborative Leadership: Exceptional capability to lead cross-functional technical teams, drive consensus across conflicting technical visions, and clearly articulate technical roadmaps to both engineering teams and non-technical stakeholders.