{"schemaVersion":"jobsearcher.job.v1","id":"07381c67ae2e4b7e2e41e639","url":"https://jobsearcher.com/jobs/07381c67ae2e4b7e2e41e639","canonicalUrl":"https://jobsearcher.com/jobs/07381c67ae2e4b7e2e41e639","title":"Senior Databricks Engineer","description":"Senior Databricks Engineer / Technical Lead\n\nWe are seeking a Senior Databricks Engineer / Technical Lead with 10+ years of experience in data engineering, data warehousing, and ETL/ELT, including significant hands-on experience delivering modern cloud data solutions.\n\nThis role is approximately equal parts technical leadership, hands-on development, and solution architecture. The ideal candidate can define the technical direction for a Databricks implementation, guide and mentor engineering teams, and personally develop and troubleshoot complex data solutions when needed.\n\nThe individual will work closely with business stakeholders, data architects, analysts, and engineering teams to translate business requirements into scalable, secure, and maintainable data solutions.\n\nKey Responsibilities\nLeadership & Delivery\nProvide technical leadership for Databricks and broader data engineering initiatives.\nLead and mentor data engineers through design, development, testing, deployment, and production support.\nBreak architectural designs and business requirements into actionable engineering work.\nEstablish development standards, patterns, and reusable frameworks for the engineering team.\nLead code and design reviews and ensure solutions meet performance, quality, security, and maintainability standards.\nCoordinate technical delivery across engineering, architecture, analytics, governance, and business teams.\nIdentify technical risks, dependencies, and delivery challenges and drive them to resolution.\nMentor junior and mid-level engineers while remaining actively engaged in delivery.\nHands-On Development\nDesign and develop scalable data pipelines using Databricks, Apache Spark, PySpark, Python, and SQL.\nBuild ingestion, transformation, and publishing processes across Bronze, Silver, and Gold layers.\nDevelop and optimize Delta Lake tables and large-scale Spark workloads.\nBuild batch and streaming solutions using capabilities such as Databricks Workflows, Auto Loader, Structured Streaming, and Lakeflow Declarative Pipelines where appropriate.\nDevelop reusable data engineering frameworks for ingestion, transformation, validation, monitoring, and exception handling.\nImplement data quality controls, reconciliation processes, logging, and operational monitoring.\nTroubleshoot complex data, performance, integration, and production issues.\nEstablish automated testing and CI/CD practices for Databricks solutions.\nArchitecture & Design\nDesign scalable lakehouse and data platform architectures using Databricks and cloud-native technologies.\nDefine appropriate patterns for data ingestion, transformation, storage, consumption, and integration.\nApply Medallion Architecture and other modern data engineering patterns based on solution requirements.\nDesign data models and processing patterns that support analytics, reporting, data science, and AI use cases.\nDefine strategies for performance, scalability, resiliency, security, and operational support.\nEstablish data governance and access patterns using Unity Catalog, including permissions, metadata, lineage, and data organization.\nEvaluate technical options and make pragmatic architecture decisions based on business requirements, cost, complexity, and maintainability.\nEnsure solutions integrate effectively with existing enterprise data warehouses, data lakes, databases, APIs, and downstream applications.\nRequired Qualifications\n10+ years of experience in data engineering, data warehousing, ETL/ELT, business intelligence, or enterprise data platforms.\nSignificant hands-on experience designing and developing solutions using Databricks.\nDemonstrated experience leading data engineering teams and providing technical direction.\nStrong experience with:\nDatabricks\nApache Spark\nPySpark / Python\nSQL\nDelta Lake\nData warehousing and dimensional modeling\nETL/ELT architecture and development\nStrong understanding of lakehouse, data lake, and enterprise data warehouse architectures.\nExperience designing and implementing Medallion Architecture patterns.\nExperience with Unity Catalog and enterprise data governance concepts.\nExperience with at least one major cloud platform, preferably Microsoft Azure, AWS, or Google Cloud.\nExperience implementing source control, automated testing, and CI/CD for data engineering solutions.\nStrong understanding of data quality, metadata, lineage, security, access management, and governance.\nAbility to translate business requirements into technical architecture and executable engineering plans.\nStrong communication skills with the ability to work with both engineering teams and senior business/technology stakeholders.\nPreferred Qualifications\nExperience with Azure Data Lake Storage, Azure Data Factory, Azure DevOps, and related Azure services.\nExperience with Databricks Asset Bundles and automated deployment approaches.\nExperience modernizing or migrating traditional ETL/data warehouse platforms to Databricks.\nExperience integrating Databricks with analytics platforms such as Power BI.\nExperience supporting data science, machine learning, GenAI, or advanced analytics workloads.\nDatabricks professional-level certification or equivalent demonstrated expertise.\nWhat We're Looking For\n\nThis is not a pure architect, engineering manager, or individual contributor role.\n\nThe successful candidate should be equally comfortable:\n\nLeading\nProviding technical direction, mentoring engineers, managing technical delivery, conducting reviews, and communicating with stakeholders.\n\nBuilding\nWriting PySpark, Python, and SQL; building pipelines; optimizing workloads; troubleshooting difficult problems; and producing production-quality solutions.\n\nArchitecting\nDesigning the overall solution, selecting appropriate patterns and technologies, defining integration and governance approaches, and ensuring the platform can scale.\n\nA strong candidate should be able to whiteboard the architecture in the morning, build or review the critical pipeline in the afternoon, and lead the engineering team through delivery.\n\nEqual Employment Opportunity Statement\n\nTechstra Solutions is an equal opportunity employer. The Company makes its decisions on merit, and its policy of equal opportunity prohibits discrimination in all phases of the employment process, including, but not limited to, recruitment, hiring, promotion, selection, transfer, demotion, layoff, termination, compensation, benefits, and other terms and conditions of employment. The policy of equal opportunity applies without regard to race, color, creed, religion, gender, sexual orientation, gender identification, pregnancy, marital status, national origin, ancestry, age, disability that can reasonably be accommodated without undue hardship, military status, veteran status, genetic predisposition or carrier status, alienage or citizenship, domestic partnership status, arrest or conviction record, status as a victim of domestic violence, or any other protected categories under federal, state, or local law. The Company also prohibits discrimination or harassment based upon the perception that a person has, or is associated with a person who has, any of these characteristics.\n\nCandidate Data Privacy and Use:\nTechstra Solutions respects the privacy of our candidates. By submitting your application, you acknowledge that Techstra Solutions may collect, store, and process the information you provide for recruiting, candidate evaluation, and consideration for current or future opportunities. Techstra may use applicant tracking systems, analytics, automation, and AI-assisted tools to support these activities. Candidate information will be protected using appropriate safeguards and retained or deleted in accordance with applicable law and Techstra’s data retention practices.\neAgbobrNjR","company":"Techstra Solutions","rawCompany":"techstra solutions","city":"Pittsburgh","state":"PA","isRemote":false,"isActive":false,"createdAt":"2026-11-05T09:40:44.939Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Senior Databricks Engineer","description":"Senior Databricks Engineer / Technical Lead\n\nWe are seeking a Senior Databricks Engineer / Technical Lead with 10+ years of experience in data engineering, data warehousing, and ETL/ELT, including significant hands-on experience delivering modern cloud data solutions.\n\nThis role is approximately equal parts technical leadership, hands-on development, and solution architecture. The ideal candidate can define the technical direction for a Databricks implementation, guide and mentor engineering teams, and personally develop and troubleshoot complex data solutions when needed.\n\nThe individual will work closely with business stakeholders, data architects, analysts, and engineering teams to translate business requirements into scalable, secure, and maintainable data solutions.\n\nKey Responsibilities\nLeadership & Delivery\nProvide technical leadership for Databricks and broader data engineering initiatives.\nLead and mentor data engineers through design, development, testing, deployment, and production support.\nBreak architectural designs and business requirements into actionable engineering work.\nEstablish development standards, patterns, and reusable frameworks for the engineering team.\nLead code and design reviews and ensure solutions meet performance, quality, security, and maintainability standards.\nCoordinate technical delivery across engineering, architecture, analytics, governance, and business teams.\nIdentify technical risks, dependencies, and delivery challenges and drive them to resolution.\nMentor junior and mid-level engineers while remaining actively engaged in delivery.\nHands-On Development\nDesign and develop scalable data pipelines using Databricks, Apache Spark, PySpark, Python, and SQL.\nBuild ingestion, transformation, and publishing processes across Bronze, Silver, and Gold layers.\nDevelop and optimize Delta Lake tables and large-scale Spark workloads.\nBuild batch and streaming solutions using capabilities such as Databricks Workflows, Auto Loader, Structured Streaming, and Lakeflow Declarative Pipelines where appropriate.\nDevelop reusable data engineering frameworks for ingestion, transformation, validation, monitoring, and exception handling.\nImplement data quality controls, reconciliation processes, logging, and operational monitoring.\nTroubleshoot complex data, performance, integration, and production issues.\nEstablish automated testing and CI/CD practices for Databricks solutions.\nArchitecture & Design\nDesign scalable lakehouse and data platform architectures using Databricks and cloud-native technologies.\nDefine appropriate patterns for data ingestion, transformation, storage, consumption, and integration.\nApply Medallion Architecture and other modern data engineering patterns based on solution requirements.\nDesign data models and processing patterns that support analytics, reporting, data science, and AI use cases.\nDefine strategies for performance, scalability, resiliency, security, and operational support.\nEstablish data governance and access patterns using Unity Catalog, including permissions, metadata, lineage, and data organization.\nEvaluate technical options and make pragmatic architecture decisions based on business requirements, cost, complexity, and maintainability.\nEnsure solutions integrate effectively with existing enterprise data warehouses, data lakes, databases, APIs, and downstream applications.\nRequired Qualifications\n10+ years of experience in data engineering, data warehousing, ETL/ELT, business intelligence, or enterprise data platforms.\nSignificant hands-on experience designing and developing solutions using Databricks.\nDemonstrated experience leading data engineering teams and providing technical direction.\nStrong experience with:\nDatabricks\nApache Spark\nPySpark / Python\nSQL\nDelta Lake\nData warehousing and dimensional modeling\nETL/ELT architecture and development\nStrong understanding of lakehouse, data lake, and enterprise data warehouse architectures.\nExperience designing and implementing Medallion Architecture patterns.\nExperience with Unity Catalog and enterprise data governance concepts.\nExperience with at least one major cloud platform, preferably Microsoft Azure, AWS, or Google Cloud.\nExperience implementing source control, automated testing, and CI/CD for data engineering solutions.\nStrong understanding of data quality, metadata, lineage, security, access management, and governance.\nAbility to translate business requirements into technical architecture and executable engineering plans.\nStrong communication skills with the ability to work with both engineering teams and senior business/technology stakeholders.\nPreferred Qualifications\nExperience with Azure Data Lake Storage, Azure Data Factory, Azure DevOps, and related Azure services.\nExperience with Databricks Asset Bundles and automated deployment approaches.\nExperience modernizing or migrating traditional ETL/data warehouse platforms to Databricks.\nExperience integrating Databricks with analytics platforms such as Power BI.\nExperience supporting data science, machine learning, GenAI, or advanced analytics workloads.\nDatabricks professional-level certification or equivalent demonstrated expertise.\nWhat We're Looking For\n\nThis is not a pure architect, engineering manager, or individual contributor role.\n\nThe successful candidate should be equally comfortable:\n\nLeading\nProviding technical direction, mentoring engineers, managing technical delivery, conducting reviews, and communicating with stakeholders.\n\nBuilding\nWriting PySpark, Python, and SQL; building pipelines; optimizing workloads; troubleshooting difficult problems; and producing production-quality solutions.\n\nArchitecting\nDesigning the overall solution, selecting appropriate patterns and technologies, defining integration and governance approaches, and ensuring the platform can scale.\n\nA strong candidate should be able to whiteboard the architecture in the morning, build or review the critical pipeline in the afternoon, and lead the engineering team through delivery.\n\nEqual Employment Opportunity Statement\n\nTechstra Solutions is an equal opportunity employer. The Company makes its decisions on merit, and its policy of equal opportunity prohibits discrimination in all phases of the employment process, including, but not limited to, recruitment, hiring, promotion, selection, transfer, demotion, layoff, termination, compensation, benefits, and other terms and conditions of employment. The policy of equal opportunity applies without regard to race, color, creed, religion, gender, sexual orientation, gender identification, pregnancy, marital status, national origin, ancestry, age, disability that can reasonably be accommodated without undue hardship, military status, veteran status, genetic predisposition or carrier status, alienage or citizenship, domestic partnership status, arrest or conviction record, status as a victim of domestic violence, or any other protected categories under federal, state, or local law. The Company also prohibits discrimination or harassment based upon the perception that a person has, or is associated with a person who has, any of these characteristics.\n\nCandidate Data Privacy and Use:\nTechstra Solutions respects the privacy of our candidates. By submitting your application, you acknowledge that Techstra Solutions may collect, store, and process the information you provide for recruiting, candidate evaluation, and consideration for current or future opportunities. Techstra may use applicant tracking systems, analytics, automation, and AI-assisted tools to support these activities. Candidate information will be protected using appropriate safeguards and retained or deleted in accordance with applicable law and Techstra’s data retention practices.\neAgbobrNjR","datePosted":"2026-11-05T09:40:44.939Z","dateModified":"2026-11-05T09:40:44.939Z","hiringOrganization":{"@type":"Organization","name":"Techstra Solutions","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Pittsburgh","addressRegion":"PA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"07381c67ae2e4b7e2e41e639"},"url":"https://jobsearcher.com/jobs/07381c67ae2e4b7e2e41e639"}}