Lead Data Engineer / Data Architect (Local to San Jose, CA)
Job Title: Lead Data Engineer / Data ArchitectLocation: San Jose, CA (ONSITE – LOCAL CANDIDATES ONLY - F2F)Duration: 12+ MonthsEmployment Type: ContractExperience Required: 8–10 YearsJob SummaryWe are seeking an experienced Lead Data Engineer / Data Architect to design, scale, and optimize enterprise-grade Lakehouse data platforms. The ideal candidate will have deep expertise in Databricks, Apache Spark, Delta Lake, cloud data platforms, and enterprise data architecture, with proven experience building production-grade scalable data ecosystems.This role requires a strong technical leader who can drive architecture decisions, establish data engineering standards, optimize platform performance and cost, and mentor engineering teams.Only local San Jose, CA candidates available for onsite work should be considered.Required Experience8+ years of Data Engineering / Data Platform Engineering experienceProven experience in enterprise data architecture and large-scale data platform designStrong leadership experience guiding engineering teams and architecture decisionsHands-on experience building production-grade data platformsMust Have Technical SkillsDatabricks / Big Data EngineeringStrong Hands-on Expertise WithDatabricksApache SparkPySparkScalaDelta LakeLakehouse ArchitectureDistributed data processingLarge-scale ETL / ELT pipelinesData ArchitectureStrong Experience InEnterprise data platform architectureScalable data pipeline designData platform modernizationLakehouse implementationPerformance optimizationCost optimizationProduction data platform engineeringData ModelingStrong Understanding OfMedallion ArchitectureBronze LayerSilver LayerGold LayerData warehousing conceptsDimensional modelingData governance principlesMetadata-driven architectureCloud PlatformsHands-on Experience With One Or MoreAWSAzureGoogle Cloud PlatformDatabase / Query OptimizationStrong Expertise InSQLQuery optimizationLarge-scale data transformationsData performance tuningPartitioning / indexing strategiesCompute optimizationEngineering / DevOpsExperience WithCI/CD pipelines for data engineeringVersion control (Git)Deployment automationMonitoring / observabilityProduction support best practicesKey ResponsibilitiesDesign and lead enterprise Lakehouse architecture implementation using DatabricksBuild scalable, secure, and production-ready data platformsArchitect and optimize large-scale data pipelines and transformation frameworksDrive engineering standards, architecture governance, and best practicesOptimize Spark / Databricks performance, scalability, and cost efficiencyDesign and implement Delta Lake-based data architecturesDefine and enforce Medallion architecture data modeling standardsCollaborate with business, analytics, engineering, and architecture stakeholdersMentor and guide data engineering teamsSupport platform modernization and cloud data transformation initiativesTroubleshoot production issues and optimize platform reliabilityPreferred QualificationsExperience in enterprise-scale data modernization programsArchitecture leadership experienceExposure to data governance and security frameworksExperience with streaming data platforms is a plus