Databricks Engineer

🚨 URGENT HIRING | Share your profile: shyam@npoleinc.com | +1 512-999-7875 / +91 8309049094 (Employers)Job Title: Databricks Engineer (AWS – Large Scale Data Migration)Location: Plano, TX – 5 Days Onsite RoleLong Term Project15+ Years - Local Candidates OnlyJob SummaryWe are seeking a highly skilled Databricks Engineer with strong expertise in large-scale data migrations (petabyte-scale) and deep experience in the AWS ecosystem. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and migrating enterprise data platforms to Databricks Lakehouse architecture.Required Skills & QualificationsExperienced Data Engineering / Big DataJava is mustStrong hands-on experience with Databricks (mandatory)Expertise in Apache Spark (PySpark/Scala)Proven experience in large-scale data migration (petabyte-level preferred)Strong knowledge of AWS cloud services (S3, Glue, Lambda, Redshift, Kinesis, etc.)Experience with data lake architectures and Delta LakeSolid understanding of ETL/ELT frameworks and data modelingExperience with SQL, Python, or ScalaKey ResponsibilitiesData Migration & EngineeringLead and execute end-to-end data migration from legacy/on-prem/cloud systems to Databricks (Lakehouse platform)Handle petabyte-scale data migration, ensuring data integrity, performance, and minimal downtimeDesign and implement scalable ETL/ELT pipelines using Databricks, Spark, and Delta LakePerform data validation, reconciliation, and optimization post-migrationDatabricks DevelopmentDevelop and optimize PySpark/Scala-based data processing workflowsImplement Delta Lake architecture including partitioning, indexing, and performance tuningBuild reusable frameworks for ingestion, transformation, and data quality checksWork with Databricks Workflows, Jobs, and Notebooks