{"schemaVersion":"jobsearcher.job.v1","id":"fa9d7aecb707d4f62d4f102d","url":"https://jobsearcher.com/jobs/fa9d7aecb707d4f62d4f102d","canonicalUrl":"https://jobsearcher.com/jobs/fa9d7aecb707d4f62d4f102d","title":"Data Bricks Migration and Support engineer","description":"Role: Data Bricks Migration and Support engineer\r\n8 - 15 years of experience\r\nJob Description\r\nMust Have Technical/Functional Skills\r\nSuccessfully executed a data migration or modernization to Data Bricks, preferably IBM Data Stage to Data Bricks on AWS\r\nShould have Experience in handling Large Migrations to Data Bricks.\r\nShould have good analytical skills to compare the legacy and modern data platform end to end right from source to target.\r\nGood understanding of DataBricks implementation of Medallion layer architecture.\r\nIndependently Lead and Managed large Data Bricks migrations.\r\nCI/CD Integration: Implement version control (e.g., Git) and automated deployment processes for Databricks assets\r\nTechnical and architectural skills required are below.\r\nCore Data Engineering Languages\r\nExperience in Advanced SQL for building modular analytics workflows, utilizing advanced Common Table Expressions (CTEs), and writing high-performance queries inside Data Bricks SQL Analytics.\r\nExperience in Python or Scala to build, optimize, and debug complex data transformation scripts, custom functions, and machine learning pipelines.\r\nBig Data & Architecture Core\r\nExperience in Apache Spark Ecosystem for understanding cluster execution flow, memory allocation, driver/worker nodes, and handling data frames.\r\nExperience in Delta Lake Architecture to understand ACID transactions on object storage, data skipping, partition strategies, and automated data compaction.\r\nDatabricks Platform Expertise\r\nExperience in Delta Live Tables (DLT) & Workflows for constructing and orchestrating production-ready, declarative streaming, and batch ETL pipelines.\r\nExperience in Unity Catalog for setting up data governance, column/row-level access control, and tracking end-to-end data lineage across workspaces.\r\nExperience in Auto Loader for implementing modern, incremental data ingestion patterns from cloud blob storage into the lakehouse.\r\nCode Translation & Refactoring\r\nPipeline Conversion: Translate visual DataStage Parallel Jobs and Sequences into Python/PySpark scripts or Data bricks Notebooks\r\nLegacy Refactoring: Modernize legacy logic rather than applying \"lift and shift\" anti-patterns; adapt workflows to think in distributed DataFrames rather than DataStage stages.\r\nLogic Mapping: Map DataStage components—such as Aggregators, Joiners, Transformers, and Sort stages—to equivalent Spark operations\r\nValidation & Reconciliation: Build automated reconciliation frameworks to compare row counts, checksums, and aggregate sums between legacy DataStage outputs and new Databricks output\r\nData Cleansing: Identify and resolve data type discrepancies, null-handling differences, and encoding issues during the extraction and loading phases\r\nPlatform Orc hestration & Governance\r\nOrchestration: Replace DataStage sequence jobs with Databricks workflows ( or external orchestrators like Azure Data Factory/Airflow) to schedule and manage dependencies\r\nData Governance: Enforce data lineage, security, and cataloging using Unity Catalog to ensure compliance in the new Lakehouse environment.\r\nCloud Providers (AWS): Understanding underlying cloud object storage , identity access management (IAM), and network security configurations.\r\nDevOps & Bundles: Familiarity with Databricks Asset Bundles (DABs) and CI/CD tools to automate the deployment of workspaces and pipeline assets.\r\nLegacy Assessment & Migration Mechanics\r\nCode Conversion & Translation: The ability to parse legacy code structures and refactor them into Databricks-native code.\r\nAI-Assisted Migration: Skills in using AI coding assistants and open framework agent tools to analyze application interdependencies, automate schema mapping, and accelerate lift-and-shift workloads\r\nCode Conversion & Translation: The ability to parse legacy code structures from ETL pipelines, Informatica, data Stage preferred\r\nExperience working in Agile teams and understanding of data governance frameworks.\r\nJ-18808-Ljbffr","company":"Envision Technology Solutions","rawCompany":"envision technology solutions","city":"Dallas","state":"TX","isRemote":false,"isActive":false,"createdAt":"2026-08-08T00:48:25.236Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Bricks Migration and Support engineer","description":"Role: Data Bricks Migration and Support engineer\r\n8 - 15 years of experience\r\nJob Description\r\nMust Have Technical/Functional Skills\r\nSuccessfully executed a data migration or modernization to Data Bricks, preferably IBM Data Stage to Data Bricks on AWS\r\nShould have Experience in handling Large Migrations to Data Bricks.\r\nShould have good analytical skills to compare the legacy and modern data platform end to end right from source to target.\r\nGood understanding of DataBricks implementation of Medallion layer architecture.\r\nIndependently Lead and Managed large Data Bricks migrations.\r\nCI/CD Integration: Implement version control (e.g., Git) and automated deployment processes for Databricks assets\r\nTechnical and architectural skills required are below.\r\nCore Data Engineering Languages\r\nExperience in Advanced SQL for building modular analytics workflows, utilizing advanced Common Table Expressions (CTEs), and writing high-performance queries inside Data Bricks SQL Analytics.\r\nExperience in Python or Scala to build, optimize, and debug complex data transformation scripts, custom functions, and machine learning pipelines.\r\nBig Data & Architecture Core\r\nExperience in Apache Spark Ecosystem for understanding cluster execution flow, memory allocation, driver/worker nodes, and handling data frames.\r\nExperience in Delta Lake Architecture to understand ACID transactions on object storage, data skipping, partition strategies, and automated data compaction.\r\nDatabricks Platform Expertise\r\nExperience in Delta Live Tables (DLT) & Workflows for constructing and orchestrating production-ready, declarative streaming, and batch ETL pipelines.\r\nExperience in Unity Catalog for setting up data governance, column/row-level access control, and tracking end-to-end data lineage across workspaces.\r\nExperience in Auto Loader for implementing modern, incremental data ingestion patterns from cloud blob storage into the lakehouse.\r\nCode Translation & Refactoring\r\nPipeline Conversion: Translate visual DataStage Parallel Jobs and Sequences into Python/PySpark scripts or Data bricks Notebooks\r\nLegacy Refactoring: Modernize legacy logic rather than applying \"lift and shift\" anti-patterns; adapt workflows to think in distributed DataFrames rather than DataStage stages.\r\nLogic Mapping: Map DataStage components—such as Aggregators, Joiners, Transformers, and Sort stages—to equivalent Spark operations\r\nValidation & Reconciliation: Build automated reconciliation frameworks to compare row counts, checksums, and aggregate sums between legacy DataStage outputs and new Databricks output\r\nData Cleansing: Identify and resolve data type discrepancies, null-handling differences, and encoding issues during the extraction and loading phases\r\nPlatform Orc hestration & Governance\r\nOrchestration: Replace DataStage sequence jobs with Databricks workflows ( or external orchestrators like Azure Data Factory/Airflow) to schedule and manage dependencies\r\nData Governance: Enforce data lineage, security, and cataloging using Unity Catalog to ensure compliance in the new Lakehouse environment.\r\nCloud Providers (AWS): Understanding underlying cloud object storage , identity access management (IAM), and network security configurations.\r\nDevOps & Bundles: Familiarity with Databricks Asset Bundles (DABs) and CI/CD tools to automate the deployment of workspaces and pipeline assets.\r\nLegacy Assessment & Migration Mechanics\r\nCode Conversion & Translation: The ability to parse legacy code structures and refactor them into Databricks-native code.\r\nAI-Assisted Migration: Skills in using AI coding assistants and open framework agent tools to analyze application interdependencies, automate schema mapping, and accelerate lift-and-shift workloads\r\nCode Conversion & Translation: The ability to parse legacy code structures from ETL pipelines, Informatica, data Stage preferred\r\nExperience working in Agile teams and understanding of data governance frameworks.\r\nJ-18808-Ljbffr","datePosted":"2026-08-08T00:48:25.236Z","dateModified":"2026-08-08T00:48:25.236Z","hiringOrganization":{"@type":"Organization","name":"Envision Technology Solutions","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Dallas","addressRegion":"TX","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"fa9d7aecb707d4f62d4f102d"},"url":"https://jobsearcher.com/jobs/fa9d7aecb707d4f62d4f102d"}}