{"schemaVersion":"jobsearcher.job.v1","id":"626dc42617a8d9ece1d09a05","url":"https://jobsearcher.com/jobs/626dc42617a8d9ece1d09a05","canonicalUrl":"https://jobsearcher.com/jobs/626dc42617a8d9ece1d09a05","title":"Big Data Engineer with Python","description":"Position: Hadoop Hive Python DeveloperLocation: Charlotte, NC (Onsite, Hybrid Mode) - LocalsJob Type: FulltimeMust Have Technical/Functional SkillsPrimary skills: Hadoop, Hive, Python, PySpark, Apache Kafka, Hadoop Ecosystem, Hive, Databricks Lakehouse Architecture, Delta Lake, Bronze/Silver/Gold Data Modeling, Big Data ETL Pipeline Development, SQL, Real-time Data Ingestion Frameworks, Data Governance & Cataloging, CI/CD Tools – Git, Jenkins, Bitbucket, Workflow Orchestration, and Cloud & On-Prem Big Data Platforms.Experience: Minimum 9+ yearsRoles & ResponsibilitiesSeeking a Senior Big Data Engineer with 9-14 years of experience specializing in Hadoop, Python, Hive PySpark, Kafka, and strong experience designing data solutions for large-scale financial systems.In addition, the candidate must possess advanced expertise in Databricks Lakehouse architecture, particularly around Bronze/Silver/Gold layer data modeling, Delta Lake optimizations, and building reliable, scalable pipelines for regulatory, risk, trading, and analytics workloads.This role focuses on delivering highly performant, well-governed data platforms that support the bank’s mission-critical global markets functions.Key Responsibilities:Big Data Platform EngineeringDesign, develop, and optimize PySpark-based ETL pipelines running on on‑prem Hadoop clusters and cloud environments.Build high‑volume ingestion frameworks using Kafka for real-time and near-real-time trading and market data. Develop, tune, and manage Hadoop ecosystem components—HDFS, YARN, MapReduce, Tez, Oozie/Airflow. Build high-performance, optimized Hive data models for regulatory reporting, trade lifecycle, and market risk processing. Databricks Lakehouse & Delta Framework Architect and implement Bronze/Silver/Gold layer modeling patterns within the Databricks Lakehouse. Apply Delta Lake best practices including: optimized file management Z-Ordering Delta Change Data Feed (CDF) schema evolution & enforcement ACID transaction handling Build reusable frameworks for ingestion, cleansing, transformation, and consumption of data across Lakehouse layers. Enable governance, lineage, and auditability using Unity Catalog or equivalent cataloging tools. Collaboration, Leadership & Delivery Collaborate closely with quants, product owners, architects, risk tech, and business users. Participate in agile ceremonies — sprint planning, refinement, design reviews. Mentor junior engineers and contribute to building strong engineering practices across tech teams. Required Skills & Experience 9-14 years of hands-on experience in Big Data engineering. · Expert skills in: PySpark — dataframe optimizations, partitioning, broadcast strategies, distributed computing. Kafka — producer/consumer design, schema registry, streaming ETLs. Hadoop ecosystem — HDFS, YARN, MapReduce/Tez, Oozie/Airflow. Hive — advanced query tuning, TEZ optimization, partition/bucket management. Extensive hands-on experience with Databricks Lakehouse, including: Bronze/Silver/Gold layer modeling Delta Lake optimizations Data quality frameworks on Lakehouse Structured & unstructured data handling Experience in Global Markets, Risk, Treasury, Trade Surveillance, or Regulatory Reporting. Strong SQL knowledge with experience working on massive datasets (TB/PB scale). Experience with CI/CD practices — Git, Jenkins, Bitbucket, build pipelines.\"Centraprise is an equal opportunity employer. Applicants must be authorized to work in the U.S. U.S. citizens and Green Card holders are strongly encouraged to apply.\"","company":"Centraprise","rawCompany":"centraprise","city":"Charlotte","state":"NC","isRemote":false,"isActive":false,"createdAt":"2026-08-31T08:46:40.144Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Big Data Engineer with Python","description":"Position: Hadoop Hive Python DeveloperLocation: Charlotte, NC (Onsite, Hybrid Mode) - LocalsJob Type: FulltimeMust Have Technical/Functional SkillsPrimary skills: Hadoop, Hive, Python, PySpark, Apache Kafka, Hadoop Ecosystem, Hive, Databricks Lakehouse Architecture, Delta Lake, Bronze/Silver/Gold Data Modeling, Big Data ETL Pipeline Development, SQL, Real-time Data Ingestion Frameworks, Data Governance & Cataloging, CI/CD Tools – Git, Jenkins, Bitbucket, Workflow Orchestration, and Cloud & On-Prem Big Data Platforms.Experience: Minimum 9+ yearsRoles & ResponsibilitiesSeeking a Senior Big Data Engineer with 9-14 years of experience specializing in Hadoop, Python, Hive PySpark, Kafka, and strong experience designing data solutions for large-scale financial systems.In addition, the candidate must possess advanced expertise in Databricks Lakehouse architecture, particularly around Bronze/Silver/Gold layer data modeling, Delta Lake optimizations, and building reliable, scalable pipelines for regulatory, risk, trading, and analytics workloads.This role focuses on delivering highly performant, well-governed data platforms that support the bank’s mission-critical global markets functions.Key Responsibilities:Big Data Platform EngineeringDesign, develop, and optimize PySpark-based ETL pipelines running on on‑prem Hadoop clusters and cloud environments.Build high‑volume ingestion frameworks using Kafka for real-time and near-real-time trading and market data. Develop, tune, and manage Hadoop ecosystem components—HDFS, YARN, MapReduce, Tez, Oozie/Airflow. Build high-performance, optimized Hive data models for regulatory reporting, trade lifecycle, and market risk processing. Databricks Lakehouse & Delta Framework Architect and implement Bronze/Silver/Gold layer modeling patterns within the Databricks Lakehouse. Apply Delta Lake best practices including: optimized file management Z-Ordering Delta Change Data Feed (CDF) schema evolution & enforcement ACID transaction handling Build reusable frameworks for ingestion, cleansing, transformation, and consumption of data across Lakehouse layers. Enable governance, lineage, and auditability using Unity Catalog or equivalent cataloging tools. Collaboration, Leadership & Delivery Collaborate closely with quants, product owners, architects, risk tech, and business users. Participate in agile ceremonies — sprint planning, refinement, design reviews. Mentor junior engineers and contribute to building strong engineering practices across tech teams. Required Skills & Experience 9-14 years of hands-on experience in Big Data engineering. · Expert skills in: PySpark — dataframe optimizations, partitioning, broadcast strategies, distributed computing. Kafka — producer/consumer design, schema registry, streaming ETLs. Hadoop ecosystem — HDFS, YARN, MapReduce/Tez, Oozie/Airflow. Hive — advanced query tuning, TEZ optimization, partition/bucket management. Extensive hands-on experience with Databricks Lakehouse, including: Bronze/Silver/Gold layer modeling Delta Lake optimizations Data quality frameworks on Lakehouse Structured & unstructured data handling Experience in Global Markets, Risk, Treasury, Trade Surveillance, or Regulatory Reporting. Strong SQL knowledge with experience working on massive datasets (TB/PB scale). Experience with CI/CD practices — Git, Jenkins, Bitbucket, build pipelines.\"Centraprise is an equal opportunity employer. Applicants must be authorized to work in the U.S. U.S. citizens and Green Card holders are strongly encouraged to apply.\"","datePosted":"2026-08-31T08:46:40.144Z","dateModified":"2026-08-31T08:46:40.144Z","hiringOrganization":{"@type":"Organization","name":"Centraprise","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Charlotte","addressRegion":"NC","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"626dc42617a8d9ece1d09a05"},"url":"https://jobsearcher.com/jobs/626dc42617a8d9ece1d09a05"}}