Data Engineer

A US commercial bank built its data engineering function from scratch four years ago — no team, no platform, and fifteen years of stacked-up business use cases waiting to be built. The team was hired, the stack was chosen and stood up, and the first major deliverables shipped: real-time fraud integration processes, followed by a Customer 360 platform that consolidates customer identity and account data across systems inherited from a long history of building, integrating and selling technology companies.The next phase is underway now, and it is urgent. Several new source systems are being added to the customer view, with identity resolution across those sources, to create refined, fit-for-purpose "gold / gold-plus" data. That data is the backbone for a 2027 programme to use AI across customer information and customer-facing product features.This is a hands-on build seat on a small, senior team. You will not be trained up — the expectation is that you have done this work before and can contribute from week one.What You'll Be Doing•      Build and deploy new data ingestions into Databricks from new and existing source systems•      Contribute to fraud detection implementations already in production•      Integrate new source data into the Customer 360 platform, including cross-system identity mapping•      Build transformations and data models to a gold / gold-plus standard suitable for downstream AI use•      Ship production-grade application code and infrastructure — not notebook-bound workLayerThe Tech StackCloudAWSLakehouseDatabricksOrchestrationAirflowTransformationdbtData qualityMonte CarloSaaS ingestionFivetranSource controlGitLabInfrastructure as codeTerraformLanguagesPython, PySpark / Spark, SQLEssential Skills•      5+ years building ingestions, transformations and data models in Python on Databricks•      Strong, demonstrable PySpark / Spark experience — including a real understanding of how PySpark works under the hood and when you would use it over the alternatives•      Proven experience building data ingestions via infrastructure as code with Terraform•      Experience deploying application code to production. Candidates whose Databricks experience is entirely inside Databricks notebooks will not be considered•      Cloud data lakehouse experience — AWS preferred; Azure or GCP experience transfers•      Working SQL competence•      Able to work Eastern time hoursAdvantageous Skills•      Financial services or banking background. A regulated-industry background is not required at all — if you have built real pipelines with Python, PySpark and Terraform on Databricks, the industry you did it in matters far less•      Airflow, dbt, Monte Carlo, Fivetran, GitLab CI