Lead Data Engineer with Databricks

Lead Data Engineer with Databricks HybridCharlotte, NCRoles and ResponsibilitiesDevelop high-quality, scalable ETL/ELT pipelines using Databricks technologies including Delta Lake, Auto Loader etc.Excellent programming and debugging skills in PythonStrong hands-on experience with PySpark to build efficient data transformation and validation logicMust be proficient in at least one cloud platform: AWS, GCP, or AzureNeed someone with Insurance domain experience - property and casualtyImplement ingestion patterns using Auto Loader with checkpointing and schema evolution for structured and semi-structured dataBuild secure and observable DLT pipelines with DLT Expectations, supporting Bronze/Silver/Gold medallion layeringConfigure Unity Catalog: set up catalogs, schemas, user/group access, enable audit logging, & define masking for PII fieldsEnable secure data access across domains & workspaces via Unity Catalog External Locations, Volumes, & lineage trackingAccess and utilize data assets from the Databricks Marketplace to support enrichment, model training, or benchmarking