Data Engineering Lead
Builders is committed to reliability, fairness, and customer-first service, earning an A rating by AM Best and recognition through Top Workplaces “Culture Excellence” honors. This role supports an insurance business built on strong partnerships and mutual respect, with an onsite opportunity in Atlanta, GA to lead a platform that helps teams modernize and govern data for property and casualty insurance.
As the technical owner of the Azure Databricks lakehouse platform, you will drive architecture, modernization, governance alignment, and engineering leadership to deliver a scalable, secure, reliable, high-performing foundation for property and casualty data.
What you’ll do:
Design theDatabricks lakehouse architectureusing abronze/silver/goldapproach onDelta Lakefor policy, rating, claims, billing, reinsurance, and customer data.
Plan and lead phased migration fromGuidewire(PolicyCenter, ClaimCenter, BillingCenter) and legacy systems into Azure Databricks, includinghistorical loads , incremental/CDCloads, reconciliation, anddecommissioning support.
Model core property and casualty insurance entities:accounts , policies , insured risks and exposures , coverages , endorsements , premiums , claims , reserves , payments, andrecoveries.
Provide data forActuarial and Financeuse cases includingwritten/earned/unearned premium , loss triangles , IBNR(incurred but not reported) inputs, loss ratios, andSchedule P and NAICstatutory reporting.
Integratethird-partysources such asgeospatialandweatherdata.
Define and operationalizeUnity Catalogdesign:metastore , catalogs , schemas , development/test/production environments, naming conventions, permissions , row/column-level security, andlineage.
Managepipeline orchestration and operationsin Databricks, includingDatabricks Workflows(Lakeflow Jobs), scheduling, dependencies, retries , alerting and SLA monitoring, and productionfailure handlingwithrunbooks.
Ownperformance and costwork: Spark tuning , partitioning , liquid clustering , Photon, andcluster/SQL warehouse sizing.
Partner with IT and a Platform Engineer on infrastructure forstorage , networking , security , identity , IaC, andCI/CD.
Set standards for pipelines, testing , code review, anddocumentation, and lead mentoring and review for Data Engineers and junior contractors.
Collaborate with thedata governance leadto implement access controls, PII masking , lineage, anddata quality rules.
Perform other duties as assigned.
What you bringAssociate degreeor an equivalent combination of education and experience.
10+ yearsof data engineering experience, including 6+ years hands-onwithDatabricks(PySpark, Spark SQL, Delta Lake), plus experience leading or mentoring engineering teams.
Databricks Data Engineer Professionalcertification; Azure data or solutions architectcertification is preferred.
Strong knowledge ofUnity Catalog : governance model, permissions, row/column-level security, lineage, and system tables.
Experience leadinglarge data migrationsfrom core insurance and legacy systems into a lakehouse or data warehouse, with property and casualty domain knowledge and familiarity with the policy and claims lifecycle and core P&C KPIs.
Knowledge ofdata modelingincluding dimensional modeling, data vault, SCD patterns, and insurance data models.
Expert technical skills acrossSparkinternals and performance tuning, streaming(structured streaming, Auto Loader), CDCpatterns, MERGE, andLakeflow/Delta Live Tables.
Expert-levelSQLandPython, with software engineering practices such as testing, code review, and modular design.
Knowledge of data infrastructure: ADLS Gen2 , Azure Data Factory , Event Hubs , Key Vault , Entra ID , VNets, andPrivate Link.
Experience scheduling and monitoring production pipelines withDatabricks Workflowsand/orAzure Data Factory, including dependencies, parameterization, and monitoring with Databricks system tables.
Experience withGit -based version control (GitHub or Bitbucket), including branching strategies, pull request reviews, and Databricks Git folders.
Infrastructure-as-code and CI/CD for data usingTerraform , Databricks Asset Bundles , Azure DevOps , GitHub Actions, orBitbucket Pipelines.
Experience withGuidewire data(Guidewire DataHub/InfoCenter, Cloud Data Access) is preferred, including supporting actuarial reserving and pricing; API and integration tools such asMuleSoftare preferred.
Knowledge of insurance data standards such asACORDis preferred.
Benefits at BuildersCompetitive Salary
Bonus Structure
Profit Sharing
Medical, Dental, Vision Insurance
Employer Paid Short Term Disability
Employer Paid Long Term Disability
Employer Paid Life Insurance
Voluntary Life Insurance
401K with Company Match
PTO
Equal OpportunityBuilders Insurance Group is an equal opportunity employer. Applicants are welcomed from all walks of life, and the company does not discriminate based on any protected status. This is part of building a diverse and inclusive workplace.