Databricks Engineer

Databricks EngineerVery strong Databricks and Healthcare experienceData transfer and ETL tools: Python, PySpark , Databricks, SQL, and relevant libraries for large-scale data processingFamiliarity with networking concepts, data storage interfaces, and scoping ETL requirementsStrong SQL proficiency (SQL flavor largely unimportant)Understanding of underlying data models and optimization strategies (partitioning, indexing, query performance)Ability to create and maintain ETL pipelines and workflows in Databricks; ensure source data is received on schedule and transformed data is available for Power BI and Tableau consumptionExperience with API ingestion , including connecting to external APIs, extracting data, and testing/validating for completeness and accuracyExperience with Delta Sharing (Deltashare) for secure, real-time sharing of data across systems and teamsFamiliarity with AWS cloud services , including S3 for data storage and CloudWatch for monitoring ETL pipelines and workflowsGood understanding of system Entities and Attributes, with ability to implement structural changes in the Data Model as requirements evolveKnowledge of data replication and Change Data Capture (CDC) processes, including tools like QlikDiverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.