Data Platform Engineer
Data Platform Engineer – Enterprise Data Modernization & AssessmentPosition OverviewWe are seeking an experienced Data Platform Engineer to support the first phase of a large-scale enterprise data modernization initiative for a leading academic medical center. This consultant will play a key role in assessing how Epic Clarity and Caboodle data is currently accessed, consumed, and utilized across the organization while helping define the future-state architecture within Databricks.Unlike a traditional data engineering role focused on building production pipelines, this position centers on technical discovery, workload analysis, automation, and migration planning. The engineer will evaluate existing database objects, query activity, and reporting workloads to determine how they should be represented within a modern Databricks-based analytics platform.A core responsibility of this engagement is designing and operating an AI-assisted query classification process that analyzes database activity, categorizes access patterns, and helps identify which workloads should be recreated in Databricks. This individual will also own the mapping of existing Clarity and Caboodle database objects to future-state Databricks capabilities while identifying functional gaps that will inform the organization's migration roadmap.The ideal candidate has successfully participated in multiple enterprise data modernization initiatives, possesses strong automation and analytical skills, and enjoys solving complex architectural challenges through scalable, thoughtful solutions.Key ResponsibilitiesDesign, build, and maintain an automated query-log classification pipeline utilizing AI-assisted techniques to categorize enterprise database access patterns.Analyze SQL query activity, reporting workloads, extracts, integrations, and downstream data consumption to identify migration candidates.Own the mapping of Epic Clarity and Caboodle database objects to future-state Databricks architecture.Perform technical assessments of tables, views, stored procedures, functions, and other database objects to determine migration feasibility and modernization strategies.Identify coverage gaps between existing database functionality and the target-state Databricks platform while developing practical recommendations for future implementation.Develop automation that accelerates discovery, object analysis, metadata collection, workload classification, and migration planning.Collaborate with Identity & Access Management resources to correlate access patterns with database object utilization and support enterprise access rationalization.Partner with business stakeholders to understand analytical workflows, reporting requirements, and downstream application dependencies.Evaluate legacy SQL objects and recommend scalable cloud-native design patterns within Databricks.Produce technical documentation, migration recommendations, object inventories, and architectural assessments that will guide future implementation phases.Provide engineering expertise to support enterprise data governance, modernization strategy, and analytics platform transformation.Required Qualifications5+ years of experience as a Data Platform Engineer, Data Engineer, Analytics Engineer, Data Architect, or similar role.Epic Clarity Data Model Certification and/or Epic Caboodle CertificationDemonstrated experience supporting multiple enterprise-scale data platform modernization or migration initiatives.Hands-on experience with Databricks and modern lakehouse architecture.Strong SQL development skills with experience analyzing complex database workloads and query patterns.Proven experience building automation using Python, PySpark, SQL, or similar technologies to improve operational efficiency and reduce manual analysis.Experience evaluating legacy data platforms and developing migration strategies for cloud-based analytics environments.Strong understanding of relational database architecture, metadata, database objects, and enterprise data modeling.Experience assessing existing reporting or analytics environments and translating legacy database structures into modern cloud architectures.Strong analytical, troubleshooting, and critical thinking skills with the ability to solve complex technical challenges independently.Excellent written and verbal communication skills with the ability to explain technical findings to both technical and non-technical stakeholders.Additional QualificationsExperience working with Epic Clarity and/or Epic Caboodle data environments.Experience supporting healthcare organizations or academic medical centers.Experience designing AI-assisted automation or machine learning workflows for data engineering, metadata discovery, or workload classification.Experience with Azure cloud services and enterprise analytics platforms.Familiarity with Power BI, Tableau, and modern enterprise reporting ecosystems.Experience supporting Oracle-to-cloud modernization initiatives.Knowledge of enterprise data governance, metadata management, and data cataloging practices.Ideal CandidateThe ideal candidate has successfully delivered multiple enterprise data modernization initiatives and understands that successful migrations begin with understanding—not moving—data. They bring a strong architectural mindset and have experience evaluating legacy environments before implementation begins.They have built automation to simplify complex engineering tasks, can leverage AI to accelerate technical analysis, and are comfortable working through ambiguity to develop practical, scalable recommendations. Rather than simply building pipelines, they enjoy investigating how enterprise data is consumed, identifying modernization opportunities, and designing future-state solutions that improve governance, scalability, and long-term maintainability.Technical EnvironmentEpic ClarityEpic CaboodleDatabricks Lakehouse PlatformSQL ServerOracle (Legacy Environment)Databricks SQLPython / PySparkAzureAI-Assisted Query ClassificationQuery Log AnalysisMetadata DiscoveryDatabase Object MappingEnterprise Data Governance