{"schemaVersion":"jobsearcher.job.v1","id":"52fcd77e92da7ed382e93452","url":"https://jobsearcher.com/jobs/52fcd77e92da7ed382e93452","canonicalUrl":"https://jobsearcher.com/jobs/52fcd77e92da7ed382e93452","title":"Data engineer(AEP)//REMOTE","description":"AXP Data EngineerRemoteINDEPENDENT VISA-USC, GC, H4EADRole SummaryThe AXP Data Engineer is a hands-on data practitioner responsible for designing, building, and operating scalable data pipelines that move, transform, and serve data across the Business–AXP integration program. This role works primarily within Databricks and the broader lakehouse architecture to ingest exposure events, AEP behavioral signals, AJO journey outcomes, and Business recommendation data, making them available for decisioning, analytics, and reporting. The Data Engineer partners closely with the Data Architect, CJA Architect, and AXP platform teams to implement governed, observable, and performant data pipelines aligned to program SLAs.Key ResponsibilitiesDesign and build batch and streaming data pipelines in Databricks (Delta Live Tables, Structured Streaming, and Databricks Workflows) to ingest, transform, and serve data from AEP, AJO, Kafka, and Business Platform sourcesDevelop and maintain Delta Lake tables, applying medallion architecture (Bronze → Silver → Gold) patterns for raw ingestion, data cleansing, enrichment, and aggregation layersImplement Kafka consumers within Databricks to process real-time AXP exposure events (Sent, Delivered, Clicked, Opened, Disposition) and Business recommendation signalsIntegrate with AEP datasets and Data Distiller to extract, query, and transform profile attributes, segment membership, and behavioral event data for downstream consumptionBuild and maintain data models that support CJA reporting, Business State Machine inputs, and ML feature engineering use casesEnforce data quality checks, schema validation, and reconciliation logic across pipeline stages to ensure accuracy, completeness, and consistencyOptimize Databricks pipeline performance: cluster configuration, auto-scaling, partitioning, caching, Z-ordering, and query plan tuning for large-scale event datasetsMaintain schema registry alignment and XDM-compatible data structures across all pipeline outputsSupport data governance standards: lineage documentation, metadata cataloging (Unity Catalog), access controls, and PII handling policiesMonitor pipeline health via structured logging, alerting, and SLA dashboards; triage and resolve data incidents in productionCollaborate with the Data Architect, CJA Architect, AXP Architect, and BI/Analytics teams to align pipeline designs with reporting and analytics requirementsCore SkillsDatabricks – Delta Live Tables, Structured Streaming, Databricks Workflows, cluster management, Unity CatalogDelta Lake / Lakehouse architecture – medallion design patterns, ACID transactions, time travel, schema evolutionPySpark and/or Scala Spark – large-scale data transformation, aggregations, windowing, and joinsSQL – complex query authoring, performance tuning, and Data Distiller query patterns for AEP datasetsKafka – real-time event consumption, offset management, schema-on-read patternsAEP / Data Distiller – dataset querying, XDM schema familiarity, profile and event dataset consumptionCloud data platform experience (AWS S3/Glue, Azure ADLS/Synapse, or GCP GCS/BigQuery)Data quality and observability tooling (Great Expectations, Monte Carlo, or equivalent)CI/CD for data pipelines: version control (Git), automated testing, and deployment automationUnderstanding of data governance principles: lineage, cataloging, access control, and PII/data privacyTypical Experience10+ years of data engineering experience, with significant hands-on Databricks delivery in production environmentsProven experience building and operating both batch and streaming pipelines at scale using Delta Lake and SparkExperience integrating with Kafka or other real-time event-streaming platforms as a consumerFamiliarity with AEP, Data Distiller, or Adobe analytics data models preferredExperience with lakehouse or cloud data warehouse architectures (Delta Lake, Snowflake, BigQuery, or Redshift)Exposure to MarTech, CDP, or personalization platform data flows (AJO, AEP, or equivalent) is a strong plusThanks & RegardsShyam (SAM)Sr.RecruiterEmail: sam.s@navasoftware.com","company":"Nava Software Solutions","rawCompany":"nava software solutions","isRemote":true,"isActive":false,"createdAt":"2026-07-04T09:19:39.482Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data engineer(AEP)//REMOTE","description":"AXP Data EngineerRemoteINDEPENDENT VISA-USC, GC, H4EADRole SummaryThe AXP Data Engineer is a hands-on data practitioner responsible for designing, building, and operating scalable data pipelines that move, transform, and serve data across the Business–AXP integration program. This role works primarily within Databricks and the broader lakehouse architecture to ingest exposure events, AEP behavioral signals, AJO journey outcomes, and Business recommendation data, making them available for decisioning, analytics, and reporting. The Data Engineer partners closely with the Data Architect, CJA Architect, and AXP platform teams to implement governed, observable, and performant data pipelines aligned to program SLAs.Key ResponsibilitiesDesign and build batch and streaming data pipelines in Databricks (Delta Live Tables, Structured Streaming, and Databricks Workflows) to ingest, transform, and serve data from AEP, AJO, Kafka, and Business Platform sourcesDevelop and maintain Delta Lake tables, applying medallion architecture (Bronze → Silver → Gold) patterns for raw ingestion, data cleansing, enrichment, and aggregation layersImplement Kafka consumers within Databricks to process real-time AXP exposure events (Sent, Delivered, Clicked, Opened, Disposition) and Business recommendation signalsIntegrate with AEP datasets and Data Distiller to extract, query, and transform profile attributes, segment membership, and behavioral event data for downstream consumptionBuild and maintain data models that support CJA reporting, Business State Machine inputs, and ML feature engineering use casesEnforce data quality checks, schema validation, and reconciliation logic across pipeline stages to ensure accuracy, completeness, and consistencyOptimize Databricks pipeline performance: cluster configuration, auto-scaling, partitioning, caching, Z-ordering, and query plan tuning for large-scale event datasetsMaintain schema registry alignment and XDM-compatible data structures across all pipeline outputsSupport data governance standards: lineage documentation, metadata cataloging (Unity Catalog), access controls, and PII handling policiesMonitor pipeline health via structured logging, alerting, and SLA dashboards; triage and resolve data incidents in productionCollaborate with the Data Architect, CJA Architect, AXP Architect, and BI/Analytics teams to align pipeline designs with reporting and analytics requirementsCore SkillsDatabricks – Delta Live Tables, Structured Streaming, Databricks Workflows, cluster management, Unity CatalogDelta Lake / Lakehouse architecture – medallion design patterns, ACID transactions, time travel, schema evolutionPySpark and/or Scala Spark – large-scale data transformation, aggregations, windowing, and joinsSQL – complex query authoring, performance tuning, and Data Distiller query patterns for AEP datasetsKafka – real-time event consumption, offset management, schema-on-read patternsAEP / Data Distiller – dataset querying, XDM schema familiarity, profile and event dataset consumptionCloud data platform experience (AWS S3/Glue, Azure ADLS/Synapse, or GCP GCS/BigQuery)Data quality and observability tooling (Great Expectations, Monte Carlo, or equivalent)CI/CD for data pipelines: version control (Git), automated testing, and deployment automationUnderstanding of data governance principles: lineage, cataloging, access control, and PII/data privacyTypical Experience10+ years of data engineering experience, with significant hands-on Databricks delivery in production environmentsProven experience building and operating both batch and streaming pipelines at scale using Delta Lake and SparkExperience integrating with Kafka or other real-time event-streaming platforms as a consumerFamiliarity with AEP, Data Distiller, or Adobe analytics data models preferredExperience with lakehouse or cloud data warehouse architectures (Delta Lake, Snowflake, BigQuery, or Redshift)Exposure to MarTech, CDP, or personalization platform data flows (AJO, AEP, or equivalent) is a strong plusThanks & RegardsShyam (SAM)Sr.RecruiterEmail: sam.s@navasoftware.com","datePosted":"2026-07-04T09:19:39.482Z","dateModified":"2026-07-04T09:19:39.482Z","hiringOrganization":{"@type":"Organization","name":"Nava Software Solutions","sameAs":"https://jobsearcher.com"},"jobLocationType":"TELECOMMUTE","applicantLocationRequirements":{"@type":"Country","name":"US"},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"52fcd77e92da7ed382e93452"},"url":"https://jobsearcher.com/jobs/52fcd77e92da7ed382e93452"}}