Business Analyst - Retail
Request ID: 70328-1 Title: Business Analyst - Retail Locations: Arden, DE Duration: 6 Months Pay Range: $48 - $53/Hour on W2 (All inclusive) - Applicants must be willing to work on W2 Job Description Role Summary The Databricks Consultant / Developer will design, build, and support data pipelines and integrations on the Azure data platform using Databricks, Airflow, ADF, and ADLS. The role spans both project development (Build) and production support (Run), with a strong focus on DAG orchestration, data transformation, and end to end pipeline reliability across retail and supply chain use cases. Role Description: Databricks Python L2 Support Engineer Providing extensive support for the Integration Services Analyzing issues and root causes of the issues Understanding of batch programs and interfaces in Aptos Store Inventory Management System Handling high priority incidents P1P2 till closure Key Responsibilities 1. Data Pipeline & DAG Development (Build) Design and develop Spark / PySpark notebooks and jobs in Azure Databricks to ingest, transform, and publish batch and near real time data across multiple domains (merchandising, supply chain, digital, finance, store systems). Build and maintain DAGs (primarily in Airflow / Astronomer and Databricks Workflows) to orchestrate dependencies between upstream systems (e.g., RMS, e commerce, WMOS, Kafka topics) and downstream data products, ensuring correct ordering, retries, and alerting. Use ADF and other integration services where required for file, API, and database ingestion into ADLS Gen2, applying appropriate partitioning and optimization techniques for large scale workloads. Implement reusable frameworks for common ETL / ELT patterns (SFTP/Kafka ingestion, CDC, merge/upsert to Delta tables, slowly changing dimensions, data quality checks). 2. Data Transformation & Modeling Translate business and source to target requirements into robust data transformations, aggregations, and business rules implemented in Databricks (Delta Lake). Design and optimize logical and physical data models (staging, core, marts) to support reporting, analytics, and downstream applications, with attention to performance and cost optimization. Apply Databricks and Spark best practices (partitioning, clustering, caching, broadcast joins, shuffle reduction, incremental processing) to meet SLAs for nightly and intraday jobs. Required Skills & Experience 5–8+ years of overall data engineering experience, with 3+ years hands on in Azure Databricks / Spark building production grade data pipelines. Strong expertise in PySpark / Spark SQL, Delta Lake, and performance tuning for large volume ETL / ELT workloads. Solid experience with DAG based orchestration tools such as Airflow / Astronomer (or equivalent), including sensors, task dependencies, and advanced scheduling patterns. Hands on experience with Azure Data Factory, ADLS Gen2, and broader Azure data ecosystem for data movement and storage. Strong SQL skills for data profiling, validation, and reconciliation across multiple source and target systems. Proven experience supporting production data pipelines (L2/L3 or DevOps style), including incident management, root cause analysis, and permanent fixes for job failures and data issues. Experience with Git / Azure DevOps or similar tools for version control, CI/CD, and work tracking (Jira/ServiceNow/Confluence). Ability to work in a fast paced, business critical environment with multiple stakeholders, balancing project delivery with ongoing support. Competencies: 6-8+ years experience required Business Analysis in Retail Python Databricks PL/SQL ServiceNow Databricks Consultant / Developer – Data Pipelines & Integrations Company Benefits & CultureInclusive and diverse work environmentOpportunities for professional growth and developmentComprehensive health and wellness benefits Appreciate your quick response and please feel free to reach me out for any query you may have. Thanks