Data Engineering (Palantir, Spark, PySpark, Python)- W2 Role

Job Description Job Title: Technical Lead Data Engineering (Palantir, Spark, PySpark, Python)Please dont apply if you dont have PALANTIR experienceOnsite work :One or two days in a monthW2 roleOPT with 7 years exp is fineJob SummaryWe are looking for a hands-on Technical Lead in Data Engineering to drive the design, development, and delivery of scalable data solutions in a large retail enterprise. This role requires strong expertise in Palantir Foundry, Spark/PySpark, SQL, and Python, along with the ability to lead engineering teams and partner with business stakeholders across supply chain, merchandising, and store operations.Key ResponsibilitiesLead the design and implementation of scalable data platforms and pipelines using PySpark, Spark, and PythonDrive adoption and best practices for Palantir Foundry (data pipelines, ontology, workflows, and operational applications)Architect and optimize high-performance data processing solutions for large-scale datasetsProvide technical leadership and mentorship to data engineers, ensuring code quality and best practicesCollaborate with cross-functional teams (business, analytics, data science) to translate requirements into scalable solutionsDesign robust data models, ETL/ELT frameworks , and data integration strategiesEnsure data quality, governance, security, and compliance across enterprise data platformsLead performance tuning, troubleshooting, and optimization of data pipelinesDrive CI/CD implementation , code reviews, and release managementStay current with emerging data engineering technologies and recommend improvementsRequired QualificationsBachelors or Masters degree in Computer Science, Engineering, or related field8+ years of experience in data engineering , with at least 23 years in a technical leadership roleStrong hands-on experience with: Python & PySparkApache SparkAdvanced SQLExperience with Palantir Foundry or similar modern data platformsDeep understanding of data engineering principles (ETL/ELT, data modeling, distributed systems)Experience designing and managing large-scale data architecturesStrong leadership, communication, and stakeholder management skillsPreferred QualificationsExperience in retail domain (supply chain, inventory, merchandising, store analytics)Experience with cloud platforms (Azure, AWS, or GCP)Familiarity with orchestration tools (Airflow, Azure Data Factory, etc.)Experience with real-time/streaming data pipelinesExposure to DevSecOps practicesKey SkillsTechnical LeadershipData Engineering ArchitecturePySpark & SparkPython ProgrammingSQL OptimizationPalantir FoundryData Modeling & WarehousingCI/CD & DevOpsFlexible work from home options available.