Python Engineer / PySpark (Mainframe Modernization)

Senior Software Engineer – Python / PySpark (Mainframe Modernization)Location: San Francisco, CA (Onsite/Hybrid)Duration: 24 Months Contract-to-HireOverviewWe are hiring a Senior Software Engineer to support the modernization of a mission-critical, mainframe-based financial platform into a distributed, cloud-native architecture. This system serves as the system of record for financial data and plays a key role in enterprise accounting and general ledger processing.This is a highly technical, backend-focused role requiring deep expertise in Python and PySpark, along with the ability to guide teams transitioning from legacy mainframe technologies to modern engineering practices.Key ResponsibilitiesDesign and develop backend services to replace legacy mainframe functionalityBuild scalable data pipelines for ingestion, validation, and transformation using Python and PySparkDevelop RESTful APIs using FastAPI and/or FlaskCollaborate within an Agile/Scrum team with Product Owners and business stakeholdersAct as a Python SME, mentoring engineers transitioning from mainframe environmentsEnsure data accuracy and integrity for financial data feeding general ledger systemsParticipate in architecture discussions, code reviews, and technical decision-makingContribute to frontend development using React and Angular as neededSupport CI/CD pipelines and containerized deploymentsCommunicate effectively with both technical and non-technical stakeholdersRequired Qualifications7+ years of professional software engineering experienceStrong hands-on expertise in Python and PySpark (production experience required)Experience building APIs using FastAPI and/or FlaskExposure to frontend frameworks (React and/or Angular)Experience with SQL Server and/or ExadataHands-on experience with Kafka or other streaming platformsExperience working in Agile/Scrum environmentsStrong communication, presentation, and stakeholder engagement skillsProven ability to mentor and guide engineersPreferred QualificationsExperience with Agentic AI frameworks (LangGraph, LangChain)Programming: Java, Scala, SQL, HiveQLBig Data: Hadoop, Spark, Hive, HDFSCloud: AWS (Glue, Lambda, Redshift, S3, CloudWatch)ETL/ELT: Databricks, AWS Glue, PySparkData Engineering: Data lakes, real-time streaming, Kafka ecosystemsCI/CD: Jenkins, GitHub Actions, Azure DevOps, AirflowContainerization & Orchestration: KubernetesDomain ExperienceExperience in financial services or enterprise accounting environments preferredFamiliarity with general ledger systems and financial data flows is a plusIdeal Candidate ProfileSenior backend engineer with strong data engineering expertiseTechnical leader and mentor without direct people management responsibilitiesStrong communicator capable of simplifying complex systemsAdaptable and collaborative in cross-functional environmentsSkills: kafka,fastapi,angular,pyspark,agile,python,mainframe,modernization,react