Python/PySpark Developer

Python/PySpark DeveloperSoftware Guidance & Assistance, Inc., (SGA), is searching for a Python/PySpark Developer for a right to hire assignment with one of our premier financial services clients in Rutherford, NJ.ResponsibilitiesDesign, develop, and maintain scalable, enterprise-grade AI agents, supporting ELT/ETL processes to handle large data volumes using the Python, FAST API, Microservices, PySpark, Kafka, and Databricks ecosystem.Develop, deploy, and automate microservice integrations to support data-intensive applications, ensuring scalability, resilience, and maintainability using cloud native infrastructure and openshift or Kubernates architecture including CI/CD pipelines.Ensure data quality, integrity, and security throughout the entire data lifecycle.Contribute to the continuous improvement of data engineering processes, standards, and best practices within the team.Required Skills8+ years of overall experience in large-scale application development with recent mandatory platform for the secure and scalable deployment of AI agents into application contextsMinimum of 5+ years of proven experience in a Python and pyspark Engineering lead role focused on building enterprise-grade, high-volume ELT/ETL processes using the PySpark.Hands-on experience with YAML, JSON, FAST API or Spring boot, Github CopilotProven experience developing and automating microservice integrations to support data-intensive in at least one programming language commonly used for data analytics, engineering, such as Python or Scala.Strong SQL skills and experience with various relational understanding of data modeling, data warehousing concepts, Data Mesh architecture, and data communication, collaboration, and problem-solving skills