Senior Python Developer with Spark

Senior Python Developer with SparkPosition DescriptionSenior Python Developer with Spark - design, build, and optimize large-scale data processing systems in a cloud-native AWS environment. This role focuses heavily on developing high-performance data pipelines using Spark and Python, as well as working with complex relational datasets.Located at a client site in Reston, VA with a hybrid working model.Future Duties and ResponsibilitiesDiagnose and improve inefficient data workflows, especially Spark jobs, and write advanced SQL to support hierarchical and analytical use cases. Partner with cross-functional teams to deliver scalable, reliable, and well-architected data solutions that support critical business functions in a financial services setting.Required Qualifications8+ years of hands-on experience with Python for data engineering, including building and maintaining data pipelinesDeep expertise in Apache Spark, including performance tuning (partitioning, caching, broadcast joins, shuffle optimization), understanding of execution plans (DAGs, stages, tasks), memory and resource managementSolid experience with big data ecosystems such as Hadoop, Hive, and EMRAdvanced proficiency in SQL, including writing recursive CTEs for hierarchical data, query optimization, indexing strategies, and execution plan analysisStrong experience with AWS services including EMR, Lambda, Step Functions, EventBridge, Redshift, S3, and GlueExperience building and consuming APIs, along with data transformation and ingestion workflowsProven ability to work with large-scale datasets, performing data analysis and extracting actionable insightsFamiliarity with data modeling concepts (normalized/denormalized structures, handling hierarchical data)Hands-on experience with CI/CD pipelines and tools such as GitLab and TerraformStrong understanding of performance troubleshooting, including identifying bottlenecks in distributed systemsAbility to clearly explain technical decisions, especially around Spark optimization and SQL logicStrong analytical thinking, problem-solving skills, and attention to detailEffective collaboration skills in cross-functional, matrixed environmentsDesired SkillsetExperience in financial services or regulated environmentsExposure to data visualization toolsFamiliarity with event-driven architectures on AWSEducationBachelor's degree in Computer Science, Information Systems, or a related field.CompensationCompensation range: $99,200.00 - $154,300.00.BenefitsCompetitive compensationComprehensive insurance optionsMatching contributions through the 401(k) plan and the share purchase planPaid time off for vacation, holidays, and sick timePaid parental leaveLearning opportunities and tuition assistanceWellness and Well-being programEqual OpportunityQualified applicants will receive consideration for employment without regard to their race, ethnicity, ancestry, color, sex, religion, creed, age, national origin, citizenship status, disability, pregnancy, medical condition, military and veteran status, marital status, sexual orientation or perceived sexual orientation, gender, gender identity, and gender expression, familial status or responsibilities, reproductive health decisions, political affiliation, genetic information, height, weight, or any other legally protected status or characteristics to the extent required by applicable federal, state, and/or local laws where we do business.AccommodationsCGI provides reasonable accommodations to qualified individuals with disabilities. If you need an accommodation to apply for a job in the U.S., please email the CGI U.S. Employment Compliance mailbox at USEmploymentCompliance@cgi.com. You will need to reference the Position ID of the position in which you are interested.Background InvestigationAll CGI offers of employment in the U.S. are contingent upon the successful completion of a background investigation. Investigation components may vary depending on specific assignment and/or level of U.S. government security clearance held. Some investigations may include a credit check. CGI will consider qualified applicants with arrests and conviction records in accordance with all local regulations and ordinances.Pay Visibility PolicyCGI will not disclose pay information to individuals who do not otherwise have access to compensation information, except as permitted by law or formal investigation. Employees with access to compensation may disclose pay of other employees only in response to a formal complaint or charge, in furtherance of an investigation, proceeding, hearing, or action, or consistent with CGI's legal duty to furnish information.J-18808-Ljbffr