Contract Python Data Engineer (Databricks / PySpark / SQL)

Job Title: Contract Python Data Engineer (Databricks / PySpark / SQL)Location: Columbus, INRole Summary:We are seeking a senior-level contract Python engineer with strong experience in Databricks-based data platforms to support and optimize large-scale analytical and data processing workflows. The role focuses on Python performance optimization, PySpark development, and SQL-based analytics in a distributed environment.The ideal candidate is hands-on, performance-focused, and comfortable working with complex, memory-intensive pipelines.Key Responsibilities:Develop and optimize Python-based data processing pipelines on DatabricksWork extensively with PySpark DataFrames, pandas, and SQLDesign and optimize Databricks jobs, notebooks, and workflowsPerform runtime and memory profiling of Python and Spark workloadsIdentify and resolve:Memory retention issuesInefficient transformationsPerformance bottlenecks in large datasetsOptimize pandas PySpark SQL data flowsSupport long-running batch and analytical jobsCollaborate with data scientists and engineers to improve scalability and stabilityRequired Skills (Must-Have):Expert Pythonpandas, NumPyWriting memory-efficient, production-quality codeStrong Databricks experienceNotebooks, Jobs, ClustersUnderstanding Spark execution and memory behaviorPySparkDataFrame APIsCaching, persistence, partitioningSQLComplex joins, aggregations, window functionsPerformance optimizationRuntime profilingMemory analysis and debuggingExperience handling large-scale analytical datasetsNice-to-Have Skills:Distributed computing frameworks (Spark internals, Ray, multiprocessing)Experience with cloud data platformsFamiliarity with Python garbage collection and object lifecycleExperience supporting analytics, reporting, or reliability systems