Tech Lead (Python + Databricks + Streamlit)

Role: Tech Lead (Python Databricks Streamlit)

Location: Iselin, NJ (Hybrid)

Salary: $120k to $140k

Rate: $80/hr to $85/hr

Position Summary: We are looking for aTech Leadwith strong hands-on expertise inPythonandDatabricks, along with exposure toStreamlit (optional)for data-driven application development. The ideal candidate will lead technical design, architecture, and delivery of scalable data and application platforms while mentoring development teams and driving engineering best practices.

Key Responsibilities: Lead and mentor a team of developers and data engineers across the full software development lifecycle. Design and develop scalable backend solutions, APIs, and data processing frameworks usingPython. Proficient in creating HLD, LLD, and detailed design documentation using visual artifacts. Build and optimize enterprise-grade data pipelines using Databricks, PySpark, and Spark SQL. Good understanding of Medallion Architecture, Data Quality rules, exception handling/correction processes, and reprocessing of correction DAGs. Collaborate with Business analysts, technical and business stakeholders to translate requirements into technical solutions. Drive code quality through design reviews, best practices, testing, and CI/CD implementation. Ensure application reliability, scalability, security, and performance in cloud environments. Support DevOps processes, deployment automation, monitoring, and production support. Required Skills & Experience10–12 years of experience in software development and data engineering. Strong hands-on experience in Python Extensive experience with Databricks, PySpark, Good to have understanding ofStreamlitor other modern UI frameworks for data applications à Optional Experience building scalable ETL/ELT pipelines and distributed systems. Strong knowledge ofAzure(preferred) or AWS cloud platforms. Experience with Git, CI/CD pipelines, Docker, and containerized deployments. Proficiency in SQL and database technologies (PostgreSQL, MySQL, MongoDB, CosmosDB, etc.). Strong problem-solving, leadership, and stakeholder management skills.

Preferred Qualifications: Experience in Data Quality, Exception Handling, Metadata Management, Data Governance solutions. Exposure to Agile/Scrum delivery models. Experience leading teams in enterprise-scale data platform implementations.