AWS Databricks Data Engineer

Dear All,Vertex Elite is currently seeking a qualified AWS Databricks Data Engineer to join our team.If you or someone in your network might be a fit, please feel free to reach out for more details or share an updated resume.Work Authorization: USC or Any valid US work authorizationJob Type: W2 Contract with Vertex Elite LLCLocation: Malvern, PennsylvaniaRequired Qualifications:Design, develop, and maintain scalable data pipelines using Python, PySpark, Pandas, and advanced SQL.Develop data transformation and ingestion processes following object-oriented programming principles and best practices.Build and manage data solutions using Databricks Lakehouse, Workflows, and Delta Lake.Develop and optimize big data processing jobs using Apache Spark and, when required, Kafka.Implement data integration and ETL/ELT pipelines across various data sources and target systems.Use AWS services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift to build cloud-based data solutions.Perform data cleansing, validation, transformation, and quality checks to ensure data accuracy and consistency.Write, optimize, and troubleshoot complex SQL queries for data extraction and transformation.Implement data partitioning, optimization, and performance-tuning techniques for large datasets.Collaborate with data architects, analysts, software engineers, and business stakeholders to understand data requirements.Develop and maintain infrastructure automation using Terraform.Containerize applications and data workloads using Docker, where applicable.Support CI/CD implementation and automated deployments using GitHub Actions.Monitor pipeline execution, troubleshoot failures, and resolve data processing issues.Follow security, governance, documentation, and coding standards across data engineering projects.Participate in code reviews, technical discussions, testing, and production support activities.With Best Regards,Manoj Visanagiri | Vertex Elite LLC | E-Verified Company | www.vertexelites.com