Python Data Engineer
BCforward is seeking a highly motivated and experienced Senior Python Data Engineer
Note: Candidate must be local to Phoenix, AZ.
Job Title: Senior Python Data Engineer
Job Location: Phoenix, AZ 85054 (Hybrid)
Duration: Long term Contract
Pay Rate: $60/Hr. to $65/Hr. On C2CRole Overview
Core Technologies: Python, SQL, GCP, AWS, Oracle, PostgreSQL, Couchbase, Database Architecture & Design, GenAI, Agentic AI, LLMs, RAGWe are seeking a highly skilled, hands-on Data Engineer with exceptional Python development capabilities, deep database engineering expertise, strong cloud engineering experience, and practical knowledge of modern AI/GenAI technologies. The ideal candidate will have a proven track record of designing, developing, optimizing, and modernizing scalable, high-performance enterprise data platforms leveraging Python, Oracle, PostgreSQL, Couchbase, GCP, and AWS.This role requires a strong technical engineer with advanced software development skills, solid database architecture fundamentals, hands-on cloud implementation experience, and the ability to build intelligent, AI-powered data solutions using LLMs, Retrieval-Augmented Generation (RAG), and Agentic AI frameworks. The candidate must be comfortable working across the entire engineering lifecycle, from architecture and design through hands-on coding, performance optimization, cloud deployment, and production support.
Job Description:
- Key Responsibilities1. Advanced Python Development & Software EngineeringDesign, develop, and maintain complex, scalable, production-grade Python applications, data engineering frameworks, backend services, and reusable software components.
- Demonstrate expert-level Python programming skills, including object-oriented programming (OOP), functional programming, decorators, generators, iterators, context managers, and exception handling.
- Build modular, maintainable, highly optimized Python applications following clean code principles, SOLID principles, and established software design patterns.
- Develop robust Python-based ETL/ELT frameworks, data ingestion services, transformation engines, data validation utilities, and workflow automation solutions.
- Build RESTful APIs, microservices, and backend integration components using FastAPI, Flask, or similar Python frameworks.
- Implement asynchronous programming, multithreading, multiprocessing, and parallel processing to optimize high-volume data workloads.
- Work extensively with Python libraries and frameworks such as Pandas, NumPy, SQLAlchemy, psycopg, python-oracledb, PyTest, boto3, and Google Cloud SDKs.
- Develop efficient database integration components, including connection pooling, bulk data processing, transaction management, and optimized data access layers.
- Optimize Python applications for execution speed, memory utilization, CPU efficiency, scalability, and resource consumption.
- Build reusable Python utilities and automation frameworks for database migration, schema comparison, data reconciliation, data quality validation, and performance monitoring.
- Implement comprehensive unit testing, integration testing, mocking, logging, debugging, exception handling, and automated quality checks.
- Develop and maintain CI/CD pipelines for Python application testing, packaging, deployment, and release management.
- Perform code reviews, troubleshoot complex production issues, and drive continuous improvements in software quality, reliability, and performance.2. Database Architecture, Design & Performance Optimization
- Design, develop, and optimize highly scalable, resilient, secure, and high-performance enterprise database architectures.
- Demonstrate deep hands-on expertise in Oracle, PostgreSQL, and Couchbase, including database design, development, and performance optimization.
- Design efficient relational and NoSQL data models, database schemas, indexing strategies, partitioning approaches, and data access patterns.
- Develop and optimize complex SQL queries, stored procedures, functions, views, and database integration components.
- Perform advanced SQL performance tuning, execution plan analysis, query optimization, indexing, and database troubleshooting.
- Apply strong knowledge of database internals, transaction management, concurrency control, locking, isolation levels, and storage optimization.
- Design and implement database replication, high availability, backup, recovery, and disaster recovery solutions.
- Build Python-driven database automation, monitoring, migration, and performance optimization frameworks.
- Lead database modernization initiatives, including Oracle-to-PostgreSQL migrations, legacy database transformation, and on-premises-to-cloud migrations.
- Design scalable data architectures supporting transactional, analytical, and AI-driven workloads.
Ensure data integrity, security, governance, availability, and operational efficiency across enterprise database platforms.3. Cloud Engineering – GCP & AWSArchitect, develop, and deploy cloud-native data engineering solutions and Python applications across Google Cloud Platform (GCP) and Amazon Web Services (AWS).Demonstrate strong hands-on experience with GCP services such as BigQuery, Cloud SQL, AlloyDB, Cloud Storage, Dataflow, Pub/Sub, Cloud Run, and Vertex AI.Work with AWS services including RDS, Aurora, S3, Redshift, Glue, Lambda, Step Functions, and Bedrock.
Design scalable, fault-tolerant, secure, and cost-efficient cloud data architectures.
Develop Python-based serverless applications, event-driven data processing solutions, and cloud automation frameworks.
Integrate Python applications with cloud-native services using boto3, Google Cloud client libraries, REST APIs, and SDKs.
Build and optimize cloud-native data ingestion, transformation, processing, and integration pipelines.
Implement cloud infrastructure automation, deployment pipelines, monitoring, logging, and operational best practices.
Support hybrid-cloud and multi-cloud data modernization initiatives.
Optimize cloud resource utilization, data processing performance, and infrastructure costs.
Implement cloud security best practices, including IAM, encryption, secrets management, network security, and access controls.4. Data Engineering & Scalable Pipeline Development
Design and develop scalable batch, near-real-time, and streaming data processing pipelines using Python, SQL, and cloud-native data engineering services.
Build reusable frameworks for data ingestion, transformation, validation, reconciliation, and integration.
Integrate data across Oracle, PostgreSQL, Couchbase, cloud databases, APIs, object storage, and enterprise applications.
Design efficient ETL/ELT architectures capable of processing large-scale enterprise datasets.
Implement data quality checks, schema validation, exception handling, audit logging, and automated recovery mechanisms.
Optimize pipelines for throughput, latency, scalability, reliability, and cost efficiency.
Work with distributed data processing and orchestration technologies such as Apache Spark, Airflow, Kafka, and cloud-native orchestration services.
Implement data observability, monitoring, lineage, and operational support capabilities.
Develop data processing architectures supporting analytics, reporting, operational applications, and AI/ML use cases.5. AI / GenAI, Agentic AI & Intelligent Data EngineeringDesign, develop, and deploy production-grade AI/GenAI applications and intelligent agents using Python, Large Language Models (LLMs), and modern AI development frameworks.
Demonstrate hands-on experience in Agentic AI development, including building autonomous and semi-autonomous AI agents capable of reasoning, planning, tool execution, and multi-step workflow orchestration.
Build LLM-powered AI agents using frameworks such as LangChain, LangGraph, CrewAI, AutoGen, or similar agentic development frameworks.
Design and implement Retrieval-Augmented Generation (RAG) architectures, including document ingestion, text chunking, embedding generation, vector indexing, semantic search, hybrid retrieval, reranking, and context optimization.
Develop single-agent and multi-agent architectures enabling intelligent task execution, agent collaboration, tool calling, and workflow automation.
Integrate LLMs with enterprise databases, APIs, cloud services, and internal applications to build intelligent, data-driven solutions.
Develop Python-based AI applications leveraging OpenAI APIs, Google Vertex AI / Gemini, AWS Bedrock, and other enterprise LLM platforms.
Implement vector database and similarity search solutions using pgvector, Pinecone, FAISS, Chroma, or equivalent technologies.
Build intelligent agents capable of securely interacting with Oracle, PostgreSQL, Couchbase, and cloud-native data platforms for data retrieval, analysis, validation, and automation.
Develop reusable Python frameworks for LLM orchestra