Big Data Developer
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Role: Big Data Developer Location:-Charlotte, NC
Desired Skills: Big Data | Hadoop
Salary Range: -$110,000-$120,000 a year
Primary Skill:- Data Engineering, Platform Engineering or architecture roles. Deep Expertise in Pyspark.
Required Experience:
- 10+ yrs Roles & Responsibilities Required Hard and Soft Skills / Experience
- Deep Expertise in PySpark, including performance tuning and optimization
- Strong python development experience in large-scale distributed environment
- Solid knowledge of Hadoop ecosystem (HDFS, Hive/Impala, YARN)
Proven experience designing and governing enterprise, regulatory facing data platforms.
Expertise in designing data lakes, ELT/ETL pipelines, batch and real time data processing solution
Proficiency in programming languages such as Java, Scala and SQLStrong understanding of non-functional requirements and production support models
Clear written and verbal communication skills with ability to influence across organizations Preferred Skills / Experience
Financial services experience, particularly in Market Surveillance, AML, Fraud, Or Risk Technology
Experience supporting regulatory or audit facing platforms
Kafka and Spark Structured streaming exposure
Familiarity with Orchestration tools(Airflow, Control-M, Oozie)
Knowledge of data governance, lineage, and data quality controls Solution Design and Delivery Support Translate surveillance business requirements(e.g, market misconduct detection, regulatory coverage) into scalable technical designs Review and approve detailed technical designs, ensuring alignment with functional intent, regulatory requirements, and architectural standards. Provide hands-on architectural guidance to engineering teams during development, testing and implementation Design and standardization of Spark/Pyspark frameworks supporting surveillance alert generation and enrichment Modernization and Optimization of large-scale Hadoop surveillance workloads to improve performance, stability and control coverage Implementation of enterprise-consistent architecture patterns enabling audit readiness, lineage, and regulatory traceability Support end-to-end delivery across the SDLC, minimizing rework and technical debt