Data Engineer Software Developer

Overview In this role you will design, build, and operate scalable data pipelines that ingest and transform large, disparate data sources for analytics and dashboards. You contribute to unifying network operations data into a consistent platform on Elastic Stack and Confluent Kafka, enabling a single-pane-of-glass UX. You collaborate with cross-functional teams to implement robust data integration, automation, and governance in a DoD-focused environment. You’ll work within an Agile team to deliver end-to-end data solutions that support mission-critical decision making. ResponsibilitiesDesign, build, and maintain scalable data systems and pipelines for ingestion, processing, and transformation of large data sourcesIntegrate data from multiple network operations systems into a unified platform with emphasis on data quality, security, and DoD/DISA complianceDevelop and manage data pipelines using Confluent Kafka, Elastic Stack, AWS, and OpenShift, including dashboards with Kibana and front-end visuals (React/JS/HTML)Create and optimize data warehouses/data lakes to enable analytics, reporting, and operational dashboardsOptimize ingestion throughput, query performance, indexing, and storage for Elastic and KafkaAutomate ETL/ELT tasks, data validation, monitoring, and CI/CD supportCollaborate with data analysts, visualization engineers, and stakeholders; document architecture and proceduresTroubleshoot data ingestion, processing, storage, and cluster performance issues Key requirementsBachelor’s degree in Computer Science, Engineering, or related technical discipline4+ years of experience in data pipelines (ETL/ELT)Experience with data warehouse/lake tech (AWS Redshift, S3, Hadoop, Snowflake)Proficiency in Python, Java, SQL, and shell scriptingExperience with Elastic Stack and/or Kafka for data ingestion/processingFamiliarity with network operations data and related data modelsLinux/UNIX administration and automationKnowledge of DoD, DISA, RMF, STIGs data security/complianceExperience with CI/CD, containerized pipelines, Git/Bitbucket, DevOpsStrong communication/documentation skillsAbility to commute to Scott AFB, IL; ability to obtain interim DoD Secret clearance and Security+ within 14 daysStrong communicationCollaborative mindsetDetail-orientedElastic Stack (Elasticsearch, Logstash, Kibana)Kafka/ConfluentPython