{"schemaVersion":"jobsearcher.job.v1","id":"17f73ceb301411879903e83a","url":"https://jobsearcher.com/jobs/17f73ceb301411879903e83a","canonicalUrl":"https://jobsearcher.com/jobs/17f73ceb301411879903e83a","title":"Splunk Developer","description":"Job Title: Splunk DeveloperLocation: Chicago, IL (Hybrid)Duration: Long-Term ContractJob Specification: Splunk Observability EngineerSynopsis:To design, implement, and optimize a full-stack observability strategy using the Splunk Observability Cloud (formerly SignalFx) and Splunk Enterprise/Cloud. You will ensure that engineering teams have 360-degree visibility into system health, moving the organization from reactive \"firefighting\" to proactive \"pattern-based\" incident prevention.Key Responsibilities:Data Orchestration: Architect the ingestion of the \"Three Pillars\" (Metrics, Logs, Traces) using OpenTelemetry (OTel) collectors.Aggregation Strategy: Develop logic to aggregate high-cardinality data to reduce \"noise\" while maintaining \"signal\" for troubleshooting.Analytical Modeling: Use SPL (Search Processing Language) and SignalFlow to perform pattern analysis, detecting anomalies before they trigger traditional threshold alerts.Visual Storytelling: Build executive and technical dashboards that correlate disparate data points (e.g., showing how a spike in 500-errors in Logs relates to a specific span in a Trace).Required Hands on Technical Skills1. Telemetry & Data SpecializationLogs: Proficiency in \"Logging-in-Context.\" You must be able to link logs directly to trace IDs so developers can jump from a failing trace to the specific line of code in the logs.Metrics: Expertise in SignalFlow (Splunk’s background streaming analytics language). You should know how to calculate percentiles ($P95, P99$), rates of change, and historical averages.Traces: Deep understanding of Distributed Tracing. You must know how to instrument applications (Java, Python, Go) to capture spans and identify bottlenecks in microservices.2. Pattern Analysis & AggregationAnomaly Detection: Ability to configure Metric Finder and MDetector using standard deviations or \"Mean Absolute Deviation\" to find outliers.Data Scrubbing: Skills in using Splunk Ingest Actions or Edge Processors to filter, mask, or aggregate data at the edge to save on license costs and improve search speed.Pattern Discovery: Using Splunk’s machine learning commands (e.g., findkeywords, cluster) to group millions of log events into a few dozen \"patterns\" for faster root cause analysis.3. Hands on - Dashboards & VisualizationHigh-Cardinality Handling: Designing dashboards that don’t \"break\" when viewing thousands of containers.Contextual Drill-downs: Building \"Glass Tables\" (in ITSI) or Unified Dashboards that allow a user to click a metric and immediately see the associated logs.Frameworks: Familiarity with the Dashboard Studio and JSON-based dashboard definitions for version control (GitOps).Preferred Qualifications & CertificationsDevOps & IAC skillsSplunk Cloud Certified Metrics User: Focuses on the metrics and alerting side.Splunk Core Certified Power User: Essential for mastering complex SPL for log analysis.OpenTelemetry Expert: Knowledge of the OTel Collector configuration (receivers, processors, exporters) is currently the most \"in-demand\" skill for this role.","company":"Thought Byte","rawCompany":"thought byte","city":"Chicago","state":"IL","isRemote":false,"isActive":false,"createdAt":"2026-08-19T13:26:27.294Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Splunk Developer","description":"Job Title: Splunk DeveloperLocation: Chicago, IL (Hybrid)Duration: Long-Term ContractJob Specification: Splunk Observability EngineerSynopsis:To design, implement, and optimize a full-stack observability strategy using the Splunk Observability Cloud (formerly SignalFx) and Splunk Enterprise/Cloud. You will ensure that engineering teams have 360-degree visibility into system health, moving the organization from reactive \"firefighting\" to proactive \"pattern-based\" incident prevention.Key Responsibilities:Data Orchestration: Architect the ingestion of the \"Three Pillars\" (Metrics, Logs, Traces) using OpenTelemetry (OTel) collectors.Aggregation Strategy: Develop logic to aggregate high-cardinality data to reduce \"noise\" while maintaining \"signal\" for troubleshooting.Analytical Modeling: Use SPL (Search Processing Language) and SignalFlow to perform pattern analysis, detecting anomalies before they trigger traditional threshold alerts.Visual Storytelling: Build executive and technical dashboards that correlate disparate data points (e.g., showing how a spike in 500-errors in Logs relates to a specific span in a Trace).Required Hands on Technical Skills1. Telemetry & Data SpecializationLogs: Proficiency in \"Logging-in-Context.\" You must be able to link logs directly to trace IDs so developers can jump from a failing trace to the specific line of code in the logs.Metrics: Expertise in SignalFlow (Splunk’s background streaming analytics language). You should know how to calculate percentiles ($P95, P99$), rates of change, and historical averages.Traces: Deep understanding of Distributed Tracing. You must know how to instrument applications (Java, Python, Go) to capture spans and identify bottlenecks in microservices.2. Pattern Analysis & AggregationAnomaly Detection: Ability to configure Metric Finder and MDetector using standard deviations or \"Mean Absolute Deviation\" to find outliers.Data Scrubbing: Skills in using Splunk Ingest Actions or Edge Processors to filter, mask, or aggregate data at the edge to save on license costs and improve search speed.Pattern Discovery: Using Splunk’s machine learning commands (e.g., findkeywords, cluster) to group millions of log events into a few dozen \"patterns\" for faster root cause analysis.3. Hands on - Dashboards & VisualizationHigh-Cardinality Handling: Designing dashboards that don’t \"break\" when viewing thousands of containers.Contextual Drill-downs: Building \"Glass Tables\" (in ITSI) or Unified Dashboards that allow a user to click a metric and immediately see the associated logs.Frameworks: Familiarity with the Dashboard Studio and JSON-based dashboard definitions for version control (GitOps).Preferred Qualifications & CertificationsDevOps & IAC skillsSplunk Cloud Certified Metrics User: Focuses on the metrics and alerting side.Splunk Core Certified Power User: Essential for mastering complex SPL for log analysis.OpenTelemetry Expert: Knowledge of the OTel Collector configuration (receivers, processors, exporters) is currently the most \"in-demand\" skill for this role.","datePosted":"2026-08-19T13:26:27.294Z","dateModified":"2026-08-19T13:26:27.294Z","hiringOrganization":{"@type":"Organization","name":"Thought Byte","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Chicago","addressRegion":"IL","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"17f73ceb301411879903e83a"},"url":"https://jobsearcher.com/jobs/17f73ceb301411879903e83a"}}