Sr. Data Engineer - Monetization & Experimentation

RedCloud Consulting is a global business and technology consulting firm supporting clients across North America, Europe, Asia-Pacific, and the Middle East. For more than two decades, RedCloud has partnered with organizations ranging from high-growth mid-market companies to global enterprises, helping them solve complex business and operational challenges—leveraging modern data platforms, artificial intelligence, and secure digital foundations to drive smarter decisions and scalable outcomes.RedCloud seeks a Senior Data Engineer to support immediate client operations.Location: Remote, United StatesOverviewThe Senior Big Data Engineer will support core data platform delivery while driving engineering productivity through the development of internal accelerators and reusable tooling. This role is focused on real production impact rather than conceptual or research work. Ideal candidates are hands-on engineers who thrive in high scale environments, care deeply about reliability, and enjoy building systems and tools that make other engineers more effective.Responsibilities Include But Are Not Limited ToDesign, build, and maintain large-scale data pipelines supporting bidding, targeting, experimentation, and measurement. Optimize workflows under real-world constraints—sampling limits, narrow data windows, service ceilings, and high-throughput requirements. Ensure reliability, performance, and cost-efficiency across batch and streaming workloads. Build scalable solutions that deliver value today while creating a path for evolution over time. Own pipelines end-to-end, from design to deployment to maintenance. Diagnose, resolve, and eliminate recurring issues to strengthen system stability. Reduce duplicated workflows, wasted compute, and operational overhead. Establish engineering best practices around quality, testing, deployment, and observability. Partner closely with platform teams, infra teams, and other data/ML groups to unblock scaling constraints. Work directly with Data Science, ML Engineering, and Product to deliver features, unblock analysis, and support experimentation. Translate ambiguous requirements into well-structured engineering solutions. Operate with urgency in an environment where priorities shift quickly. Drive initiatives from zero to one with minimal guidance. Innovate within constraints and deliver solutions that work in the real world—not just on paper. Required Knowledge, Skills, and Abilities: Deep expertise in at least one major big data ecosystem (e.g., Spark, Flink, EMR, Hadoop-style systems). Strong proficiency with distributed compute frameworks (Spark strongly preferred). Experience building and operating pipelines at massive scale, including challenges like:Streaming ingestion High-volume batch ETL Strict SLAs or constrained windows Sampling and stateful processing Hands-on experience with cloud-based data infrastructure (AWS or equivalent), including:Object storage (e.g., S3) Data lake formats (Iceberg/Hudi-style tables) SQL engines (Redshift, Athena, Snowflake, etc.) Streaming frameworks (Kinesis or Kafka-style systems) Strong coding skills in Python, Java, and/or Scala. Experience with infrastructure-as-code frameworks (e.g., Terraform). Strong engineering discipline with Git, CI/CD, testing frameworks, and automated deployment processes. Exposure to advertising, experimentation, or ML-driven systems is highly beneficial:Real-time bidding / auction systems Feature pipelines or feature stores Measurement and attribution systems Online/offline ML model pipelines High ownership mindset: “I will figure it out and ship it.” Bias for action and comfort moving quickly with incomplete requirements. Strong ability to operate independently and deliver end-to-end solutions. Adaptability in dynamic environments with fast-changing priorities. Curiosity, learning speed, and enjoyment of solving hard scaling problems. Technology & Tooling Experience:Cloud & Compute: AWS, Kubernetes, Lambda, Step Functions Data Lake: S3, Iceberg/Hudi-like table formats Analytics: Redshift, Athena, EMR/Spark Batch & Orchestration: Spark, Airflow-style orchestration ML/Modeling: SageMaker or custom ML platforms Languages: Python, Java, Scala, SQL Infra & DevOps: Terraform, Git, CI/CD Compensation range for this position is $130,000 - $150,000/year DOE.Benefits and bonus information can be found at https://www.redcloudconsulting.com/careersRedCloud requires employees have the legal ability to work in the country hired during their employment period. During onboarding, proof of eligibility to work in the applicable country will be requested. RedCloud does not provide visa sponsorship.About UsAnchored in a foundation of integrity-based consulting, RedCloud brings together senior-level experts who work side by side with client teams to design and implement durable, high-impact solutions. Our approach emphasizes trust, accountability, and long-term value—balancing stability, growth, and innovation to help organizations operate securely and confidently in an increasingly complex global environment.RedCloud delivers a comprehensive range of services across its core practices: Data & AI, Delivery Excellence, Digital Solutions, Marketing & Sales, Privacy, Security, Data Center and Critical Infrastructure Security, and Strategy & Transformation. As our clients expand globally, so do we—providing consistent, expert support across regions while remaining agile, collaborative, and deeply invested in client success.Recognized for both performance and culture, RedCloud has been named #1 Best Company to Work For in Washington (Mid-Sized Businesses) by Seattle Business Magazine, a “Fastest Growing Company” multiple times on the Inc. 500/5000 list, and recognized by Consulting Magazine as a Leader in Technology Consulting.Visit http://www.redcloudconsulting.com/ for more info.