Engineer

Kafka+MemSQLMust Have Technical/Functional SkillsPrimary Skill: Kafka, Producers / Consumers, Kafka Topics / Partitions, Schema Registry, Event‐Driven Architecture, MemSQL / SingleStore, Real‐Time Data Pipelines, Distributed SQL, Oracle SQL, PL/SQL, Streaming Data, SQL OptimizationExperience: Minimum 10 yearsRoles & ResponsibilitiesDesign and implement event driven architectures on Kafka-model topics, partitions, and retention for high throughput, low latency data streams.Build resilient Kafka producers and consumers with proper serialization (Avro/JSON/Protobuf), batching, idempotence, and back pressure handling.Manage Kafka ecosystem components-brokers, ACLs, consumer groups, DLQs, Schema Registry, Connect, and MirrorMaker as required.Develop streaming data pipelines using Kafka Connect/Streams or custom apps to ingest, transform, and route data to MemSQL and Oracle targets.Design and optimize MemSQL (SingleStore) schemas-rowstore/columnstore choices, sharding, partitioning, pipelines, and indexes for real time analytics.Build MemSQL ingestion flows from Kafka (e.g., Pipelines) and tune for throughput, exactly-once or at least-once semantics, and failure recovery.Write high performance SQL for MemSQL and Oracle-window functions, analytics, joins, CTEs; tune queries using execution plans and indexing strategies.Develop Oracle PL/SQL packages, procedures, and functions for data transformation, validations, batch jobs, and business logic encapsulation.Ensure data quality and governance-schema evolution, contract testing, lineage tracking, and reconciliation between Kafka, MemSQL, and Oracle.Engineer reliability & error handling-retries with backoff, DLQ routing, replay strategies, idempotency keys, and compensating actions.Implement end to end security-TLS/mTLS for Kafka, SASL (SCRAM/Kerberos) if used; RBAC, secrets management, and data masking/encryption for databases.Monitor and optimize performance-lag, throughput, and consumer health in Kafka; resource utilization, shard balance, and query latency in MemSQL/Oracle.Automate CI/CD-infrastructure as code for Kafka topics/ACLs and DB objects; automated tests for streams, SQL, and PL/SQL; versioned migrations.Support production operations-on call triage, RCA for data delays or integrity issues, hotfixes, and continuous improvement of runbooks.Collaborate in Agile teams-refine requirements, size work, document data contracts and ERDs, and coordinate releases across platform and app teams.Salary Range- $110,000-$125,000 a year