Data Integration Engineer :: Remote
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Data Integration Engineer Independant Visa Preferred Primary Responsibilities Integration Development: Design, build, and maintain integration pipelines connecting source systems to downstream targets, using containerized Python services on Azure Container Apps with Service Bus and Event Hub. Event-Driven and Streaming Integration: Build and operate flows against Salesforce's Pub/Sub API over gRPC, including Platform Event publish and subscribe, Avro decode, durable replay checkpointing, and reconnection handling for long-lived subscriptions. Reliability and Failure Handling: Implement idempotent processing, deduplication, retry policies, and dead-letter consumers so at-least-once delivery does not produce duplicate or lost records in target systems. API Design and Authentication: Design and consume REST APIs, including OAuth 2.0 with token caching, rate limit handling, and error semantics that distinguish retryable from terminal failures. Transformation and Business Logic: Build and maintain Python-based transformation and validation services that enforce business rules as data moves between systems. Data Platform Integration: Land request and response event streams into Snowflake via Event Hub as a durable parallel stream, so integration activity is available for reporting, reconciliation, and audit. Observability and Handoff to Operations: Instrument integrations for distributed tracing, structured logging, and correlation across asynchronous hops, and deliver monitoring hooks, alert thresholds, runbooks, and escalation paths to Operations before go-live. Testing, CI/CD, and Requirements: Write unit and integration tests covering failure and replay paths, maintain CI/CD pipelines (e.g., GitHub Actions) across DEV, SIT, UAT, and PROD, and translate stakeholder requirements into technical designs, including which pattern applies.