Data Solution Engineer - W2/Full Time
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
*Job Title: Data Solution Engineer*
*Duration - 6 months*
*Work Location: 100% remote*
*Job Description:*
- Implement data pipelines, infrastructure components, framework - Audit, Balance, Control; Security; Exception Handling etc. using Databricks
- Collaborate with Data Solution Designer to ensure cohesive integration between systems and data models.
- Analyze Business requirements, existing systems and provide a design guidelines that meet the business needs & Architecture principles.
- Implement a metadata driven framework for processing data.
- Implement Databricks optimization techniques.
- Implement batch and real time data pipelines.
- Build a scalable ingestion framework.
- Implement a common Error/Exception handling framework.
- Configure and Implement Control M, Astronomer or Airflow for scheduling and monitoring data pipelines.
- Configure and integrate with REST API gateway like APIGEE.
- Familiarity with Informatica (IICS), Network Data Mover (NDM), Axway Managed File transfer (MFT)
- Flexible in work schedule to collaborate and guide the offshore development team
Job Type: Contract
Schedule:
- 8 hour shift
Experience:
- Databricks deployment in Azure: 2 years (Preferred)
- integrating Databricks into the Azure ecosystem: 2 years (Preferred)
Work Location: Remote