Data Engineer / Consultant

3-month Contract Start first week of November 2024 Remote Must have experience with Azure Data Factory & Synapse Azure Databricks Data Engineer (3-Month Engagement) We are seeking an experienced Data Engineer for a 3-month assignment. This role requires a strong understanding of data & analytics solution architecture, encompassing data warehousing, data lakes, ETL/ELT workload patterns, and related BI & analytics systems. Key Responsibilities: Develop and implement technical best practices for data ingestion, data quality, data cleansing, and other data integration/ETL/engineering-related activities. Manage and maintain a multi-terabyte enterprise data warehouse with incremental data pipelines. Build infrastructure for optimal extraction, transformation, and loading (ETL) of data from various sources using SQL and modern cloud technologies. Participate in meetings with key system owners to understand the current data and systems environments. Resolve source data issues and refine transformation rules, implementing data quality processes to assist with data cleansing requirements. Analyze source system data to define transformation logic and data quality through data profiling. Collaborate with technical and business representatives to determine strategies for handling data anomalies. Design ETL processes, develop source-to-target data mappings, integration workflows, and load processes. Develop, test, integrate, and deploy data pipelines using various tools and external programming/scripting languages as needed. Create technical documentation and other artifacts for data pipelines, ingestion, integration, or other data solutions. Identify problems, develop ideas, and propose solutions using analytical, evaluative, or creative thinking. Apply innovative thinking to identify possible reporting solution alternatives. Additional duties may be assigned based on project needs. Requirements and Qualifications: 3+ years of hands-on experience with the following data technologies: Azure Data Factory & Synapse Azure Databricks Lake-first approach in building data solutions On-premise data warehousing solutions ETL design and development, data marts, and star schemas Building data warehousing solutions in Azure Migrating data from on-premise to cloud environments Designing data warehouse solutions using Synapse or Azure SQL DB Building pipelines with Synapse or Azure Data Factory for data ingestion Understanding of integration run times available in Azure Lake-first approach solutions in Azure leveraging Databricks, pySpark, and delta format Advanced SQL skills with experience in relational databases and query authoring. Proficiency with scripting languages like Python, Pyspark, or Scala. Microsoft Azure Cloud platform certifications are a plus. Ability to travel to client locations as needed during the contract period. Note: This is a contract position for 3 months, with the possibility of extension based on project needs. RUwEFW4wcI