Applicantz

As a Lead Data Engineer, you will build the foundations that enable data driven marketing and the digital transformation journey.. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Dbt and AWS 'big data' technologies.. We are looking for a candidate with 8+ years of experience in a Data Engineering role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.. 5+ years of working in dbt with knowledge on advanced dbt concepts like macros and Jinja templating.. Experience with AWS cloud services used often for data engineering including S3, EC2, Glue, Lambda, RDS, or Redshift

data engineer

VISA SPONSORSHIP IS NOT AVAILABLE. Our large, Fortune Technology client is ranked as one of the best companies to work with, in the world. As a global leader in 3D design, engineering, and entertainment software, they foster progressive culture, creativity, and a flexible work environment. They use cutting-edge technologies to keep themselves ahead of the curve. Diversity in all aspects is respected. Integrity, experience, honesty, people, humanity, and passion for excellence are some other adjectives that define this global technology leader. As a Lead Data Engineer, you will build the foundations that enable data driven marketing and the digital transformation journey. You will work with data strategists to understand the business requirements and execute the tactical aspects of delivering on the vision. Your team will drive multiple tool selection initiatives, ingesting data from different source systems to the data lake, Data Wrangling to enable downstream data consumers, developing systems & processes to improve data reliability, developing a comprehensive data catalog and training downstream data consumers. You would also be responsible for leading the deliverables of your juniors as well as taking important decisions relevant to platform architecture and Responsibilities<ul><li>Create and maintain optimal data pipeline architecture.</li><li>Assemble complex data sets that meet functional / non-functional requirements.</li><li>Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.</li><li>Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Dbt and AWS 'big data' technologies.</li><li>Build analytics tools that utilize the data pipeline to provide actionable insights into employee experience, operational efficiency, and other key business performance metrics.</li><li>Work with stakeholders to assist with data-related technical issues and support associated data infrastructure needs.</li><li>Build processes supporting data transformation, data structures, metadata, dependency and workload management.</li><li>Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS and Azure) and find ways to apply them back help their team</li><li>Work with data scientists and analysts to strive for greater functionality in our data systems.</li><li>Lead a team of data engineers and provide technical guidance and mentorship.</li></ul> Qualifications<ul><li>We are looking for a candidate with 8+ years of experience in a Data Engineering role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.</li><li>5+ years of hands-on experience in Snowflake</li><li>5+ years of working in dbt with knowledge on advanced dbt concepts like macros and Jinja templating.</li><li>4+ years of work experience in B2B Marketing Domain.</li><li>Advanced working SQL experience working with relational databases, query authoring (SQL) and working familiarity with a variety of databases.</li><li>Experience with scripting languages such as Python</li><li>Experience with big data tools such as PySpark</li><li>Experience with AWS cloud services used often for data engineering including S3, EC2, Glue, Lambda, RDS, or Redshift</li><li>Experience working with APIs to pull and push data.</li><li>Experience optimizing 'big data' data pipelines, architectures and data sets.</li><li>Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.</li><li>Able to read existing code to understand functionality and debug issues</li></ul> Preferred Qualifications<ul><li>Familiarity with Agile and SCRUM methodologies is a plus</li><li>Experience working with PowerBI to develop dashboards is a plus</li><li>Analytical skills related to working with unstructured datasets.</li><li>A successful history of processing value from large, disconnected datasets.</li><li>Experience working with agile, globally distributed teams</li></ul>

START NEW SEARCH