Manager - Data Engineering

GM Financial Technology Innovation isn't just a talking point at GM Financial, it's how we operate. From generative AI and cloud-native technologies to peer-led learning and hackathons, our tech teams are building real solutions that make a difference. We're committed to AI-powered transformation, using advanced machine learning and automation to help us reimagine customer interactions and modernize operations, positioning GM Financial as a leader in digital innovation within a dynamic industry. Join us and discover a workplace where your ideas matter, your development is prioritized, and you can truly make a global impact.Work Arrangement: Hybrid – 2 days onsite, 3 days remote per weekResponsibilities About the role:We are expanding our efforts into complementary data technologies for analytics and decision support in areas of ingesting and processing large data sets. Our interests are in enabling data science and search based applications on large and low latent data sets in both a batch and streaming context for processing. To that end, this role will engage with team counterparts in exploring, developing and deploying technologies for creating data sets using a combination of batch and streaming transformation processes. These data sets support both off-line and in-line machine learning training and model execution. Other data sets support search engine based analytics. Exploration and deployment of technologies activities include identifying opportunities that impact business strategy, selecting data solutions software, and defining hardware requirements based on business requirements. Responsibility also includes coding, testing, and documentation of new or modified scalable analytic data systems including automation for deployment and monitoring. This role participates along with team counterparts to architect an end-to-end framework developed on a group of core data technologies. Other aspects of the role include developing standards and processes for data engineering projects and initiatives.What makes you a dream candidate?Evaluate, research, experiment with data engineering technologies in a lab to keep pace with industry innovation while assessing business impact and viability for use cases associated with efforts in handWork with data engineering related groups to inform on and showcase capabilities of emerging technologies and to enable the adoption of these new technologies and associated techniquesDefine and refine processes and procedures for the data engineering practiceWork closely with data scientists, data architects, ETL developers, other IT counterparts, and business partners to identify, capture, collect, and format data from the external sources, internal systems, and the data warehouse to extract features of interestCode, test, deploy, monitor, document, and troubleshoot data engineering processing and associated automationDefine data engineering architecture both hardware and software reflective of business requirements to be included in end-to-end solution architectureEducate and develop ETL developers on data engineering so as to enable transition to data engineer and practiceConduct code reviews, suggest improvements, support technology upgrades for the common libraries, handover them to the corresponding development teams for quality check and support them till deployment into productionSupport ETL developers and Operations teams to troubleshooting of the incidents for root cause analysis and assist in solutioning to meet the service level agreementsWork with Operations teams in Big Data, IT and Information Security with monitoring and troubleshooting of incidents to maintain service levelsContribute to the evolving distributed systems architecture to meet changing requirements for scaling, reliability, performance, manageability, and costReport utilization and performance metrics to user communitiesContributes to planning and implementation of new/upgraded hardware and software releasesResponsible for monitoring the Linux, Hadoop, and Spark communities and vendors and report on important defects, feature changes, and or enhancements to the teamResearch and recommend innovative, and where possible, automated approaches for administration tasksIdentify approaches to efficiencies in resource utilization, provide economies of scale, and simplify support issuesQualifications Have Other Skills? We Use These TooStrong working knowledge of Hadoop and Spark cluster security, networking connectivity and IO throughput along with other factors that affect distributed system performanceStrong working knowledge of disaster recovery, incident management, and security best practicesWorking knowledge of containers (e.g., docker) and major orchestrators (e.g., Mesos, Kubernetes, Docker Datacenter)Working knowledge of automation tools (e.g., Puppet, Chef, Ansible)Working knowledge of software defined networkingWorking knowledge of parcel based upgrades with Hadoop (i.e., Cloudera)Working knowledge of hardening Hadoop with Kerberos, TLS, and HDFS encryptionWorking knowledge with directed analytic graph stream processing using Beam, Flink, Nifi and/or SamzaExcellent knowledge of Linux, AIX, or other Unix flavorsWorking knowledge of Cloud based implementations (e.g., Microsoft Azure) with emphasis on security using ACLs and Artifactory GroupsAbility to accept change and to adapt to shifting organizational challenges and prioritiesAbility to coach, develop and lead othersAbility to evaluate problems and issues quickly, and to make recommendations for courses of actionAbility to make independent decisions and use sound judgment in relation to the management of team membersAbility to prioritize tasks and ensure their completion in a timely mannerExcellent analytical and troubleshooting skillsStrong interpersonal, verbal and written skillsWork Experience5-7 years experience with software engineering to include Java, Scala, and Python required5-7 years proficiency with processing large data sets with Kafka, RabbitMQ, Flume, Hadoop, HBase, Cassandra and/or Spark or similar distributed system required3-5 years hands-on experience with scripting with Bash, Perl, Ruby required3-5 years hands-on development / processing experience on Kafka, HBase, Solr, and Hue required2-4 years hands-on experience with ETL and Business Intelligence technologies such as Informatica, DataStage, Ab Initio, Cognos, BusinessObjects, or Oracle Business Intelligence required2-3 years hands-on experience with SQL, data modeling, and relational databases such as Oracle, DB2, and Postgres requiredProven track record with NoSQL data stores such as MongoDB, Cassandra, HBase, Redis, Riak or other technologies that embed NoSQL with search such as MarkLogic or Lily Enterprise required0-2 years management experience with data engineering team preferredHigh School Diploma or equivalent requiredBachelor's Degree in related field or equivalent work or military experience requiredAdditional Knowledge and SkillsWorking effectively within an AI enabled environment:Ability to use AI tools (e.g., Microsoft Copilot) to support daily workSkills in evaluating AI outputs for accuracy, compliance, and biasExperience integrating AI into workflows to improve efficiency or insightsFamiliarity with AI assisted research, summarization, and content generationUnderstanding of responsible AI use, including ethics and data protectionExperience with AI assisted software development or automationKnowledge of prompting techniques to improve output qualityAwareness of emerging GenAI capabilities and limitationsWhat We Offer: Generous benefits package available on day one to include: 401K matching, bonding leave for new parents (12 weeks, 100% paid), tuition assistance, training, GM employee auto discount, community service pay and nine company holidays.Our Culture: Our team members define and shape our culture — an environment that welcomes innovative ideas, fosters integrity, and creates a sense of community and belonging. Here we do more than work — we thrive.Compensation: Competitive pay and bonus eligibility.Work Life Balance: Flexible hybrid work environment, 2-days a week in office.