Data Scientist (Big Data Engineer) 2

I. DESCRIPTION OF SERVICESTexas Department of Family and Protective Services requires the services of 2 Data Scientist (Big Data Engineer) 2, hereafter referred to as Candidate(s), who meets the general qualifications of Data Scientist (Big Data Engineer) 2, Data/Database Administration and the specifications outlined in this document for the Texas Department of Family and Protective Services.All work products resulting from the project shall be considered "works made for hire" and are the property of the Texas Department of Family and Protective Services and may include pre-selection requirements that potential Vendors (and their Candidates) submit to and satisfy criminal background checks as authorized by Texas law. Texas Department of Family and Protective Services will pay no fees for interviews or discussions, which occur during the process of selecting a Candidate(s).The Worker is responsible for developing, maintaining, and optimizing big data solutions using the Databricks Unified Analytics Platform.This role supports data engineering, machine learning, and analytics initiatives within this organization that relies on large-scale data processing.Duties include:Designing and developing scalable data pipelinesImplementing ETL/ELT workflowsOptimizing Spark jobsIntegrating with Azure Data FactoryAutomating deploymentsCollaborating with cross-functional teamsEnsuring data quality, governance, and security.****DFPS NOT-TO-EXCEED RATE IS ***/HR****II. CANDIDATE SKILLS AND QUALIFICATIONSMinimum Requirements:Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity.Years Required/Preferred Experience4 Required Implement ETL/ELT workflows for both structured and unstructured data4 Required Automate deployments using CI/CD tools4 Required Collaborate with cross-functional teams including data scientists, analysts, and stakeholders4 Required Design and maintain data models, schemas, and database structures to support analytical and operational use cases4 Required Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses4 Required Implement data validation and quality checks to ensure accuracy and consistency4 Required Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging4 Required Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices4 Required Proficiency in Python and R programming languages4 Required Strong SQL querying and data manipulation skills4 Required Experience with Azure cloud platform4 Required Experience with DevOps, CI/CD pipelines, and version control systems4 Required Working in agile, multicultural environments4 Required Strong troubleshooting and debugging capabilities3 Required Design and develop scalable data pipelines using Apache Spark on Databricks3 Required Optimize Spark jobs for performance and cost-efficiency3 Required Integrate Databricks solutions with cloud services (Azure Data Factory)3 Required Ensure data quality, governance, and security using Unity Catalog or Delta Lake3 Required Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL3 Required Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake1 Preferred Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow)1 Preferred Databricks Certified Associate Developer for Apache Spark1 Preferred Azure Data Engineer Associate