{"schemaVersion":"jobsearcher.job.v1","id":"de64426022a1c72f42034373","url":"https://jobsearcher.com/jobs/de64426022a1c72f42034373","canonicalUrl":"https://jobsearcher.com/jobs/de64426022a1c72f42034373","title":"Data Engineer with AI - Remote","description":"I hope you are doing well, Please share your updated profile if you are interested in the below role. Our client seeks an Data Engineer + AI for a 12 Months project in Boston, MA. Below is the detailed requirement Job Title: Data Engineer + AI Work location : Boston, MA Duration: 12 Months Job Summary: We're looking for a Senior Data Engineer to build and scale our lakehouse and AI data pipelines on Databricks. You'll design robust ETL/ELT, enable feature engineering for ML/LLM use cases , and drive best practices for reliability, performance, and cost. What you'll do Design, build, and maintain batch/streaming pipelines in Python + PySpark on Databricks (Delta Lake, Autoloader, Structured Streaming). Implement data models (Bronze/Silver/Gold), optimize with partitioning, Z-ORDER, and indexing, and manage reliability (DLT/Jobs, monitoring, alerting). Enable ML/AI: feature engineering, MLflow experiment tracking, model registries, and model/feature serving; support RAG pipelines (embeddings, vector stores). Establish data quality checks (e.g., Great Expectations), lineage, and governance (Unity Catalog, RBAC). Collaborate with Data Science/ML and Product to productionize models and AI workflows; champion CI/CD and IaC. Troubleshoot performance and cost issues; mentor engineers and set coding standards. Must-have qualifications 6-10+ years in data engineering with a track record of production pipelines. Expert in Python and PySpark (UDFs, Window functions, Spark SQL, Catalyst basics). Deep hands-on Databricks : Delta Lake, Jobs/Workflows, Structured Streaming, SQL Warehouses; practical tuning and cost optimization. Strong SQL and data modeling (dimensional, medallion, CDC). ML/AI enablement experience: MLflow , feature stores, model deployment/monitoring; familiarity with LLM workflows (embeddings, vectorization, prompt/response logging). Cloud proficiency on AWS/Azure/GCP (object storage, IAM, networking). CI/CD (GitHub/GitLab/Azure DevOps), testing (pytest), and observability (logs/metrics). Nice to have Databricks Delta Live Tables , Unity Catalog automation, Model Serving. Orchestration (Airflow/Databricks Workflows), messaging (Kafka/Kinesis/Event Hubs). Data quality & lineage tools (Great Expectations, OpenLineage). Vector DBs (FAISS, pgvector, Pinecone), RAG frameworks (LangChain/LlamaIndex). IaC (Terraform), security/compliance (PII handling, data masking). Experience interfacing with BI tools (Power BI, Tableau, Databricks SQL).","company":"Lorven Technologies","rawCompany":"lorven technologies","isRemote":true,"isActive":false,"createdAt":"2026-08-18T02:36:18.013Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer with AI - Remote","description":"I hope you are doing well, Please share your updated profile if you are interested in the below role. Our client seeks an Data Engineer + AI for a 12 Months project in Boston, MA. Below is the detailed requirement Job Title: Data Engineer + AI Work location : Boston, MA Duration: 12 Months Job Summary: We're looking for a Senior Data Engineer to build and scale our lakehouse and AI data pipelines on Databricks. You'll design robust ETL/ELT, enable feature engineering for ML/LLM use cases , and drive best practices for reliability, performance, and cost. What you'll do Design, build, and maintain batch/streaming pipelines in Python + PySpark on Databricks (Delta Lake, Autoloader, Structured Streaming). Implement data models (Bronze/Silver/Gold), optimize with partitioning, Z-ORDER, and indexing, and manage reliability (DLT/Jobs, monitoring, alerting). Enable ML/AI: feature engineering, MLflow experiment tracking, model registries, and model/feature serving; support RAG pipelines (embeddings, vector stores). Establish data quality checks (e.g., Great Expectations), lineage, and governance (Unity Catalog, RBAC). Collaborate with Data Science/ML and Product to productionize models and AI workflows; champion CI/CD and IaC. Troubleshoot performance and cost issues; mentor engineers and set coding standards. Must-have qualifications 6-10+ years in data engineering with a track record of production pipelines. Expert in Python and PySpark (UDFs, Window functions, Spark SQL, Catalyst basics). Deep hands-on Databricks : Delta Lake, Jobs/Workflows, Structured Streaming, SQL Warehouses; practical tuning and cost optimization. Strong SQL and data modeling (dimensional, medallion, CDC). ML/AI enablement experience: MLflow , feature stores, model deployment/monitoring; familiarity with LLM workflows (embeddings, vectorization, prompt/response logging). Cloud proficiency on AWS/Azure/GCP (object storage, IAM, networking). CI/CD (GitHub/GitLab/Azure DevOps), testing (pytest), and observability (logs/metrics). Nice to have Databricks Delta Live Tables , Unity Catalog automation, Model Serving. Orchestration (Airflow/Databricks Workflows), messaging (Kafka/Kinesis/Event Hubs). Data quality & lineage tools (Great Expectations, OpenLineage). Vector DBs (FAISS, pgvector, Pinecone), RAG frameworks (LangChain/LlamaIndex). IaC (Terraform), security/compliance (PII handling, data masking). Experience interfacing with BI tools (Power BI, Tableau, Databricks SQL).","datePosted":"2026-08-18T02:36:18.013Z","dateModified":"2026-08-18T02:36:18.013Z","hiringOrganization":{"@type":"Organization","name":"Lorven Technologies","sameAs":"https://jobsearcher.com"},"jobLocationType":"TELECOMMUTE","applicantLocationRequirements":{"@type":"Country","name":"US"},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"de64426022a1c72f42034373"},"url":"https://jobsearcher.com/jobs/de64426022a1c72f42034373"}}