Data Engineer Training Data
OpenAI is hiring a Data Engineer to build and maintain the data pipelines that power our model training. You will develop systems for data collection, cleaning, deduplication, and quality assessment at massive scale.
Requirements:
5+ years of data engineering experience Strong skills in Python, SQL, and distributed data processing Experience with Apache Spark, Beam, or similar frameworks Knowledge of data quality and governance Experience with petabyte-scale data systems Nice to Have Experience with training data for language models Background in web crawling and text extraction Familiarity with data deduplication techniques Experience with data labeling and annotation platforms Unlimited PTO Free meals 401(k) match