Data Engineer
Data Engineer – Job Description
Job Summary:
A Data Engineer is responsible for designing, building, and maintaining data pipelines and data infrastructure to collect, process, transform, and store large volumes of data for analytics and business reporting.
Key Responsibilities:
Design, develop, and maintain scalable ETL/ELT data pipelines.
Extract data from multiple sources, transform it, and load it into data warehouses or data lakes.
Write and optimize SQL queries for data processing and reporting.
Develop data processing scripts using Python.
Work with big data technologies such as Apache Spark and PySpark.
Build and maintain data warehouses using platforms such as Snowflake, Amazon Redshift, or Google BigQuery.
Work with cloud platforms such as AWS, Azure, or Google Cloud.
Ensure data quality, accuracy, security, and reliability.
Monitor pipeline performance and troubleshoot data processing issues.
Collaborate with data analysts, data scientists, and business teams.
Required Skills:
- SkillTechnologiesProgrammingPython, SQLETL/ELTInformatica, dbt, or Apache Airflow
- Big DataApache Spark, PySpark, HadoopCloudAWS, Azure, or GCPDatabasesMySQL, PostgreSQL, SQL ServerData Warehousing
- Snowflake, Redshift, BigQueryVersion Control
- Git, GitHubData Modeling
- Fact tables, dimension tables, star schema