AWS Data Engineer
Experience
Salary
Location
Job Description
JOB DESCRIPTION:
- AWS Data Engineer with minimum of 5 to 7 years of experience.
- Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines.
- Connect with stakeholders daily to discuss project progress and updates.
- Work within an Agile process to deliver projects in a timely and efficient manner.
- Design and develop Airflow DAGs to schedule and manage ETL workflows.
- Transform SQL queries into Spark SQL code for ETL pipelines.
- Develop custom Python functions to handle data quality and validation.
- Write PySpark scripts to process data and perform transformations.
- Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes.
- Run Spark jobs on AWS EMR cluster using Airflow DAGs.
- Monitor and troubleshoot ETL pipelines to ensure smooth operation.
- Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture.
- Collaborate with other members of the data engineering team to improve processes and implement new technologies.
- Stay up to date with emerging trends and technologies in data engineering and suggest ways to improve the team's efficiency and effectiveness.
Designation
AWS Data Engineer
Key Skills
Industry
IT-ITeS
Functional Area
IT Services
Employment Type
Full Time
Education
Bachelor Degree
Job Type
Work From Office
Related Jobs
Business Development Executive
Asesa Digital
Join the AsesaDigital team as a Full Time Business Development Executive on Erekrut, specializing in Business ...
0-3 Lakhs
0-1 Years
Maharashtr..
Business Dev...
Senior Data Scientist
Agivant Technologies
Join the Agivant Technologies team as a Full Time Senior Data Scientist on Erekrut, specializing in Senior Dat...
15-25 Lakhs
4-6 Years
Maharashtr..
Data Analyst
Business Development Associate
TWOSS Learning Private Limited
Join the TWOSS Learning Private Limited team as a Full Time Business Development Associate on Erekrut, specia...
3-6 Lakhs
0-1 Years
Haryana-Gu..
Business Dev...
Tele Debt Management
NAVI
Join the NAVI team as a Full Time Tele Debt Management on Erekrut, specializing in Tele Debt Management in the...
0-3 Lakhs
0-1 Years
Karnataka-..
Telecalling ...
Business Development Manager
Samaaro
Join the Samaaro team as a Full Time Business Development Manager on Erekrut, specializing in Business Develop...
6-10 Lakhs
1-3 Years
Karnataka-..
Accounting
Exclutive Job Opening
Weekdays
Join the Weekdays team as a Full Time Exclutive Job Opening on Erekrut, specializing in Exclutive Job Openin...
15-25 Lakhs
7-10 Years
Karnataka-..
BPO
Join the Accion labs team as a Full Time Angular Developer on Erekrut, specializing in Angular Developer in th...
10-15 Lakhs
11-15 Years
Karnataka-..
Information ...
Senior BDE
GREYTIP SOFTWARE PRIVATE LIMITED
Join GREYTIP SOFTWARE PRIVATE LIMITED as a Senior BDE in the IT-ITeS industry. This full-time role focuses on ...
6-10 Lakhs
1-3 Years
Tamil Nadu..
Sales