AWS Data Engineer
Experience
Salary
Location
Job Description
JOB DESCRIPTION:
- AWS Data Engineer with minimum of 5 to 7 years of experience.
- Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines.
- Connect with stakeholders daily to discuss project progress and updates.
- Work within an Agile process to deliver projects in a timely and efficient manner.
- Design and develop Airflow DAGs to schedule and manage ETL workflows.
- Transform SQL queries into Spark SQL code for ETL pipelines.
- Develop custom Python functions to handle data quality and validation.
- Write PySpark scripts to process data and perform transformations.
- Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes.
- Run Spark jobs on AWS EMR cluster using Airflow DAGs.
- Monitor and troubleshoot ETL pipelines to ensure smooth operation.
- Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture.
- Collaborate with other members of the data engineering team to improve processes and implement new technologies.
- Stay up to date with emerging trends and technologies in data engineering and suggest ways to improve the team's efficiency and effectiveness.
Designation
AWS Data Engineer
Key Skills
Industry
IT-ITeS
Functional Area
IT Services
Employment Type
Full Time
Education
Bachelor Degree
Job Type
Work From Office
Related Jobs
Sales Executive
PMV Groups
Join the PMV Groups team as a Full Time Sales Executive on Erekrut, specializing in Sales Executive in the HR ...
3-6 Lakhs
4-6 Years
Delhi-New ..
Sales
Business Development Executive
Payfx Fintech Solutions
Join the Payfx Fintech Solutions team as a Full Time Business Development Executive on Erekrut, specializing i...
0-3 Lakhs
0-1 Years
Tamil Nadu..
Sales
HR Recruiter
FNB Financial Consultancy
Join the FNB Financial consultancy team as a Full Time HR Recruiter on Erekrut, specializing in HR Recruiter ...
0-3 Lakhs
0-1 Years
Tamil Nadu..
Human Resour...
Loan Sales
Ninjacart
Join the Ninjacart team as a Part Time Loan sales on Erekrut, specializing in Loan sales in the Agriculture...
0-3 Lakhs
1-3 Years
Tamil Nadu..
Agriculture
Field Sales Executive
Teamlease
Join the staffing team as a Full Time field sales exicutive on Erekrut, specializing in field sales exicutiv...
3-6 Lakhs
0-1 Years
Karnataka-..
Field Execut...
International Voice Process
Alvineient Consultant
Join the Tech mahindra team as a Full Time International voice process on Erekrut, specializing in Internation...
3-6 Lakhs
1-3 Years
Maharashtr..
Customer Sup...
.jpeg)
Senior Interior Designer
Homzinterio
Join the Homzinterio team as a Full Time Senior Interior Designer on Erekrut, specializing in Senior Interior...
3-6 Lakhs
1-3 Years
Karnataka-..
Interior Des...
Customer Support
Object IT
Join the Object IT team as a Full Time Customer Support on Erekrut, specializing in Customer Support in the HR...
0-3 Lakhs
0-1 Years
Karnataka-..
Customer Sup...