SENIOR DATA ENGINEER
Experience
Salary
Location
Job Description
Roles and Responsibilities
- Design and develop scalable and efficient data pipelines using Azure Data Factory, Azure Databricks, Kafka, and other relevant tools.
- Extract data from various applications using APIs, webhooks, or other methods.
- Ingest and process real-time data streams using Kafka.
- Transform and clean data to ensure data quality and consistency.
- Load data into ADLS Gen2 in optimized formats (e.g., Parquet, Delta Lake).
- Build and maintain data models and schemas.
- Collaborate with data analysts and business stakeholders to understand data requirements and deliver actionable insights.
- Optimize data pipelines for performance and cost-efficiency.
- Implement data governance and security best practices.
- Mentor junior data engineers and provide technical guidance.
Skills and Qualifications
- Strong proficiency in Python, Scala, or Java programming languages.
- Expertise in data engineering tools and technologies such as Azure Data Factory, Azure Databricks, ADLS Gen2, Kafka, and Delta Lake.
- Experience with cloud computing platforms, preferably Azure.
- Knowledge of data modelling, ETL/ELT processes, data warehousing, and real-time data processing concepts.
- Strong problem-solving and analytical skills.
- Ability to work independently and as part of a team.
- Excellent communication and interpersonal skills.
- Bachelor's degree in computer science, engineering, or a related field.
Preferred Qualifications
- Knowledge of machine learning and AI concepts.
- Certifications in Azure or related technologies.
- Experience with Node.js, Java or .NET Core for developing connector APIs.
Must-Have Technologies
- Python
- Scala or Java
- Azure Data Factory
- Azure Databricks
- ADLS Gen2
- Kafka
- Delta Lake
- SQL
This role offers a unique opportunity to work on challenging data engineering projects and contribute to the growth of our data-driven organization.
Designation
SENIOR DATA ENGINEER
Key Skills
Industry
Software Development
Functional Area
Technology, Information and Internet
Employment Type
Full Time
Education
Bachelor Degree
Job Type
Work From Office
Related Jobs
Field Sales
LIVE CONNECTIONS PLACEMENTS PVT LTD
Join the Live Connections Placements Pvt Ltd team as a Full Time Field Sales on Erekrut, specializing in Field...
3-6 Lakhs
0-1 Years
Tamil Nadu..
Sales
Azure DevOps, Scripting , Co
HR Recruiter
Join the contract team as a Full Time Azure DevOps, scripting , co on Erekrut, specializing in Azure DevOp...
10-15 Lakhs
4-6 Years
Karnataka-..
Computer and...
Medical Representative
Inex Medicaments Private Limited
Join the Inex Medicaments Private Limited team as a Full Time Medical Representative on Erekrut, specializing ...
0-3 Lakhs
0-1 Years
Tamil Nadu..
Sales & ...
Home Loan Sales Executive
LIVE CONNECTIONS PLACEMENTS PVT LTD
Join the banking industry team as a Full Time Home loan Sales executive on Erekrut, specializing in Home loa...
0-3 Lakhs
1-3 Years
Tamil Nadu..
Sales & ...
Senior Resident & Professo...
Quench Techno
Join the Quench Techno team as a Full Time Senior Resident & Professor on Erekrut, specializing in Senior Resi...
Confidential
0-1 Years
Tamil Nadu..
Healthcare
MuleSoft Platform Admin
2 COMS
Join the 2 COMS team as a Full Time MuleSoft Platform Admin on Erekrut, specializing in MuleSoft Platform Admi...
15-25 Lakhs
4-6 Years
Tamil Nadu..
IT / Softwar...
Sales Executive
HEPL
Join the Hepl team as a Full Time Sales Executive on Erekrut, specializing in Sales Executive in the HR Serv...
3-6 Lakhs
0-1 Years
Tamil Nadu..
Sales
Customer Support
LIVE CONNECTIONS PLACEMENTS PVT LTD
Join the Live Connections Placements Pvt Ltd team as a Full Time Customer Support on Erekrut, specializing in ...
0-3 Lakhs
0-1 Years
Tamil Nadu..
Customer Sup...