Senior Data Engineer [T500-**]

  • hyderabad
Position Overview: Join the team that manages the end-to-end technology and life cycle management of the applications hosted both on premise and in the AWS cloud. We are looking for an experienced Software Engineering Advisor with more than 13 years of experience to join the Data and Analytics Engineering (D&AE) operational effectiveness team. The Senior Advisor will be responsible for ensuring the development of reports and data feeds using Databricks based platform. The ideal candidate will have a strong technical background and experience in creating & managing complex data queries and pipelines.

Responsibilities: Strong working knowledge of DataBricks with experience in Python and any cloud. Develop and manage report and data feeds involving complex queries. Perform root cause analysis on for data pipeline failures and data discrepancies. Work with business partners to resolve defects identified in a timely manner. Understand the requirements and use cases provided. Automation of application health checks and manual workarounds using python. Enable the self-healing capabilities to reduce the human interference, user impact and to increase the availability. Partner with other teams and stake holders for any clarifications needed for the timely deliverables. Ability to provide alternative solutions. Experienced with agile methodology. Responsible for the team deliverables

Qualifications Required Skills: Experience with large datasets management Experience in Visual Studio Code, Experience with Databricks CLI (command - line interface) to manage Databricks clusters, jobs, notebooks, Databricks File Systems Experience with data pipeline scheduling Experience with continuous integration and continuous delivery (CI/CD) tools like GitHub OR Jenkins. Excellent problem-solving skills and attention to details Strong communication and collaboration skills

Required Experience & Education: Bachelor’s degree in the related field. 13+ years of experience in data engineering field. 8+ years of hands-on development experience with Python 5+ years of experience with DataBricks 6+ years of experience in Any cloud.

Equal Opportunity Statement: It is an Equal Opportunity Employer actively encouraging and supporting organization-wide involvement of staff in diversity, equity, and inclusion efforts to educate, inform and advance both internal practices and external work with diverse client populations.