Senior Data Engineer

  • Bengaluru
  • Ecolab
  • Minimum of 4 years of experience in a Data Engineering role, with a focus on Azure data services.
  • Expertise in Azure Data Factory (ADF) and designing ETL/ELT data pipelines.
  • Proficiency in Azure Databricks for big data processing, analytics, and machine learning workflow implementations.
  • Experience maintaining and extending production data systems to evolve with complex needs.
  • Familiarity with Unity Catalog and its integration with Azure Databricks for data governance.
  • Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals
  • Design and deployment of performant end-to-end data architectures
  • Strong understanding of SQL, Python, Java and/or Scala for data manipulation and transformation.
  • Experience with Azure services such as Azure Data Lake Storage, Azure Synapse Analytics, and Azure SQL Database.
  • Knowledge of data modeling, warehousing, and data lake patterns and practices in Azure.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work independently and collaboratively in a fast-paced and dynamic environment.