SENIOR DATA ENGINEER

Senior Data Engineer - We are a Purpose-driven company that exists to save lives and create a sustainable future for our clients, employees and communities. In pursuit of this Purpose, we put people at the centre of everything we do, and we are a consultancy with heart. The Role We are seeking an experienced Data Engineer with expertise in Azure Databricks to design, build, and maintain robust data pipelines and architectures. This role is critical in enabling our teams to derive actionable insights from complex data sets, supporting strategic decision-making and business transformation. Key Responsibilities Data Architecture and Pipeline Development: Design and implement scalable, efficient data pipelines and workflows using Azure Databricks and associated Azure tools. Optimize data ingestion, transformation, and storage for diverse data types (structured, semi-structured, and unstructured). Cloud Data Solutions: Build and manage cloud-based data environments, ensuring high availability, security, and performance. Integrate various data sources into unified platforms to support analytics and machine learning applications. Collaboration with Data Teams: Partner with data scientists, analysts, and stakeholders to ensure data solutions align with business goals. Facilitate smooth handoffs between engineering and data science teams for advanced analytics workflows. Performance Optimization: Monitor, troubleshoot, and optimize data pipelines for speed, reliability, and scalability. Apply best practices for code optimization, data partitioning, and cost efficiency in cloud environments. Data Governance and Quality: Implement data governance frameworks, ensuring compliance with security, privacy, and industry regulations. Develop and maintain data quality monitoring tools and processes. What You Bring Mandatory Expertise in Azure Databricks: Proven experience in designing, implementing, and optimizing solutions on the Azure Databricks platform. Programming Skills: Advanced knowledge of SQL, Python, PySpark or Scala for data engineering tasks. Big Data Frameworks: Experience with Apache Spark, Delta Lake, and similar technologies. Problem-Solving Ability: Demonstrated skill in debugging and resolving performance bottlenecks in data systems. Collaboration and Communication: Ability to work effectively with cross-functional teams and communicate technical concepts to non-technical stakeholders. Agile Practices: Experience working in agile environments with iterative delivery and continuous improvement methodologies. Preferred Qualifications Familiarity with CI/CD pipelines for data workflows. Understanding of data modeling and schema design principles. Experience with tools like Databricks Workflows, MLflow, or Azure DevOps. Let's see where our shared vision leads. We have a strong growth trajectory and a global footprint, which means unprecedented opportunity for the people on our teams to advance their career. As a Global Company, our professionals have the possibility to contribute to grow with the company. Every employee can influence and impact our company, and we value innovators and entrepreneurs. Beyond that, be yourself. At dss+, we embrace and celebrate diversity. Above all, we are an inclusive culture in which everyone contributes and has the chance to develop their talents further. #J-18808-Ljbffr