Lead Data Engineer (London)

Available in: London

Apply by: 31 December 2018

Minimum (essential) requirements:

  • Software development experience with distributed data storage and processing technologies including Hadoop and Spark and using JVM languages.
  • Experience of leading the development of substantial components for large-scale data processing solutions, taking responsibility for non-functional needs of ETL/ELT data processing pipelines such as robustness, performance and security. ‘Development’ incorporates design, code, test defect resolution and operational readiness, and includes setting the standards for these activities.
  • Coaching and mentoring experience in data development disciplines with the ability to gain the respect of a junior team.
  • Ability to advise architects and other stakeholder on detailed technology and development practice and on development estimates.
  • Ability to make effective decisions within fast-moving Agile delivery and to lead on troubleshooting.
  • Strong understanding of tools, design methodologies and best practice.
  • Ability to clearly communicate technical design both written and verbally.


  • Software development experience with Cloudera’s distribution of Apache Hadoop and with Python.
  • Experience of data visualisation and data complex data transformations, including ETL tools such as Talend.
  • Able to productionise machine learning algorithms.
  • Understanding of text processing including Natural Language Processing.
  • Experience with steaming and event-processing architectures including technologies such as Kafka and change-data-capture products.
  • Data modelling experience with data storage technology, such as document, graph, log stores and other non-relational platforms.
  • Open source contributon

Who you are:

Our vision is to enable outstanding people to create digital solutions that have a positive impact on people’s lives. Our values aren't abstract; they are the behaviours we expect from each other every day, and underpin everything that we do. We expect everyone to display our values by being determined in how obstacles are overcome; honest when dealing with others; respectful of how you treat others; creative to find solutions to complex problems and cooperative by sharing information, knowledge and experience. 

These values, applied collectively, help to produce an outstanding Kainos person, team and culture. 

About us 

Kainos is a high-growth IT services company providing digital technology solutions and agile software development to enterprise customers. Across our 30-year history, we have worked on transformational projects across government, NHS and a myriad of private sector clients.  

We’re an inclusive bunch at Kainos and we believe in equal opportunity. We welcome applications from all walks of life, including those with criminal records. Everyone who is offered a position here undergoes a background check, however no one is treated differently due to their background where it isn’t relevant to the role. It’s important to note – failure to reveal information which is relevant to the position sought, could result in us having to withdraw the conditional offer of employment.

In short: Having a criminal record will not exclude you from a career with Kainos. This depends entirely on the nature and requirements of the position, together with the circumstances and background of offences.