Data - does it bite?

I’m not usually an adventurous person, but six years ago I decided to turn my life upside down, replied to a recruiter and fast-forward - I signed the contract with Kainos and relocated to Reading, from Poznań in Poland at that point. It was a small step back with the job - I had a bit more responsibility in my previous role, but the new job at Kainos looked great, and I had the opportunity to learn lots of new things, which was a big motivator for me.

It wasn’t always this way- shh! don't tell anyone... I haven’t always enjoyed attending countless training courses, especially back at university and on occasion where I was learning about things for the sake of knowing them. But the moment I see a tool or a technique that interests me, I fasten my seatbelt and drive 112 Km/h towards my new direction. (FYI - Motorways in England are slow 😟!). I wanted to share some insight into my career trajectory from engineering into data, and some of my experiences along the way.

My projects

In the following few years, I worked on a couple of projects for the UK government. They were all very different. Every time I started a new role, a lot of the time I learned from scratch! One month I was working on a project with over one-month release cycles, long-living branches. The following week I joined a project with Continuous Delivery, which blew my mind! Of course, I read about it beforehand, learned theory, and always wanted to see it in practice. It was so great to feel empowered, see the code on production in a matter of minutes after merging to the main branch. It was great to learn and experience this in practice. Indeed, it's not a silver bullet for all the problems in the world (you can't deploy a new version of code to a driving car, can you? 🤔), but it teaches us a lot of things, i.e. puts testing and automation on the top and shows why those things are so important.

Data - does it bite?

Last year, I started to feel the need to try something new. After my previous project, I knew it would be challenging to satisfy my hunger for new things and approaches, and I was ready for it. What could it be this time? I didn't want to leave Kainos, as I felt at home here, so I wanted to explore other options.

In the meantime, I got a new project opportunity. It was related to what I had done before. I was supposed to lead a team (which I enjoyed doing in the past couple of years), to move the data from an old system to a new one this time. It sounded interesting to me. At that point, data started to feel like my new target.

I started to learn new tools on the project, including a data integration tool I hadn't used before. I was like – ok, nice buttons and all, but how do you test it? How do you collaborate with others or do code reviews? I started to think, wouldn't it be better to write code to do simple data operations? I found the challenges and the intricacies of this project really fascinating. That was only the tip of the iceberg of the plethora of things awaiting me in the following months. And the introduction to a wealth of knowledge for me to explore and consume.

Big Data Academy

While working on that new project, I was contacted by one of my colleagues from a previous project to see if I would be interested in joining our Big Data Academy, and he sent me some materials to read more. The Big Data Academy is a ten person-academy organised by us delivered by outside lecturers, where a couple of new joiners and people changing sides (like me) learn current tools from the Big Data world. It starts with two-weeks of formal training, after which we have a small project to do and follow up with two weeks of self-study to obtain one of the data certifications. I couldn't say no! The opportunity landed at the perfect time for me.

There is a whole world waiting for me. A place where I can learn a lot and maybe give back my knowledge from the software engineering world. What principles can I apply? Which one works in data, and which doesn't? Is writing code in a Notebook a "new quick-feedback loop"? No, I can't write regular Java code in that time. I like this tool a lot more than the first one I had picked up!

The future?

What awaits me in the following months and years? How much of my previous knowledge and experience will I be able to use?

Although the tools seemed different initially, all the principles I have learnt and followed for years still apply! The lines and boxes in Azure Data Factory are JSON under the hood, and it supports GIT integration out of the hood, so we code review our pipelines (although it's not as straightforward as with regular code)! Our data tools have robust out of the box support for monitoring, so I feel already at home! I have encountered some challenges, though! For example, how can we test our pipelines properly? We all know that production data is a place where miracles and unexpected things happen. How can we prove our solution works when the data is central and the most important thing?

Next steps.. 

I look forward to all of the challenges the data world has in store for me. I am sure there are lots of other concepts I have learnt in the last years that will be relevant. Data at Kainos  - it’s an exciting place to be. We’re always looking for more people to join our team so maybe the next (unexpected) step in your career will be as interesting as mine was!

Check out our opportunities here.