My name is Bartosz and since joining Kainos in April 2021, I have slightly shifted my career path. It is still all about data but instead of focusing on the business and consulting side, I am now responsible for the technical aspects- in short, I am now an engineer.
Of course, all this was possible without any problems thanks to a great initiative offered by Kainos called the Data Academy. I've shared some of my insights on my Data Academy experience below. You can also find out more about the 2022 Academy and apply today!
Why did I choose the Big Data Academy?
As I mentioned, I changed my career path, and as we all know it is often more difficult than it seems. Certainly, doing it the way we would like, i.e. with a well-thought-out and comprehensive training program for commercial work, we will not often find such opportunities.
The advertisement for the Big Data Academy fell into my hands by accident, but as soon as I read it carefully, I applied without thinking.
Everything matched my plans for the future: agile methodology, programming languages, the entire technological stack and opportunities for development and work in extremely interesting projects that come from the completion of the academy program. At the forefront of all this was a company with a great reputation on the Tri-City IT job market.
How was my Big Data Academy experience?
Together with a group of colleagues admitted to the academy- ranging from trainee to architect, giving attendees great opportunities for learning and exchanging experiences- we took part in intensive workshops on big data.
Each day was divided into lectures and practical classes during which we practiced the knowledge gained on a given day and on the previous days. The technologies we learned and had the opportunity to use while learning were, among others:
- Big data introduction: general knowledge aimed at allowing a broader understanding of the world of big data and related issues, such as HDFS, Data Storage, Transactions, Cloud Storage, Data Security etc.
- Databases: OLTP vs OLAP, SQL vs NoSQL, Azure SQL, Azure Cosmos DB, Snowflake
- Data streaming: Kafka, Azure Event Hub,
- ELK Stack: Elasticsearch, Logstash, Kibana,
- Spark: MapReduce, Yarn, RDD, Data Frames, Streaming, Batches, Distributed computing, Spark SQL, Spark tuning,
- Azure Data Factory: ETL, ELT, Data Flows, Pipelines, Linked Services, Triggers etc.
- Databricks
- Data Lakes
Learning in practice
Of course, the academy itself did not end with lectures and practical exercises designed for each of them. We also built our own big data project from scratch, and each participant (including those who had no commercial experience) could see what working in a "real" project looked like.
Our team worked in accordance with the Scrum principles, every day we had a short “daily meeting” where everyone could update what they worked on the previous day and tell the others what they will do today.
During our project, we received stream data from API providing us with stock data, which we read using Kafka technology and Azure EventHub. Then the data was sent or pulled to several places depending on its later destination (Azure Stream Analytics, Databricks, Table Storage and Blob Storage). Our main data warehouse was Azure Cosmos DB, from which the data was visualized directly in PowerBI. We also set up a fully automated connection called Snowpipe with the recently demanded data warehouse Snowflake, based on the data from it, and we also created visualizations directly in PowerBI.
A component of the Azure cloud that was not directly used in the project, but we also had the opportunity to test, was Azure Synapse Analytics.
Scala was the main programming language during our project, but we also used Python. I consider the use of both languages to be an additional advantage and an incentive to participate in the academy!
A high-level overview of Bartosz's Big Data Academy project.
Would I recommend the Big Data Academy?
I believe that the Big Data Academy is a great initiative and an unprecedented opportunity to develop and change your career path.
The entire program is tailored so as to convey as much useful knowledge as possible and then maximize its use, which certainly translates into actual, not just theoretical skills. In addition, Kainos provides super working conditions and an atmosphere that makes us want to get the most out of it.
Do I recommend taking part in the entire initiative? Certainly yes!
Although, as I mentioned for me, taking part started with accidentally noticing the recruitment advertisement. If I knew what I know now- I would have applied in a heartbeat.
If you have any questions about the Big Data Academy or my Kainos experience so far, you can easily find me on LinkedIn!