Although there is no exact date for its creation, it can be said that the term Big Data began to be widely used in the mid-1990s and early 2000s. It is incredible how in just two decades, Big Data has evolved in terms of tools, technologies, data analysis methods, and even the professional profiles that work with it. The technology market is advancing at an unstoppable pace.
As a recruiter, I have witnessed firsthand how Big Data has transformed the way companies manage and utilise their data. But above all, I have seen how hiring needs in companies have been changing and, consequently, how professional profiles have been evolving.
I still remember when I took my first steps in the world of recruitment. By some stroke of luck, I have always been involved in selection processes for data analysis departments, which has allowed me to experience every change. It’s amazing how much everything has changed in a decade. And when the market transforms so quickly, one must study, investigate, and adapt to the changes in order not to be left behind.
From Database Administrators and Data Analysts…
In the past, the focus was on hiring database administrators, professionals with skills in storing, cleaning, and processing data. At some point, I can’t remember when exactly, the role of Data Analyst emerged, where in addition to technical knowledge, they needed to have a more analytical profile and be able to use visualization tools to make the numbers understandable to everyone. This profile sparked a revolutionary shift.
Data analysts were the key professionals in the field of Big Data. These profiles had strong technical skills, such as database knowledge, SQL, and data visualization tools. Their main function was to extract, clean, and transform data for analysis and presentation. However, as data volumes increased and became more complex, it became clear that more advanced skills were needed to harness the full potential of Big Data.
…to Data Scientists
It is in this context that the role of the Data Scientist emerged. Data Scientists are multidisciplinary professionals who combine technical skills, programming expertise, knowledge of mathematics and statistics, and experience in machine learning. In other words, suddenly companies no longer asked me to find a professional with computer skills and knowledge of SQL. Now they needed individuals capable of using advanced analytical techniques that went beyond basic descriptive analysis. Suddenly, new concepts appeared in my dictionary: algorithms, automations, predictions… My goodness, no one told me when I studied Political Science or Human Resources Management that I would have to learn all this new vocabulary. I suppose that justifies why I ultimately decided to study Data Analysis. If you can’t beat the enemy, better join them. Or at least that’s what people say.
And just when you think you know everything about the world of data, new terms start to emerge.
And then came Business Intelligence…
Sometimes, it seems like trends even reach the recruitment processes. I remember the first time I heard the term Business Intelligence. Technically, it wasn’t a new term, but it arrived as a trend that was here to stay. While data scientists delve into the depths of data, Business Intelligence professionals are the guides who show us the way. They are the masters of visualisation tools and data analysis geared towards decision-making.
Overnight, recruiters learned about tools like Tableau, Power BI, or QlikView. And the best part… we realised that we could use them too. The data market had been liberalised, no longer requiring one to be a computer engineer to work with data. It’s interesting to see how the perception of companies has changed regarding the importance of data handling. Nowadays, there are even strategic departments analysing data in human resources, marketing, and more.
But that isn’t all. Every year, new concepts emerge: Data Engineering, Data Governance, Machine Learning. In such a dynamic field, challenges and learning opportunities are always present.
The Future
Every year, new tools focused on data analysis, new ways of leveraging it, and new business needs emerge in the market. In the fascinating world of Big Data, where technological advancements and new analytical techniques emerge at a rapid pace, adaptability and continuous learning have become vital skills for professionals in this field.
As a recruiter, I have firsthand witnessed the importance of studying and adapting to stay ahead. Profiles that were once highly sought after can quickly become obsolete due to technological advancements or changes in market demands. That’s why I believe it is essential to never stop studying or learning, in order to understand the market and your clients.
Additionally, it is crucial to seek candidates who not only have current skills and knowledge but also have the ability to adapt and learn new tools, techniques, and concepts that the future will demand from them.
Conclusion
It has been a decade since I started working as a recruiter, searching for data analysis profiles. And it almost seems like all those roles I work with will soon become obsolete again. Perhaps, in a short time, recruiters will be seeking professionals who will exclusively work in Artificial Intelligence, Spatial Data Mining, or learning algorithms for nanobots. It sounds like science fiction, but the truth is that many are pointing towards it being the future. In the face of this imminent future, adaptability and continuous learning will be crucial characteristics for both recruiters and candidates.