“Like it or not, we live interesting times” Robert Kennedy said to South African students graduating from Capetown University back in 1966.

In 2024, we indeed enjoy both the pleasure and the privilege of living in extraordinary interesting times! Intelligent robots are relieving the burden of dull, difficult and dangerous tasks. The Internet of Things has transformed our world from disconnected offline islands to a thriving connected society. And the advent of Artificial Intelligence and Big Data unlock new avenues with promising potential and sheer unlimited possibilities.

I must admit, I grew up as a digital dinosaur, in an age where computers were offline machines and cell phones were still used to call your parents. Growing up along that meandering journey between aspiration, audacity and What Life Answers You, I have witnessed the lightning speed revolutions of internet, social media and robotics with my own eyes. More recently, I have welcomed the advent of Artificial Intelligence and the Big Data explosion with delight and amazement. But as with every new innovation, these game-changing digital technologies can be both a blessing and a burden.

As the name suggests, Big Data is first and foremost Big – in fact, tremendously huge . The on-board computer of Apollo 11 – the very spacecraft that put a man on the moon – had a memory of merely 46 kilobytes. Meanwhile, the search engines that have become our shared memory process well over 100 petabytes (!) each day. Every week, we generate more data than our parents did during an entire generation. Big Data is not only huge, but also incredibly fast. Hypermarkets monitor more than a thousand transactions per second, and every single hour we all collectively upload more than 10 million selfies and roughly 50 000 hours of home-grown video footage on social media platform.

In this day and age, we are all flooded with a deluge of data. Hence: Big Data is big and fast, but also chaotic. The data we are emitting is unstructured, and often downright messy. We are left to guess how to address Peter Antoine by his first name, and it remains unclear whether his birthday is on April 3^rd or March 5^th. And so big data is a mess, flooding us in cosmic quantities and at the speed of light.

Data is the New Oil

And yet, (big) data provides us with unprecedented opportunities. In recent years, the exponential development of Artificial Intelligence has paved the way for Big Data Mining. We have succeeded in constructing intelligent machines, that seem to be capable of educating themselves and identifying unexpected patterns in this data avalanche.

As a result, data is becoming the new oil of our connected society. It should be seen as a reservoir filled to the brim with a precious resource that has the potential of unlocking previously unseen added value. Provided, of course, that we know where to drill for data, and that we have access to the tools to distil meaningful insights from that raw data.

If we understand how to read Big Data, we can start to harness its astonishing predictive power. Our tweets turn out to be a very strong leading indicator to predict the box office success of new movies. Our queries in search engines enable fertile minds to anticipate dynamics on the real estate market. A couple of years ago, tracing strings such as “coughing”, “difficult breathing” and “lung problems” allowed Internauts to also follow the worldwide spread of the COVID-19 pandemic almost in real-time. We can only begin to image the potential of Big Data Mining when its brute force is released on targeted topics such as healthcare, climate change or customer experience.

Correlation beats causality

Unsurprisingly, Big Data Mining has found its way to customer experience as one the first target markets. One of the bigger US hypermarket chains has aggregated vast silos of data, and started crossing that new ‘oil reservoir’ with other databases such as the local weather forecast. From this judicious cross-fertilization of data sources, most interesting correlations between product purchases could be derived. When heavy weather was expected, the sales of umbrellas and flashlights soared. I think it is a safe bet to state that you do not need artificial intelligence to come to such conclusions. However, during hurricane season, the hypermarket also observed a statistically significant inclination in the preferred cereal for breakfast. It is extremely difficult –if not impossible- to explain such a pattern, but it surfaces nonetheless. By exploiting this surprising insight, the retailer managed to increase sales of cereal sevenfold, just by filling the shelves based on the weather forecast. And by trusting the findings of an algorithm they could no longer understand.

Big Data embraces correlation and invites us to abandon our longing for causality. As an engineer, I still find it hard to relax the requirements that promise that consequences follows their causes. That may not be hard to accept when it comes to the taste of cereal. But what if an AI system tells you that you have a very high chance of developing a fatal long tumor, and then suggests (based on an opaque algorithm that we do not comprehend) to remove your left lung as a preventive mitigation measure?!

Emission of Data makes us Transparent

Breakfast cereals show us a glimpse of what Big Data Mining can offer, but also force us to contemplate the drawbacks. The data that we are all – often carelessly – emitting makes us transparent. And it enables data brokers and online market players to profile us with laser-sharp accuracy. We are illuminated by obscure algorithms as consumers, but also as humans. A quick online search will reveal sobering stories of algorithmic cruelty. A young lady that suffered a miscarriage, but continues to be haunted by advertisements for baby bottles, cradles and colourful clothes for newborns. A grieving father that sees the smile of his deceased son as the headline of his feed that shows his “Last Year in Review!”.

Most of us have a loyalty card at our local supermarket or grocery store. We think we a enjoy a modest discount, but we often fail to acknowledge that the card acts as a magnifying glass. In addition, online instruments such as social media platforms and search engines are seemingly free, but harvest our likes, preferences and whereabouts. In our connected society, our personal and often intimate data has become a currency. More than a decade ago, we already started paying with our privacy to indulge in online entertainment.

Private Data provides Purchasing Power

It begs the question: is privacy an outdated concept, or even an illusion, in our digital world? Or is it  a fundamental right that requires a new regulatory framework to safeguard our identity in the Internet of Things? The answer depends, as often, to whom you ask the question. In China, where face recognition traces (and scores!) you throughout the day, privacy is utopian. A US citizen is more likely to perceive personal data as a raw resource that can be freely traded. In Europe, a continent that still bears the marks of two World Wars, respect and dignity for the individual has a higher moral weight than the liberal laws of the open market. The General Data Protection Regulation (GDPR) imposed by the European Commission is a testament of EU’s to commitment to protecting online consumers.

Still: the internet has no borders, and digital data travels through fibre optics at the speed of light. It is a poorly kept secret that legislation almost always lags the introduction of new technology. And it turns out it is ridiculously easy to pinpoint individuals in aggregated data sets which are assumed to be ‘anonymized’. Therefore, enforcing privacy in an online world is by no means an obvious task. This is why I would like to make a warm plea to be(come) conscious consumers when shopping online. If we engage in digital activities, we expose ourselves to the merits – but also the limits – of our Connected Society. We should be acutely aware of the data we are emitting, the value of our personal preferences, and the purchasing power it grants us.

Filip Van den Abeele is engineer, entrepreneur and author of the book ‘Technology Wonders’, that highlights 13 ethical dilemma’s raised by new technology