How Edge Computing Enables the Move from Big Data to Smart Data

edge computing smart data banner

Big Data is transforming industry. It is also overwhelming it. In order for industry to move to the truly hybrid physical/digital realm that is often called Industry 4.0, it will have to overcome the problems that big data brings with it. Fortunately, advances in edge computing can turn that big data into smart data, decreasing data volume and speeding the ability to make crucial decisions.

What is big data?

Big data is best defined not in terms of some arbitrary number of petabytes, but of how that data has to be handled in order to be useful. Big data is what you have when you can no longer process and use that data adequately to achieve your goals.

Rapid growth in the number of deployed sensor and other data sources in the industrial Internet of Things (IoT) system increasingly ensures that the volume of generated data will continue to grow fast enough to outpace the development of the networks, algorithms, and processing capacity necessary to move and deal with it.

Industry 4.0 and big data

The basis of Industry 4.0 and IoT is the interconnection of smart machines that increasingly generate and use large amounts of data. The goal is responsive processes that can react to changing conditions, unexpected errors, and novel goals. 

Edge computing industry 4.0

If data transmission were cheap and infinitely fast, and cloud servers cheap and able to increase capacity without additional cost, it might make sense to transmit all data, do all computations in the cloud, and send all decisions back to each device.

But the costs of data transmission rise with data volume and distance. And the physical limits of transmission speed combined with compute time leads to unpredictable latency. Big data easily becomes expensive and slow data, a burden rather than a competitive advantage.

From big data to smart data

A key way of keeping big data from becoming overwhelming involves cleaning, filtering, and contextualizing it as close to the point of its creation as possible, thus turning it into smart data before it is used for anything else. Smart data’s volume is vastly less than the big data it is derived from, and is structured in ways that minimizes additional required computation time.

Smart data is sometimes termed “data that makes sense,” presumably to people. But it is not only people who benefit from smaller amounts of higher quality data. Smart devices with inherently limited computing capabilities and very specific data needs likewise function far better with smart data.

Edge computing, and smartness where it matters most

The development of a wide range of highly capable sensors, sensor networks, gateways, and other smart devices means that a significant amount of data can be processed at the edge of the network, right where these devices are operating. 

Since the now-smart data does not need to travel and does not require additional processing, it can be used right where it is created, for decisions in time sensitive situations. The farther away this edge is from the center, and the more time-sensitive the required decisions, the more important edge computing and smart data become. Remote oil and mining operations, railways and other transportation networks, wind turbines, autonomous vehicles, and distributed manufacturing facilities all will increasingly rely on edge computing to manage assets and maintain operations.

At the same time, the right set of smart data can still go on to the cloud, to be combined with a wide range of other data, analyzed, and used to optimize operations globally, track changing performance across the network, and identify early warning signs of problems. This deeper understanding can then refine the decision-making process at the edge. Both the edge and the cloud have their place in a system that continually improves itself.

Smart data enables operational visibility and control

Smart data generated by edge computing can also affect how other decisions are made, by providing operations people with a clearer view into the real-time situations they are managing. Those with the most intimate understanding of the machines that do the work will be able to gain an even better grip on events as they occur. 

You will be able to track all of your devices, sensors, readers, and other devices and provision and upgrade them as needed. Since there are so many, and they can essentially keep an eye on each other, there are no single-point failures, and emerging problems are surfaced long before they turn into anything that affects operations.

Operational technology (OT) and information technology (IT) will increasingly converge, and edge computing is a major path to this convergence. While IT/OT considerations have mostly been at higher levels, such as interactions with product lifecycle management (PLM), enterprise resource planning (ERP), and manufacturing execution systems (MES), they will now interact at every level, down to individual edge devices, as well as existing programmable logic controllers (PLCs) and other machine-to-machine (M2M) devices.

If OT has felt left out or ignored in organizational IT transformation, that is unlikely to be a problem in the future. OT will have a lot more computational power at its disposal, allowing for both visibility and control of the processes that are its responsibility.

So get to the edge and smarten up your data

There are a variety of ways to increase the capabilities of the edge and make big data usable. This is the time to start investigating if you haven’t already. To learn what Cognex has to offer, download the Cognex Edge Intelligence Datasheet.

More Posts on


Join MyCognex