Edge AI has become a buzzword in recent years. As the demand for real-time data processing is growing, being able to handle it directly on the user’s device such as a smartphone or IoT is becoming more important. The number of publications related to edge computing has increased dramatically in the last decade. In 2010, there were only 240 articles on this topic, whereas in 2023 there are already more than 42,700.
In this article, you will learn more about how edge AI works and where it is used.
What does edge AI mean?
Edge AI is an approach to developing and deploying artificial intelligence systems. It uses edge computing to execute machine learning models on user devices instead of sending data to the server. For example, if your Apple Watch collects data about your heartbeat and other parameters, instead of sending it to the company’s server to go through a neural network to drive insights, a neural network is deployed on your watch.
Another important use case for edge AI is self-driving vehicles. On the road, there are many situations where the speed of response matters. Processing data real-time is vital here, and edge computing can make it much easier to achieve.
These edge devices include anything from smartphones, smart cameras, and sensors to industrial robots and autonomous vehicles. The primary feature of edge AI is that it allows these devices to process data locally and make decisions without needing to send it to a remote server.
Brief history of edge AI
The concept of edge AI has evolved alongside advancements in AI, computing power, and the proliferation of IoT devices. Here’s a brief timeline:
- Early 2010s. Deep machine learning has just emerged and is developing fast. Computations are handled by centralized cloud servers. However, as IoT devices gained ground, the limitations of cloud dependency—such as latency, bandwidth, and privacy concerns—became apparent.
- Mid-2010s. Powerful processors and microchips appear, like mobile GPUs and custom AI chips. This prepares the ground for AI to be deployed on devices.
- 2017. Frameworks like TensorFlow Lite and PyTorch Mobile emerge, making it possible to deploy AI models on mobile and embedded devices. Around this time, the term “edge AI” started to gain traction, highlighting the shift towards localized AI processing.
- Early 2020s. Edge AI saw increasing adoption across industries, from smart homes and autonomous vehicles to industrial automation and healthcare. The COVID-19 pandemic further accelerated this trend as industries sought to implement remote monitoring and automation.
Edge AI continues to advance, driven by innovations in AI hardware, software optimization, and the expanding ecosystem of smart devices. The technology is now a crucial component of the broader AI landscape, enabling new applications and use cases across various sectors.
How edge AI works
The functionality of edge AI revolves around the integration of AI models into edge devices, supported by three core components: hardware, software, and communication networks.
Hardware
Edge devices must be equipped with specialized processors capable of handling AI computations.
- CPUs (Central Processing Units). Central processors for edge AI tasks today are powerful enough to handle basic AI tasks. However, for more complex AI workloads, such as in smart cities, it is important to use specialized processors with increased bandwidth.
- GPUs (Graphics Processing Units). Graphic processors allow processing of computer graphics and visuals. Many edge devices use GPUs to accelerate AI model processing, particularly for image and video analysis.
- NPUs (Neural Processing Units). NPUs are dedicated AI accelerators optimized for tasks such as image recognition and natural language processing. They are commonly integrated into smartphones and IoT devices. Examples include Apple’s Neural Engine in its A-series chips and Huawei’s Kirin processors.
Sufficient RAM is necessary to store AI models and process data in real time. Edge devices typically need low-power, high-bandwidth memory to handle the intensive computational requirements of AI tasks. For devices using GPUs, VRAM is essential for storing and rapidly accessing large amounts of data, such as images or video frames, which are processed by AI models.
If a device needs to collect data for AI processing, they are also equipped with various sensors, cameras, and microphones. For instance, cameras capture images for computer vision tasks, while microphones enable voice recognition. Other sensors might include those for temperature, humidity, or motion detection.
Software
The software stack includes model optimization techniques like quantization, pruning, and compression to reduce the computational load and energy consumption.
TensorFlow Lite. This is a lightweight version of TensorFlow designed for mobile and embedded devices. TensorFlow Lite enables the deployment of machine learning models on edge devices with low latency and minimal power consumption.
PyTorch Mobile. A version of the PyTorch framework tailored for mobile and edge deployment. It supports model optimization and conversion tools that make it easier to run deep learning models on edge hardware.
Apache MXNet. A deep learning framework that supports edge deployments through its ability to export models to formats compatible with mobile and embedded devices.
Edge Impulse. A development platform specifically focused on enabling edge machine learning, offering tools to create, optimize, and deploy models directly onto edge devices, particularly microcontrollers and low-power hardware.
However, it’s important to note that larger companies usually develop their own custom libraries for edge AI. Their libraries work best with the hardware they use.
Finally, it’s necessary to deploy Linux-based Systems (e.g., Ubuntu, Yocto). Many edge devices, particularly those with higher processing capabilities, run on Linux-based operating systems. These systems provide a stable and flexible environment for deploying AI applications and managing device operations.
Android OS is used extensively in smartphones, tablets, and other smart devices, Android OS supports AI applications through native libraries and frameworks like TensorFlow Lite and ML Kit.
Communication networks
Edge AI devices may still need to communicate with cloud servers or other devices, but this communication is minimized or used selectively.
For communication with other devices or cloud servers, edge devices often need robust connectivity options such as Wi-Fi or Bluetooth. These modules ensure that data can be shared or offloaded efficiently.
In industrial or smart city applications, edge devices might need to connect via low-power wide-area networks (LPWANs), such as LoRa or NB-IoT, to facilitate communication over long distances with minimal power consumption.
Federated learning
Federated learning is the practice of processing data from different devices in a decentralized way. While edge AI processes most of the data on users’ devices, some of it can still be passed to the cloud. To avoid reverse engineering, engineers use federated learning where the training of the model happens collectively across several devices or servers.
Why edge AI is important
The rise of edge AI is driven by several key advantages:
- Low latency. Edge AI reduces the time needed to analyze data, which is crucial in real time analytics.
- Privacy and security. By keeping data on the device, edge AI enhances privacy and security. Sensitive information isn’t sent via networks and, therefore, can’t be stolen.
- Reduced bandwidth usage. Edge AI minimizes the need to send large volumes of data to the cloud for processing. This is particularly beneficial in environments with limited or costly internet connectivity, such as rural areas or remote industrial sites.
- Scalability. Edge AI enables scaling AI applications across numerous devices without overwhelming centralized infrastructure. This decentralized approach allows for the deployment of AI in diverse environments, from smart cities to personal gadgets.
Conclusion
Edge AI represents a significant shift in how AI is deployed and utilized, moving the power of AI closer to where data is generated. Its importance lies in its ability to deliver low-latency responses, enhance data privacy, reduce bandwidth usage, and scale AI applications across diverse environments. As the number of connected devices continues to grow, and the demand for real-time decision-making increases, edge AI is poised to play an even greater ole in the future.
Read more:
.jpg)
