Introduction to Tiny ML

TinyML – machine learning in embedded devices – is actively gaining popularity. According to ABI Research’s latest whitepaper, TinyML device shipments will grow to 2.5 Billion in 2030, up from 15 million in 2020. With the vast proliferation of intelligent devices, it has become vital to equip them with machine-learning capabilities that run smoothly and fast, even in limited memory and computational power conditions.

This post will explain how TinyML works and why it is important.

TinyML Vs. embedded systems Vs. IoT

When talking about TinyML, it’s helpful to first talk about embedded systems. The reason is simple – TinyML runs on embedded hardware.

An embedded system is a computer system designed to perform specific tasks and functions within a more extensive system or product. Unlike general-purpose computers, which are designed to be versatile and can run a variety of software applications, embedded systems are typically built to perform a single task or set of functions in a specific environment or context. Embedded software runs on microcontrollers. They are small, special purpose computers. For example, a smart TV has a microcontroller in its system that controls the volume, the selection of channels or the connection to Smart TV.

Embedded systems are also tightly connected with IoT (Internet of Things). The Internet of Things is a concept that promotes building systems of devices that are connected via the internet. To do that, IoT also relies on embedded architecture.

“The proliferation and democratization of AI has fueled the growth of Internet of Things (IoT) analytics. Data collected from IoT devices are used to train Machine Learning (ML) models, generating valuable new insights into the IoT overall. These applications require powerful and expensive solutions that rely on complex chipsets.” — Lian Jye Su, AI & ML Principal Analyst at ABI Research.

Embedded systems are often found in everyday devices, such as home appliances, automobiles, medical equipment, and industrial control systems. The growth of microcontroller sales in recent years and the increasing demand for smart devices in everyday life have provided for the development of machine learning technologies that power such devices.

TinyML is a set of optimized machine learning techniques that enables machine learning models to run on embedded systems that use microcontrollers. TinyML backs up IoT systems allowing them to process and analyze large amounts of data while using very little power. \

Here are some of the ways in which TinyML is currently being achieved:

• Quantization. To reduce memory and computation requirements, you can reduce the precision of the weights and activations in a machine learning model. This makes ML models much more small device-friendly. A library that you can use for quantization is PyTorch Quantization API.
• Pruning. Pruning is a technique that involves removing unnecessary connections between neurons in a neural network. It also reduces the size of the model and improves its efficiency. PyTorch can also be used for pruning models.
• Model compression. Model compression techniques involve compressing the weights and activations in a machine learning model. A compressed model runs easily when the resources are limited. AwesomeML is a library that can be used for model compression.
• Knowledge distillation. You can train a smaller, simpler model to mimic the behavior of a larger, more complex model. Such models can be run on a less powerful hardware while largely remaining effective for the task. To perform knowledge distillation, something like this library can help.
• Hardware acceleration. Specialized hardware can be used to accelerate the computation required for machine learning on small devices, which can greatly improve their performance and efficiency.
• Edge computing. Edge computing involves performing machine learning tasks on the device itself, rather than relying on cloud-based services, which can greatly reduce latency and improve privacy and security.

Why is ML for embedded systems/TinyML important?

65% of Americans own smart devices at home. These include speakers, kitchen appliances, security systems, and lighting systems. In the past 5 years, the interest and the demand for TinyML has skyrocketed, as this Google Trends graph suggests.

Each device must function efficiently, learn about usage patterns, and adapt to the household’s needs. Moreover, technology is shrinking in general. Computers that used to occupy whole rooms now occupy just a lap space. Smartphones are becoming smaller and thinner. But we must maintain quality, regardless of the size.

TinyML is important because it allows us to perform the same advanced machine learning functions on smaller devices. The model is optimized to run on smaller devices that don’t have as much memory and computational power. An average TinyML model is just 18 KB, while the size of some regular neural networks can reach 0.8TB.

Advantages of TinyML

Here are some essential advantages that using TinyML can provide to an embedded device.

Improved performance

ML algorithms can enable embedded systems to perform tasks more efficiently and accurately. You don’t need a large multifunctional model to run in a lightbulb – while it is definitely less efficient than a big model in general, it performs just well enough in dedicated tasks. A smart lighting system can adjust the light level in the room based on its functionality, for example, soft, warm light in the living room and bright white light for reading in the office, and increase the brightness as the day progresses.

Reduced power consumption

Many embedded systems are designed to operate on battery power or limited power sources, and optimizing power consumption is crucial for efficient operation. TinyML algorithms can enable the energy-efficient operation of these devices by reducing the computational load and minimizing data transfer. For example, hearing aids powered by a battery can assess the noise level around them and save resources when the aids aren’t being actively used.

Increased real-time responsiveness

ML algorithms can enable embedded systems to perform complex tasks in real-time, such as speech recognition or image processing. For example, TinyML, which runs in autonomous vehicles, analyzes the environment in real time and responds accurately to changing circumstances.

Improved cost-efficiency

With advancements in hardware and software, TinyML can be implemented on low-cost microcontrollers and sensors, making it an affordable option for many applications.

Challenges of establishing TinyML ecosystems

While TinyML provides many benefits, it also introduces specific challenges.

Memory constraint is an obvious disadvantage of TinyML systems, and thorough work on their optimization must be conducted for the software to function correctly. However, big tech companies have been working to introduce fast and cheap solutions to the problem like IBM3 Microdrive. Such an HDD disk was a couple centimeters wide but functions exactly like traditional hard drives that could be found in any computer.

Today SSDs are used for processing and storing files, as they are faster and more efficient than HDD. Some examples include Arduino Nano 33 BLE Sense, the SparkFun Edge and the STMicroelectronics STM32 Discovery Kit.

Applications of ML for embedded systems

Now let us look at how different industries use TinyML for their devices.

Consumer electronics

ML algorithms can be used in consumer electronics devices such as smart speakers, wearable devices, and intelligent home appliances. For example, TinyML enables smartphones to recognize your voice, process photos and videos, and detect your location.

Automotive industry

The automotive industry can benefit from TinyML in various ways. ML algorithms can be used in cars under the hood, for example, for exhaustion and combustion tuning with microcontrollers. TinyML can also be used in autonomous vehicles for object detection, lane detection, and traffic sign recognition.

Healthcare industry

ML algorithms can be used in healthcare applications such as in wearable personal monitoring devices. For example, TinyML can detect early signs of diseases such as cancer or monitor vital signs such as heart rate and blood pressure in real time.

Industrial automation

TinyML can be used in industrial automation applications where robots and automated vehicles of any kind are involved. Robotics uses TinyML to make machines smart but optimize the use of resources in the conditions of memory and power constraints.

Conclusion

TinyML represents a major breakthrough in the field of machine learning, enabling the creation of intelligent, low-power devices that can operate without being tethered to the cloud. With the ability to perform complex tasks such as image recognition, speech processing, and predictive analytics on devices as small as a microcontroller, TinyML is opening up new possibilities for a wide range of applications, from wearable devices to smart homes, from industrial automation to agriculture, and from healthcare to transportation.

As TinyML becomes more accessible and widely adopted, it has the potential to transform the way we live and work, enabling us to achieve new levels of efficiency, productivity, and convenience. The future of TinyML is bright, and we can expect to see many exciting developments in the years to come.