Edge Intelligence – Deep Learning And Edge Computing To Democratize AI


With the breakthroughs in deep learning, recent years have witnessed a booming of artificial intelligence (AI) applications and services. Driven by advances in mobile computing and Internet-of-Things (IoT), billions of mobile and IoT devices are connected to the Internet, generating zillions Bytes of data at the network edge.

Driven by AI and IoT, there is an urgent need to push the AI frontiers to the network edge to fully unleash the potential of the edge big data. To realize this trend, Edge Computing is a promising solution to support computation-intensive AI applications on edge devices.

Edge Intelligence or Edge AI is a combination of AI and Edge Computing, it enables the deployment of machine learning algorithms to the edge device where the data is generated. Edge Intelligence has the potential to provide artificial intelligence for every person and every organization at any place.

In this article, we discuss the following topics:

  1. What is Edge Computing?
  2. Why we need Edge Computing
  3. What is Edge Intelligence or Edge AI?
  4. Moving Deep Learning Applications to the Edge
  5. On-Device AI and Inference at the Edge
  6. Edge Intelligence enables AI democratization

What is Edge Computing

Edge computing is the concept of capturing, storing, processing, and analyzing data closer to the location where it is needed to improve response times and save bandwidth. Hence, edge computing is a distributed computing framework that brings applications closer to data sources such as IoT devices, local end devices, or edge servers.

The rationale of edge computing is that computing should happen in the proximity of data sources. We envision that edge computing could have as big an impact on our society as we have witnessed with cloud computing.

Concept of edge computing
Concept of Edge Computing – Source

Why We Need Edge Computing

Data Is Generated At the Network Edge

As a key driver that boosts AI development, big data has recently gone through a radical shift of data sources from the mega-scale cloud data centers to the increasingly widespread end devices, such as mobile and IoT devices.  Traditionally, big data, such as online shopping records, social media contents and business informatics, were mainly born and stored at mega-scale data centers. However, with the emergence of mobile computing and IoT, the trend is reversing now.

Today, large numbers of sensors and smart devices are generating massive amounts of data, and ever-increasing computing power is driving the core of computations and services from the cloud to the edge of the network. Today, over 50 billion IoT devices are connected to the Internet, and the IDC forecasts that, by 2025, 80 billion IoT devices and sensors will be online.

Cisco’s Global Cloud Index estimates that nearly 850 Zettabytes (ZB) of data will be generated each year outside the cloud by 2021, while global data center traffic is only 20.6 ZB. This indicates, that the sources of data are undergoing a transformation – from large-scale cloud data centers to an increasingly wide range of edge devices.

Meanwhile, cloud computing is gradually unable to manage these massively distributed computing power and analyze their data:

  1. Resources: Moving a tremendous amount of data across the wide-area network (WAN) poses serious challenges to network capacity and the computing power of cloud computing infrastructures.
  2. Latency: For cloud-based computing, the transmission delay can be prohibitively high. Many new types of applications have challenging delay requirements that the cloud would have difficulty meeting consistently (e.g. cooperative autonomous driving).
Edge Computing Offers Data Processing At the Data Source

Edge Computing is a paradigm to push cloud services from the network core to the network edges. The goal of Edge Computing is to host computation tasks as close as possible to the data sources and end-users.

Certainly, edge computing and cloud computing are not mutually exclusive. Instead, the edge complements and extends the cloud. The main advantages of combining edge computing with cloud computing are the following:

  1. Backbone network performance: Distributed edge computing nodes can handle many computation tasks without exchanging the underlying data with the cloud. This allows optimizing the traffic load of the network.
  2. Agile service response: Applications deployed at the edge can significantly reduce the delay of data transmissions and improve the response speed.
  3. Powerful cloud backup: In situations where the edge cannot afford it, the cloud can provide powerful processing capabilities and massive, scalable storage.

Data is increasingly produced at the edge of the network, and it would be more efficient to also process the data at the edge of the network. Hence, edge computing is an important solution to break the bottleneck of emerging technologies based on its advantages of reducing data transmission, improving service latency, and easing cloud computing pressure.

Edge Intelligence Combines AI and Edge Computing

Data Generated At the Network Edge Needs AI To Unlock Its Potential

The skyrocketing numbers and types of mobile and IoT devices lead to the generation of massive amounts of multi-modal data (audio, pictures, video) of the device’s physical surroundings that are continuously sensed.

AI is functionally necessary due to its ability to quickly analyze those huge data volumes and extract insights from them for high-quality decision-making. Gartner forecasted, that by 2022, more than 80% of enterprise IoT projects will include an AI component.

One of the most popular AI techniques, deep learning, brings the ability to identify patterns and detect anomalies in the data sensed by the edge device, for example, population distribution, traffic flow, humidity, temperature, pressure, and air quality.

The insights extracted from the sensed data are then fed to the real-time predictive decision-making (e.g. public transportation planning, traffic control, and driving alert) in response to the fast-changing environments, increasing the operational efficiency. 

What is Edge Intelligence and Edge AI

The combination of Edge Computing and AI has given rise to a new research area named “Edge Intelligence” or “Edge AI”. Edge Intelligence makes use of the widespread edge resources to power AI applications, without entirely relying on the cloud. While the term Edge AI or Edge Intelligence is brand new, practices in this direction have begun early with Microsoft building an edge-based prototype to support mobile voice command recognition in 2009.

However, despite the early beginning of exploration, there is still not a formal definition for edge intelligence. Currently, most organizations and presses refer to Edge Intelligence as “the paradigm of running AI algorithms locally on an end device, with data (sensor data or signals) that are created on the device.”

Edge AI and Edge Intelligence are widely regarded areas for research and commercial innovation. Due to the superiority and necessity of running AI applications on the edge, Edge AI has recently received great attention.

The Gartner Hype Cycles names Edge Intelligence as an emerging technology that will reach a plateau of productivity in the following 5 to 10 years. Multiple major enterprises and technology leaders including Google, Microsoft, IBM, and Intel demonstrated the advantages of edge computing in bridging the last mile of AI. These efforts include a wide range of AI applications, such as real-time video analytics, cognitive assistance, precision agriculture, smart home, and industrial IoT.


Concept of Edge intelligence and intelligent edge
Concept of Edge Intelligence and Intelligent Edge – Source


Cloud Is Not Enough to Power Deep Learning Applications

Artificial Intelligence and deep learning based intelligent services and applications have changed many aspects of people’s lives due to the great advantages of deep learning in the fields of Computer Vision (CV) and Natural Language Processing (NLP).

However, due to efficiency and latency issues, the current cloud computing service architecture is not enough to provide artificial intelligence for every person and every organization at any place. For a wider range of application scenarios, such as smart factories and cities, face recognition, medical imaging, etc., there are only a limited number of intelligent services offered due to the following factors:

  • Cost: The training and inference of deep learning models in the cloud require devices or users to transmit massive amounts of data to the cloud. This consumes an immense amount of network bandwidth.
  • Latency: The delay to access cloud services is generally not guaranteed and might not be short enough for many time-critical applications.
  • Reliability: Most cloud computing applications depend on wireless communications and backbone networks for connecting users to services. For many industrial scenarios, intelligent services must be highly reliable, even when network connections are lost.
  • Privacy: Deep Learning often involves a huge massive amount of private information. Privacy issues are critical to areas such as smart homes and cities. In some cases, even the transmission of sensitive data may not be possible.

Since the edge is closer to users than the cloud, edge computing is expected to solve many of these issues.

Advantages of Moving Deep Learning to the Edge

The fusion of AI and edge computing is natural since there is a clear intersection between them. On one hand, data generated at the network edge needs AI to fully unlock the full potential. On the other hand, edge computing is able to prosper with richer data and application scenarios.

Edge intelligence is expected to push deep learning computations from the cloud to the edge as much as possible. This enables the development of various distributed, low-latency, and reliable intelligent services. The advantages of deploying deep learning to the edge include:

  1. Low-Latency: Deep Learning services are deployed close to the requesting users. This significantly reduces the latency and cost of sending data to the cloud for processing.
  2. Privacy Preservation: Privacy is enhanced since the raw data required for deep learning services is stored locally on the edge devices or user devices themselves instead of the cloud.
  3. Increased Reliability: Decentralized and hierarchical computing architecture provides more reliable deep learning computation.
  4. Scalable Deep Learning: With richer data and application scenarios, edge computing can promote the widespread application of deep learning across industries and drive AI adoption.
  5. Commercialization: Diversified and valuable deep learning services can broaden the commercial value of edge computing and accelerate its deployment and growth.

Unleashing deep learning services using resources at the network edge, near the data sources, has emerged as a desirable solution. Therefore, edge intelligence aims to facilitate the deployment of deep learning services using edge computing.

Capabilities comparison of cloud, on-device and edge intelligence
Capabilities comparison of cloud, on-device and edge intelligence – Source
Edge Computing Is the Key Infrastructure for AI Democratization

AI technologies have witnessed great success in many digital products or services in our daily life (e-commerce, service recommendation, video surveillance, smart home devices, etc.). Also, AI is a key driving force behind emerging innovative frontiers, such as self-driving cars, intelligent finance, cancer diagnosis, smart city, intelligent transportation, and medical discovery.

Based on those examples, leaders in AI push to enable a richer set of deep learning applications and push the boundaries of what is possible. Hence, AI democratization or ubiquitous AI is a goal declared by major IT companies, with the vision of  “making AI for every person and every organization at everywhere”. Therefore, AI should move “closer” to the people, data and end devices. Obviously, edge computing is more competent than cloud computing in achieving this goal:

  1. Compared to cloud data centers, edge servers are in closer proximity to people, data sources, and devices.
  2. Compared to cloud computing, edge computing is more affordable and accessible.
  3. Edge computing has the potential to provide more diverse application scenarios of AI than cloud computing.

Due to these advantages, edge computing is naturally a key enabler for ubiquitous AI.

Deployment of Machine Learning Algorithms at the Network Edge

The unprecedented amount of data, together with the recent breakthroughs in artificial intelligence (AI), enables the use of deep learning technology. Edge Intelligence enables the deployment of machine-learning algorithms at the network edge.

The key motivation of pushing learning towards the edge is to allow rapid access to the enormous real-time data generated by the edge devices for fast AI-model training and inferencing, which in turn endows on the devices human-like intelligence to respond to real-time events.

On-device analytics run AI applications on the device to process the gathered data locally. Because many AI applications require high computational power that greatly outweighs the capacity of resource- and energy-constrained edge devices. Therefore, the lack of performance and energy efficiency are common challenges of Edge AI.

Different Levels of Edge Intelligence

Most concepts of Edge Intelligence generally focus on the inference phase (running the AI model) and assume that the training of the AI model is performed in cloud data centers, mostly due to the high resource consumption of the training phase.

However, the full scope of Edge Intelligence fully exploits available data and resources across the hierarchy of end devices, edge nodes, and cloud data centers to optimize the overall performance of training and inferencing a Deep Neural Network model. Therefore, Edge Intelligence does not necessarily require the deep learning model to be fully trained or inferenced at the edge, but you’ll also work in a cloud-edge coordination via data offloading.

Edge Intelligence - Different Levels of Cloud and Edge computing
Edge Intelligence: Scope of Cloud and Edge Computing. – Source

There is no “best-level” in general because the optimal setting of Edge Intelligence is application-dependent and is determined by jointly considering multiple criteria such as latency, privacy, energy efficiency, resource cost, and bandwidth cost.

  • Cloud Intelligence is the training and inferencing of AI models fully in the cloud
  • On-device Inference includes AI model training in the cloud, while AI inferencing is applied in a fully local on-device manner. On-device inference means, that no data would be offloaded.
  • All On-Device is performing both training and inferencing of AI models fully on-device.

By shifting tasks towards the edge, transmission latency of data offloading decreases, data privacy increases, and cloud resource and bandwidth costs are reduced. However, this is achieved at the cost of increased energy consumption and computational latency at the edge.

On-device Inference is currently a promising approach for various on-device AI applications that have been proven to be optimally balanced for many use cases.

Deep Learning On-Device Inference at the Edge

AI models, more specifically Deep Neural Networks (DNNs), require larger-scale datasets to further improve their accuracy. This indicates, that computation costs dramatically increase. The outstanding performance of Deep Learning models demands high-level hardware. It is difficult to deploy them to the edge which comes with resource constraints. Therefore, large-scale deep learning models are generally deployed in the cloud while end devices just send input data to the cloud and then wait for the deep learning inference results.

However, the cloud-only inference limits the ubiquitous use of deep learning services:

  • Inference Latency. Specifically, it cannot guarantee the delay requirement of real-time applications, such as real-time detection with strict latency demands.
  • Privacy. Data safety and privacy protection are important limitations of cloud-based inference systems.

To address those challenges, deep learning services tend to resort to edge computing. Therefore, deep learning models have to be customized to fit in the resource-constrained edge. Meanwhile, the deep learning applications need to be carefully optimized to balance the trade-off between inference accuracy and execution latency.

What’s Next?

With the emergence of both AI and IoT comes the need to pushing the AI frontier from the cloud to the edge device. Edge computing has been a widely recognized solution to support computation-intensive AI applications in resource-constrained environments.

Intelligent Edge is a novel paradigm of bringing edge computing and AI together with the goal to power ubiquitous AI applications for organizations across industries.

We recommend you read the following articles that cover related topics:



  • Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (Source)
  • Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (Source)
Share on linkedin
Share on twitter
Share on whatsapp
Share on facebook
Share on email
Related Articles

Want to use Computer Vision applications?

Get the all-in-one Suite to build and deliver Computer Vision Applications. 
Learn more

This website uses cookies. By continuing to browse this site, you agree to this use.