AI, Blockchain, IoT System Implementation Insights

1
©2020 TechIPm,LLC All RightsReservedhttp://www.techipm.com/
AI, Blockchain, IoT Convergence Use Case System Implementation Insights from Patents1
Alex G. Lee2
Contents
I. AI, Blockchain, IoT, and Their ConvergenceTechnologyInnovation Status
II. AI, Blockchain, IoT ConvergenceUse Case SystemImplementation Examples
1. Blockchain-basedPrivacy-Preserving FederatedLearning System
2. Blockchain-basedDecentralizedIoT & AI Data Marketplace
3. Blockchain-basedTrustworthy AI & IoT Systems
4. Blockchain-basedDecentralizedParallelEdge Machine Learning
5. Predictive Maintenance Platform for Industrial Machine using Industrial IoT
6. AI+BlockchainSystemfor Car Sharing Service
7. ConnectedAutonomous Vehicle Communication ManagementSystem
8. 5G-basedAI+Blockchain+IoTEdge Computing System
1
This research was supported partly by the WEF Global Partnership Program for the Fourth Industrial Revolution through KAIST KPC4IR funded
by the Ministry of Science and ICT of S. Korea.
2
Alex G. Lee,Ph.D/Patent Attorney,is a principal consultant at TechIPm,LLC.

2
I. AI, Blockchain, IoT, and Their ConvergenceTechnologyInnovation Status3
Patents are a good information resource for obtaining the state of the art of technology innovation insights. Patents
that specifically describe the major technology filed of a specific technology innovation are a good indicator of the
technology innovation status in a specific innovation entity. A patent counting for growth in patenting over a
period of times can be a good measuring tool for monitoring the evolution of technology innovation.
To find AI, blockchain, IoT, and their convergence technology innovation status, patent applications in the USPTO,
EPO, IPO during the period of January 1, 2010 - September 30, 2020 in priority date (technology innovation date)
that specifically describe the major AI, blockchain, IoT, and their convergence technologies are searched and
reviewed. Total of 48,000, 17,000, 32,000, 1,288 published patent applications that are related to the key AI,
blockchain, IoT, and their convergence technology innovation respectively are selected for detail analysis. Figure 1
summarizes our patent analysis results.
Figure 1 (1) shows the top 100 AI, blockchain, and IoT technology innovation entities respectively selected by the
total number of published patent applications. The top 100 AI technology innovation entities represent 23,593
patent applications. The top 10 AI innovation leaders are IBM, Google, Microsoft, Samsung Electronics, Intel,
3
AI, Blockchain, IoT Convergence Insights from Patents: https://www.slideshare.net/alexglee/ai-blockchain-iot-convergence-insights-from-patents

3
Siemens, Facebook, Philips, GE, and Accenture. The top 100 blockchain technology innovation entities represent
7,809 patent applications. The top 10 blockchain innovation leaders are Alibaba Group, IBM, nChain, Mastercard,
Walmart, Bank of America, Visa, Microsoft, Intel, and Accenture. The top 100 IoT technology innovation entities
represent 18,786 patent applications. The top 10 IoT innovation leaders are Qualcomm, Ericsson, LG Electronics,
Samsung Electronics, Intel, Ford, IBM, Huawei, GM, and Toyota.
Figure 2 (2) shows the AI, blockchain, and IoT patenting activities with respect to the technology innovation date
(priority year) respectively. The AI patent application activity chart indicates that the AI technology innovation
activity started in a rapid growth stage from 2016. Since there is usually a time lag between the initial application
date (priority year) and the publication date by around two years, the AI patent application activity chart indicates
that the AI technology innovation activity is still in a growth stage. The blockchain patent application activity chart
indicates that the Blockchain technology innovation activity started in a rapid growth stage from 2016. The
Blockchain patent application activity chart indicates that the Blockchain technology innovation activity is still in a
growth stage. The IoT patent application activity chart indicates that the IoT technology innovation activity started
in a growth stage already before 2010. The IoT patent application activity chart indicates that the IoT technology
innovation activity became matured in 2018.
Figure 1 (3) shows the convergence status among AI, blockchain, and IoT, the top 10 AI, blockchain, and IoT
convergence technology innovation entities, and the AI, blockchain, IoT convergence patenting activities with

4
respect to the technology innovation date (priority year) respectively. The key AI, Blockchain, IoT convergence
innovation entities are IBM, Strong ForceIP, Intel, Accenture , Microsoft , Bank of America , Bao Tran, Capital
One Services, LG Electronics, Cisco, Ericsson, Samsung Electronics, HP, nChain. Nokia, Inmentis, LLC,
Salesforce, Tata Consultancy Services, Siemens, and T-Mobile. AI+IoT convergence is the most innovated AI,
Blockchain, IoT convergence technology followed by AI+Blockchain, Blockchain+IoT, and AI+Blockchain+IoT.
Patent application activity chart indicates that the AI, Blockchain, IoT convergence technology innovation activity
is in a rapid growth stage.

5
Figure 1. AI, blockchain, IoT, and their convergence patent analysis results

6
II. AI, Blockchain, IoT Convergence Use Case SystemImplementation Examples
Patent information can provide many valuable insights that can be exploited for developing and implementing new
technologies/ products/services. Patents also can be exploited to identify new use case development opportunities.
Our patent analysis indicates that the AI, blockchain, and IoT convergence technology innovation covers the AI,
blockchain, and IoT convergence systems that can used for diverse use cases4:AI+Blockchain+IoT convergence
for privacy-preserving IoT/AI systems, AI+Blockchain+IoT convergence for IoT/AI data/software marketplace
systems; Blockchain-based crowdsourced AI storage/computing systems; AI systems for IoT/blockchain network
performance improvements (e.g., secure IoT networks, smarter smart contracts, improved consensus mechanism,
highly scalable networks), AI+Blockchain+IoT convergence for regulation compliance systems (e.g., automatic
GDPR compliance5), and Trustworthy IoT/AI systems.
Our patent analysis also indicates that the AI, blockchain, and IoT convergence technology innovation covers very
broad industry wise business applications6: advertisement services, agriculture management, automotive
4
AI+Blockchain+IoT Convergence AT A Glance: https://www2.slideshare.net/alexglee/aiblockchainiot-convergence-at-a-glance
5
AI, Blockchain, IoT GDPR Compliance AT A Glance: https://www2.slideshare.net/alexglee/ai-blockchain-iot-gdpr-compliance-at-a-glance
6
AI Blockchain IoT Convergence System Technology & Business Development: https://www.slideshare.net/alexglee/ai-blockchain-iot-
convergence-system-technology-business-development-239302652

7
manufacturing and transportation, consumer electronics products, cybersecurity systems, data marketplace, electric
grids, financial services including insurance, healthcare services, public services, retail, real estate, sharing services,
smart home, industrial machinery, smart factory, supply chain management, hospitality, social networking service,
sports applications, telecommunications, and tourism.

8
1. Blockchain-basedPrivacy-Preserving FederatedLearning System
A federated learning system trains a machine learning model based on data generated from large numbers of users
interacting with their devices while the users maintain their data locally in their devices. Each data owner (e.g.
federated learning participant) only provides locally trained machine learning model to a trusted third party
(aggregator) for a single consolidated and improved global model. The global machine learning model is then sent
back to user devices iteratively for updating the local machine learning model. Such collaboration among the
federated learning participants lead to more accurate machine learning model than any party could learn in
isolation. The federated learning system is a collaborative machine learning method which alleviates privacy issue
by performing the machine learning training process in a distributed manner, without the need of centralizing
private data.
Federated learning systems, however, provide insufficient data privacy. To protect the privacy of the datasets,
federated learning systems need to also consider inferences derived from the machine learning process and
information that can be traced back to its source in the resulting trained model. The conventional attempts to ensure
adequate data privacy in federated learning systems have resulted in poorpredictive performance of the resulting
model. For example, federated learning systems using local differential privacy7 can result in the generation of an
7
Typical privacy-preserving techniques for machine learning include homomorphic encryption, multiparty computation, and differential privacy.

9
abundant amount of noise, which can deteriorate machine model performance. To provide additional measures for
privacy protection, blockchain can be integrated to form decentralized federated learning systems.
IBM’s patent application US20200394552 illustrates a blockchain-based federated learning system. In the
blockchain-based federated learning system, client nodes of the blockchain network act as federated learning
participants. Each training participant client trains a machine learning model using a stochastic gradient descent
and its variants. Stochastic gradient descentis a standard optimization technique used in training machine learning
models. This technique involves computation of the so called "gradients" of a particular loss function defined on
the training dataset and the type of machine learning model involved. In a typical machine learning training process,
one starts from an initial machine learning model and updates the model iteratively based on a stochastic gradient
descent calculated in each step of iteration. The role of the blockchain-based federated learning system is to ensure
that the model is being properly trained by verifying gradients calculated by each federated learning participant at
each step of iteration.
As an implementation example of the blockchain-based federated learning system, the Hyperledger Fabric
blockchain network is used for fast verification of machine learning training procedures. In conventional
Hyperledger Fabric blockchain networks, the consensus mechanism to verify gradient descentupdates involves
multiple endorsing peer nodes (endorsers) performing the same gradients computation comparing the results with
the gradients calculated by each federated learning participant at for each iteration as part of a smart contract on

10
blockchain (chaincode). The endorsers arrive at the consensus if and only if a specified number of them agree on
the results. This is a wasteful and highly inefficient approach. Thus, a significantly more efficient and
computationally light approach is needed to be developed.
The example implementation of the blockchain-based federated learning system is based on approximate and
provable correct verification protocols instead of re-compute processesfor providing efficiently verifiable
consensus mechanisms. The example implementation also can enable real-time auditability by the real-time
computation guarantees. The example implementation is based on a new type of smart contract that accompanies
the machine learning training procedure. To this end, smart contracttransactions have relevant details about parts
of training data and machine learning model updates. Instead of performing the computationally expensive model
training step again, an endorser verifies the training computation using the metadata of the smart contract
transactions.
Figure 2 illustrates a block diagram of the example implementation of the blockchain-based federated learning
system. The training participant network 112 includes M training participant clients 104, each including M training
datatsets 108. The training aggregator 132 collects gradient calculations 136 from the training participants 104 and
combines them to create an aggregate model. Model parameters 136 specify a machine learning model in
mathematical terms. The training aggregator 132 is a trusted blockchain node. The training aggregator generates
verify gradient transaction proposals 124 in responseto the received gradient calculations 136.

11
Figure 2. Block diagram of the example implementation of the blockchain-based federated learning system.

12
Figure 3 illustrates a block diagram of a gradient calculation verification subsystem 150 of the blockchain-based
federated learning system. The gradient calculation verification subsystem 150 includes endorsers 116. The
endorsers 116 can access the independent auditor 154 that includes the verify gradient smart contract120. The
independent auditor 154 is an auditing entity responsible in verifying the progress of a machine learning training
process. Endorsers 116 refer to the independent auditor 154 to execute the verify gradient smart contract 120 and
verify gradient computation steps. In responseto receiving the gradient computation transaction proposals 124
from the training participant network 112, the endorsers forward the gradient computation transaction proposals
162 to the independent auditor 154. The independent auditor 154 executes the verify gradient smart contract 120 on
the received forwarded gradient computation transaction proposals 162. The verification results 166 include
endorsements or rejected transaction proposals, and are returned to the endorsers 116. The endorsed verification
results are eventually included in the shared ledger 158 of the gradient calculation verification subsystem150.

13
Figure 3. Block diagram of a gradient calculationverification subsystemof the blockchain-basedfederated
learning system.

14
2. Blockchain-basedDecentralizedIoT & AI Data Marketplace
Interconnected IoT devices can generate a huge amount of date that can be used for IoT applications. IoT data
marketplace can connect IoT data sellers and buyers so that IoT data can be collected, processedand finally
consumed by different parties for IoT applications. Blockchain enables a trusted IoT data marketplace that allows
secure and anonymous trading of IoT data. The decentralized nature of the data marketplace based on blockchain
means that any participant who qualifies can enter the marketplace as a data seller or a data buyer. Decentralization
also means that there is no central authority to regulate the participants of the market. There is no central data
repository. The data sellers are the owners of, and remain in full control over, their data.
Grandata Inc's patent application US20200058023 illustrates a blockchain-based decentralized data marketplace
that can trade IoT data and other types of data. An example of a data buyer is a company who would like to train its
own machine learning models using data purchased via the decentralized data marketplace. Figure 4 illustrates a
block diagram of the blockchain-based decentralized data marketplace and protocolfor exchanging data within the
marketplace. Example participants of the data marketplace 100 include a data buyer 102, a data seller 104, and a
notary 106. Each participant is associated with a correspondingcomputing device 110/112/114 that can be used for
access the data marketplace 100 by virtue of a client application 108.

15
Figure 4. Block diagram of the blockchain-baseddecentralizeddata marketplace and protocolfor
exchanging data within the marketplace.

16
The notary 106 validates the data quality and trustworthiness (e.g., data is not falsified or fabricated) that is being
offered in the marketplace before the data is purchased by a data buyer. The notary can access to “ground truth”
data that is usable to validate the offered data for accuracy. For example, in case of a data seller who wants to sell
his/her financial transaction data, the seller's bank is an ideal notary becausethe bank already has the financial
transaction data and can validate the offered data.
The data marketplace can provide: (i) the infrastructure for a exchange; (ii) mechanisms for trading data to be
evaluated and valued; (iii) incentives for participants to ensure that data is trustworthy and to provide quality data.
The data marketplace also can be a privacy-preserving marketplace by allowing users to sell private data while
providing them with privacy guarantees including (i) participants' anonymity; (ii) transparency over data usage;
and (iii) control over data usage.
Participants can register with the smart contracts so that the smart contracts 118 can be utilized by the participants.
In responseto registering with the smart contract, a participant receives an identifier associated with that smart
contract. The data exchange smart contract 118(1) is used by data buyers to create data orders for requested data
that the buyers are interested in buying. Thus, it provides a querying system for buyers to communicate their data
requirements by placing data orders on the blockchain. The use of on-chain operations provides a secure
mechanism for exchanging data. The data exchange smart contract maintains references to the data orders created
by data buyers so that the data orders are accessible to other market participants.

17
The batch payments smart contract 118(2) provides a payment protocolthat enables the direct transfer of payments
between participants. The batch payments smart contract manages the costs ofsmart contract transactions by
batching multiple payments associated with the smart contract transaction together into a single batch of payments,
and performing a single smart contract transaction to pay potentially multiple participants, which helps to reduce
transaction costs associated with operations in the blockchain.
The marketplace protocolsetups the data exchange procedure and payments for that data within the marketplace. It
is executed partly “on-chain” and partly “off-chain”. On-chain operations (denoted by the dashed lines as indicated
in the Key 116) are performed with respect to smart contracts 118 of the blockchain network. Off-chain operations
(denoted by the solid lines as indicated in the Key 116) are performed independent of the smart contract.
Figure 4 illustrates an example marketplace protocolfor the exchange of data between a data buyer and a data
seller, which is partitioned into separate phases: a setup phase, and a transaction phase. At step 1, during a setup
phase, a buyer device receives user input from a data buyer to create a new data order 122 for requested data. The
data buyer can indicate, in the data order, the intended audience and the requested data, which can both be
specified using a data ontology 120. The data ontology is a publicly available document that formalizes naming,
definition, structure and relationships for the marketplace's data and can be used as a reference to generate

18
audiences and data requests. The creation of the data order causes an event to be emitted via the blockchain so that
participants of the marketplace 1are aware of this new data order.
With the data order created in the data exchange smart contract, the notary device 114 provides a notification of
this data order, and the notary indicates its agreement to validate requested data for the newly-created data order by
providing user input to the notary device. In response to this user input, the notary is listed as one of the available
notaries for auditing the exchange of data associated with the newly-created data order. The notary device, using its
client application 108, generates a master key that is usable to encrypt cryptographic keys generated by seller
devices of sellers who indicate that they are interested in selling their data to the data buyer in fulfillment of the
data order. The notary device also generates a lock that can be used by the buyer with the batchpayments smart
contract during a transaction phase. The lock is sent to the public address of the buyer, which is specified in the
data order.
At step 2, sellers can monitor data orders that have been created using the data exchange smart contract in order to
look for opportunities where they match the audience, agree on the requested data, acceptthe price offered by the
data buyer, acceptthe terms and conditions of data use, and acceptone of the suggested notaries in the set of
notaries.

19
At step 3a, during a transaction phase, when the seller 104 selects one of the data orders, the seller device 112
encrypts relevant data 124 owned by the seller using a cryptographic key. The seller device sends the encrypted
data and the cryptographic key which was used to encrypt the data to a public address of the notary. At step 3b, the
seller device sends the encrypted data to the public address of the data buyer 102. The public address of the data
buyer is determined from the selected data order. The seller device sends the encrypted data without sending the
cryptographic key so that the data remains inaccessible to the data buyer until the notary reveals a master key. The
steps 3a and 3b are examples of “off-chain” operations.
At step 4, the buyer device sends, to a public address of the notary, an identifier of a data seller that the buyer has
selected, as well as a hash of the encrypted data provided by that seller who selected the data order. At step 5, the
notary can validate the seller's data, encrypt the cryptographic key of the seller using a master key. The notary also
can verify that it has the same seller data as the buyer has in his/her possessionbased on the hash value the buyer
device sent to the public address of the notary. After the notary performs the notarization and approves the data, at
step 5, the notary device sends, to the public address ofthe data buyer, the master key and the notarization result.
At step 6, the buyer device executes instructions to call a method of the batch payments smart contract. The
method specifying an identifier(s) of a seller(s) and an identifier of the notary to authorize respective payments to
be made to respective accounts of the data seller(s) and the notary. At step 7, the notary executes instructions to
call a method of the batch payments smart contract 118 to reveal the master key.

20
At step 8a, the seller device executes instructions to call a method of the batch payments smart contract to receive a
batch of payments that includes the payment for the data sold to, and purchased by, the data buyer. Similarly, at
step 8b, the notary device executes instructions to call the method of the batch payments smart contractto receive a
batch of payments that includes the payment for notarizing the data purchased by the data buyer. This method can
be called at any time by the devices of the seller and the notary.
IBM’s patent application US20190287027 illustrates a blockchain based AI data/model marketplace that can keep
the data hidden and secure while still enabling a model to be built. AI data/model consumers can interact with the
AI data/model marketplace to obtain additional data/models to improve accuracy, efficiency, etc., of their AI
model without having to generate the data on their own. However, proprietary data is sensitive and often protected
data of clients, trade secrets, financial, and other types of data that should not be exposed to outside parties.
Figure 5 illustrates a block diagram of a blockchain based AI data/model marketplace. The AI marketplace 100
includes a consumer 11, a consumer node 110, an aggregator node120, a plurality of producernodes 130, and a
blockchain node 140. The blockchain node 140 can store a blockchain that can be accessed byany of the other
nodes in the AI marketplace that are connected through the network 150.

21
Figure 5. Block diagram of a blockchain based AI data/model marketplace.
Figure 6 illustrates a communication sequence between nodes of the AI marketplace. The consumer node 281
initiates a new AI project by transmitting a request to the aggregator 282 node. The request includes an initial data

22
set 290 of the consumer node (a validation set) which is hashed using a locality preserving hash function and
unreadable to the aggregator node 120, metadata of the initial data set, and desired AI model. The request is stored
on blockchain 220.
Figure 6. Communication sequence between nodes of the AI marketplace.

23
In 291, the aggregator node retrieves the hashed summary either directly from the consumer node or from the
blockchain, and generates a broadcasthash summary transaction on the blockchain. The hash summary includes
the hashed initial data set, the metadata, and any other information about the AI model. The hash function used by
the aggregator node is different from the hash function used by the consumer and any of the producernodes.
In 292, the first producernode 283 and the second producernode284 detect the new AI project based on the
broadcasthash summary transaction and execute an improvement value function on the hashed initial data to
determine how much value data of a respective producernode can provide to the initial data set. Here, the
improvement value can be determined based on how similar the producerdata is to the consumer data.
In 293 and 294, the first and second producernodes submit their respective bids including value to be provided.
The bids can be submitted directly to the aggregator node or they can be stored on the blockchain and detected by
the aggregator node. The aggregator node then selects a winning bid based on the improvement value of the
different bids, and notifies the winning producernode in 296. The winning producernode then generate a hash data
set of its data using the locality-preserving hash which encodes the data while enabling an improvement value to be
calculated/verified the aggregator node, and transmits the hashed data to either the aggregator node or the
blockchain, in 297.

24
Upon successfulverification of the improvement value provided from the producernode within the bid 293 based
on the hashed data received in 297, the aggregator node updates the hashed initial data set and metadata stored on
the blockchain (an initial summary) with the hashed data of the winning producer node to generate an updated
summary, in 298. The aggregator node iteratively solicit additional bids from the producernodes until no more
winning bids are left or some other condition such as the AI model is completed, the expiration of period of time,
etc.
The blockchain-based AI marketplace enables distributed composition of models and data with fair and trustable
value attribution. Consumers can pick and combine AI models and data for their given need. In addition, producers
can monetize AI models and data where more useful models can receive higher value. A combination of locality
preserving hash-exchanges and blockchain enables composition with value attribution. The system enables,
simultaneously, consumers to determine how best to composeexisting data/models, producers to be compensated
for their AI moels/data used, and privacy of the consumer's and the producer's assets.

25
3. Blockchain-basedTrustworthyAI & IoT Systems
A lack of track and trace data used for a AI model can lead to vulnerabilities that can be exploited in different types
of attacks on the AI model and that can produceinferences that are susceptible to forms of discrimination and bias.
Another known issue with current AI models is that it is impossible to know how a deep learning model arrives at a
prediction or decision with respect to performing the specific task. In other words, a deep learning process canbe
thought of as a “black box,”to which input is provided to achieve a desired output. While the output may be
independently verified, to within a measurable degree of statistical probability, as being accurate or inaccurate,
understanding how the deep learning is used the input to arrive at the output is more complicated (e.g., Explainable
AI (XAI)).
Acronis International GmbH's patent application US20190228006 illustrates a blockchain-based machine learning
anti-discrimination verification system. Figure 7 is a block diagram depicting operations for verifying
discrimination or bias of a machine learning operation. The machine learning (ML) model has been trained by the
training data set 112. The machine ML module 102 applies the ML model to the input data set 106 to generate an
initial prediction 201. Then, the ML module determines whether the initial prediction violates a nondiscrimination
policy based on an evaluation data set (e.g., test set). If so, the ML module modifies the initial prediction, using a
corrective model 202, to generate a corrected prediction 205 based on a corrective algorithm. The corrective model
provides an offset or coefficient factor to be applied to the initial prediction to calculate the corrected prediction.

26
In another implementation aspect, the ML module has been trained the ML model using a corrective input data set
204 to achieve a requested level of non-discrimination by the ML model. The corrective input data set includes
multiple inputs selected to modify the internal rules of the ML model such that a level of discrimination in relation
to a protected attribute (e.g., race) is removed or reduced.
Figure 7 Block diagramdepicting operations for verifying discrimination or bias of a machine learning
operation.

27
The ML module generates a blockchain transaction data structure 114 and transmit the transaction to the
blockchain network 103. The blockchain transaction data structure includes a field 210 that references a prior
transaction data structure using a cryptographic hash, an identifier specifying the prediction 205 associated with the
transaction data structure. The blockchain transaction data structure also includes a field 212 that is a state of the
ML model at the time of generating the prediction, a copyof the input data set 106, and an indication 214 of a
correction by the ML model.
If the correction to the ML model was the use of the corrective model, the indication 214 of the correction includes
a copyof the initial output by the ML model prior to correction (e.g., prediction 201), a state of the corrective
algorithm 202, and a result (prediction 205) of the corrective algorithm 202. If the correction to the ML model
involved a dedicated training using a corrective training data set 204, the indication of the correction includes an
indication that the ML model has been verified using an evaluation data set subsequent to training using the
corrective input data set.
At a later time, blockchain transaction data structures can be retrieved from the blockchain network 103, and used
to validate compliance of a nondiscrimination policy based on the received second transaction. For example, the
state of the ML model 110 and corrective model 202 can be used to re-create the prediction and ML output under
review.

28
IBM's patent application US20190165949 illustrates a blockchain-based trustworthy IoT system. Blockchain
assigns a unique transaction key to each respective IoT device so that the blockchain can verify whether the IoT
data sent to the blockchain are from a trusted source. The IoT data also can be anonymized by the blockchain
system. Thus, the blockchain maintains a record of where and who provided the data based on the unique
transaction key signature, while also shielding the user/device that captured the data. In addition, the blockchain
can remove details from the data captured that is not relevant to a particular event. For example, the anonymization
process extracts specific details of an event while preventing other details from being revealed thereby maintain
privacy of individuals out in public environments as well as maintaining a privacy of the user or the entity that
captured the IoT data.

29
4. Blockchain-basedDecentralizedParallelEdge Machine Learning
Efficient AI model building requires large volumes of data. While distributed computing has been developed to
coordinate large AI computing tasks using multiple computers, applications to large scale machine learning (ML)
problems is difficult: For example, in distributed AI computing environments (e.g., Interconnected IoT sensors in a
smart factory, Interconnected IoT devices for smart mobility services, connected vehicles), the accessibility of
large and sometimes private training datasets across the distributed devices can be prohibitive and changes in
topology and scale of the network over time makes coordination and real-time scaling difficult.
HPE’s patent application US20190332955 illustrates a blockchain based decentralized machine learning that is
performed at blockcahin nodes where local training datasets are generated to build AI models. The blockchain is
used to coordinate decentralized machine learning over a series of iterations. Foreach iteration, a distributed ledger
is used to coordinate the nodes. Each blockchain node can participate in a consensus decision to enroll another
physical computing node to participate in the first iteration. The consensus decision applies only to the first
iteration and cannot register the second physical computing node to participate in subsequentiterations.
Upon registration of a specified number of blockchain nodes required for an iteration of training, each node obtains
a local training dataset accessible locally but not accessible at other ones of the physical computing nodes in the
blockchain network. Each participant node trains the first local AI model based on the local training dataset during
the first iteration, and obtains at least the first training parameter based on the first local model. In this manner,

30
each participant node trains on data that is locally accessible but cannot be shared with other nodes. Each
participant nodegenerates a blockchain transaction comprising an indication that the participant nodeis ready to
share the first training parameter, and transmit the first training parameter to a master node.
The master node generates a new transaction to be added as a ledger block to each copyof the distributed ledger
based on the indication that the first physical computing node is ready to share the first training parameter. The
master node can be elected from among the participant nodes by consensus decision. The master node obtains the
training parameters from each of the participant nodes, and creates final training parameters based on the obtained
training parameters. Upon generation of the final training parameters, the master node broadcasts an indication to
the blockchain network that the final training parameters are available and relinquishes status as a master node for
the current iteration.
Each participant node obtains, from the blockchain network, the final training parameters that were generated by
the master node based on the first training parameter and at least the second training parameter generated at, and
shared from, the second physical computing node. Each participant node then applies the final training parameters
to the first local model.
Smart contracts can enforce node participation in the iteration of AI model building and parameter sharing, as well
as provide logic for electing a node that serves as a master node for the iteration. The master node obtains the AI

31
model parameters from the nodes and generates final parameters based on the obtained parameters. The master
node writes its state to the distributed ledger indicating that the final parameters are available. Each node can
discover the master node's state and obtain and apply the final parameters to its local model via its copyof the
distributed ledger, thereby learning from other nodes.
Computing power at the IoT edge devices can serve as computing nodes to perform model training. These IoT
edge nodes can be placed at or near the boundary of an IT infrastructure where the real world interacts with large
information technology infrastructure. For example, autonomous vehicles can include more than one computing
device that can communicate with fixed server assets. Another example includes real-time traffic management in
smart cities, which divert their data to a data center.
Because decentralized machine learning occurs over multiple iterations and different sets of nodes can enroll to
participate in any iteration, the decentralized model building activity can dynamically scaled as the availability of
nodes changes. Furthermore, dynamic scaling does not cause degradation of model accuracy. By using a
distributed ledger to coordinate activity and smart contracts to enforce synchronization by not permitting stale or
otherwise uninitialized nodes from participating in an iteration, the stale gradients problem can be avoided. Use of
the decentralized ledger and smart contracts also can make the system fault-tolerant. Node restarts and other
downtimes can be handled seamlessly without loss of model accuracy by dynamically scaling participant nodes
and synchronizing learned parameters.

32
Figure 8 illustrates an example process 300 of an iteration of model building using blockchain according, where
operations 302-312 and 318 are applicable to participant nodes and operations 314, 316, and 320 are applicable to
master node. In the operation 302, each participant node enrolls to participate in an iteration of model building. In
an implementation, the smart contracts encoderules for enrolling a node for participation in an iteration of model
building. In the operation 304, each of the participant nodes executes local model training on its local training
dataset.
Figure 8. Example process of an iteration of model building using blockchain.

33
In the operation 306, each of the participant nodes generates local parameters based on the local training and keeps
them ready for sharing with the blockchain network. In the operation 308, each participant node checks in with the
blockchain network for co-ordination. In the operation 310, participant nodes collectively elect a master node for
the iteration. For example, the smart contracts can encoderules for electing the master node. In the operation 312,
participant nodes that are not a master node periodically check the state of the master node to monitor whether the
master node has completed generation of parameters based on local parameters shared by participant nodes.
In the operation 314, the master node enters a sharing phase in which some or all participant nodes are ready to
share their local parameters. In an implementation, the master node writse the transactions as a block on the
distributed ledger 42 using a blockchain API. In the operation 316, the master node signals completion of the
combination. Forexample, the master node transmits a blockchain transaction indicating its state (that it combined
the local parameters into the final parameters).
In the operation 318, each participant node obtains and applies the final parameters on their local models. In the
operation 320, the master node signals completion of an iteration and relinquishes controlas master nodefor the
iteration.
Figure 9 illustrates an example process 400 at a node that participates in an iteration of model building using
blockchain. In the operation 402, a participant nod enrolls with the blockchain network to participate in an iteration

34
of model training. In the operation 404, the participant node participates in a consensus decision to enroll a second
node that requests to participate in the iteration. The consensus decision can be based on factors such as, for
example, the requesting node's credentials/permission, current state, whether it has stale data, and/or other factors.
In the operation 406, the participant node obtains local training data. The local training data can be accessible at the
participant node, but not accessible to the other participant nodes. In the operation 408, the participant node trains a
local model based on the local training dataset. Such model training is based on the machine learning that is
executed on the local training dataset. In the operation 410, the participant node obtains the local training
parameter. For example, the local training parameter can be an output of model training at the participant node.
In the operation 412, the participant node generates a blockchain transaction that indicates it is ready to share its
local training parameters. In the operation 414, the participant node provides its training parameters to a master
node, which is elected by the participant node along with other participant nodes in the blockchain network. In the
operation 416, the participant node obtains final training parameters that were generated at the master node, which
generated the final training parameters based on the training parameters provided by the participant node and other
participant nodes for the iteration as well. In the operation 418, the participant node applies the final training
parameters to the local model and updates its state (indicating that the local model has been updated with the
current iteration's final training parameters).

35
Figure 9. Example process at a node that participates in an iteration of model building using blockchain.

36
Figure 10 illustrates an example process 500 at a master node elected to generate final training parameters based on
training parameters from participant nodes in an iteration of model building using blockchain. In the operation 502,
the master node generates a distributed ledger block that indicates a sharing phase is in progress. In the operation
504, the master node obtains blockchain transactions from participant nodes. These transactions include indications
that a participant node is ready to share its local training parameters.
In the operation 506, the master node writes the transactions to a distributed ledger block, and adds the block to the
distributed ledger. In the operation 508, the master node identifies a location of training parameters generated by
the participant nodes that submitted the transactions. In the operation 510, the master node generates final training
parameters based on the obtained training parameters. Forexample, the master node merges the obtained training
parameters to generate the final training parameters.
In the operation 512, the master node makes the final training parameters available to the participant nodes. Each
participant nodeobtains the final training parameters to update its local model using the final training parameters.
In the operation 514, the master node updates its state to indicate that the final training parameters are available.

37
Figure 10. Example process at a master node elected to generate final training parameters.

38
5. Predictive Maintenance Platform for Industrial Machine using Industrial IoT
Heavy industrial environments, such as environments for large scale manufacturing (e.g., aircraft, ship, automobile
manufacturing), energy production environments (e.g., oil and gas plants), energy extraction environments (e.g.,
mining), construction environments (e.g., construction of large buildings) involve highly complex machines and
workflows, in which operators must account for a host of parameters and metrics in order to optimize design,
development, deployment, and operation of different technologies in order to improve overall results.
Many of the large industrial machines that require ongoing maintenance, service and repairs are involved in high
stakes production processesand other processes, suchas energy production, manufacturing, mining, drilling, and
transportation, that preferably involve minimal or no interruption. An unanticipated problem, or an extended delay
in a service operation that requires a shutdown of a machine that is critical to such a process cancostthousands, or
even millions of dollars per day.
IoT enables automatic data collection in industrial environments and AI enables improved methods and systems
using collected data to provide improved monitoring, control, intelligent diagnosis of problems and intelligent
optimization of operations in various heavy industrial environments. IoT sensors can collect, and AI can process
data from industrial machines for predicting faults, anticipating needs for maintenance, and facilitating repairs

39
(predictive maintenance). Blockchain based distributed ledger can record and track maintenance transactions
related to the industrial machine.
Strong Force IoT’s patent application US20200133257 illustrates a system for detecting operating characteristics
of an industrial machine. Figure 11 is a diagrammatic view that depicts portions of an overall view of an industrial
IoT data collection, monitoring and controlsystem 100. The system 100 has a server based portion 10 of the
industrial IoT system that can be deployed in the cloud or on an enterprise owner's or operator's premises. The
server portion 10 includes network coding including self-organizing network coding and/or automated
configuration. The network coding model is based on feedback measures, network conditions, or the like, for
highly efficient transport of large amounts of data across the network to and from data collection systems and the
cloud. Network coding provides a wide range of capabilities for intelligence, analytics, remote control, remote
operation, remote optimization, various storage configurations and the like. The various storages include
blockchain storage for supporting transactional data of the system.
A mobile ad hoc network 20 forms a secure, temporal network connection 22 with a cloud 30 or other remote
networking system so that network functions may occurover the mobile ad hoc network within the environment,
without the need for external networks, but at other times information can be sent to and from the server portion 10.
This allows the industrial environment to use the benefits of networking and controltechnologies, while also
providing security, such as preventing cyber-attacks.

40
Figure 11. Diagrammatic view of an industrial IoT data collection, monitoring and control system.

41
The intelligent data collection system is deployed locally, at the edge of an IoT deployment, where heavy industrial
machines are located. This includes various sensors 52, IoT devices 54, data storage capabilities (e.g., data pools
60, or distributed ledger 62), sensor fusion, and the like. The data pools 60 collect data published by machines or
sensors that detect conditions of machines, such as for later consumption by local or remote intelligence. The
distributed ledger system 62 distributes storage throughout the system. The on-device sensorfusion 80 analyzes
data from multiple analog sensors 82 locally or in the cloud using a machine learning 84.
The data marketplace 70 provides the available data that is collected in industrial environments, such as from data
collectors, data pools, and distributed ledgers.

42
6. AI+BlockchainSystemfor Car Sharing Service
Ford’s patent application US20200134592 illustrates a blockchain based car sharing service. A matchmaking
system uses AI to provide the best car sharing service based on clients’ and providers’ records and transaction
history along with proximity and availability of the provider with respectto the client who made the car sharing
request.
Figure 12 shows a block diagram of an example system for the car sharing service. The system architecture 100
includes users (e.g., client 102) and service providers (e.g., provider 104). The client 102 and the provider 104 are
linked via the system architecture to perform a car sharing service 116. Devices associated with the users are
connected on a peer-to-peer network 112, where the user devices represent blockchain nodes on that peer-to-peer
network.

43
Figure 12. Block diagram of an example system for the car sharing service.
When a client 102 makes a request 103, a decentralized matchmaking algorithm 114 matchs the client 102 that
made the request 103 to a service provider 104 on the peer-to-peer network 112. The decentralized matchmaking
algorithm uses AI algorithms and techniques. The decentralized matchmaking algorithm use the client's or
provider's records and transaction history along with proximity and availability of the provider with respectto the

44
client who made the request. The records and transaction history, proximity, availability, and other relevant
information are stored on the peer-to-peer network in accordancewith a blockchain protocol113. Additionally, the
transactions and records are stored and validated on the peer-to-peer network by using the blockchain protocol. The
peer-to-peer network comprises multiple nodes on a public infrastructure or a private infrastructure. Further, the
request is delivered to the provider who accepts the request 109. Accordingly, a payment 106 is facilitated between
the client's digital wallet 105 and the provider's digital wallet 107.
A validation 110 comprises automated validators that serve to validate the request, the service, and the payment for
the service. The blockchain protocolincludes the smart contracts regarding details of the underlying business logic
and transactional details (e.g., details related to the parties involved, terms of service, terms of payment, and the
like in a digital contract format). The smart contracts serve to ensure that the provider will be paid by the client
upon successfulcompletion of a service.
Data stored on the peer-to-peer network under a blockchain protocolare distributed and replicated across the nodes
of the peer-to-peer network, which enables transparency and additional tamper-resistance. A payment processing
server can be used to process a payment between users (e.g., clients and providers).
Figure 13 shows a diagram of an example use case application and process flow 200. A given user 205 requests a
service 212 from a provider 207 on a local peer-to-peer network comprising multiple nodes 203. The provider

45
requests payment 214 to the user. The payments can be made using tokens (e.g., cryptocurrencies). The user of the
local peer-to-peer network 210 can convert the tokes to currency 204 through a currency conversion service 211
provided by a third-party service-provider. The user rewards 213 can be determined based on the user's records
(e.g., transaction history, reputation, and the like), which may be captured as data stored on the files written to the
local peer-to-peer network in accordancewith the blockchain protocol. The analytics data 215 can be generated
from an analysis of the local peer-to-peer network using AI-based algorithm (e.g., using machine learning). The
analytics data can include usage trends, demographic analysis, environmental context data, and related trends, and
the like. The analytics data provides insights to a given service provider such that it improves its service offerings.

46
Figure 13. Diagram of an example use case application and process flow.

47
7. ConnectedAutonomous Vehicle Communication ManagementSystem
Autonomous vehicles can determine surrounding environmental conditions using technology such as radar, laser,
odometry, global positioning system, and computer vision. Control systems in an autonomous vehicle utilize the
environmental information to controlthe autonomous vehicle as it autonomously navigates paths and obstacles,
while abiding by relevant traffic signals and signs. Autonomous vehicles can be applied to public or ride-sharing
transportation (e.g., self-driving taxis or other types of ride-sharing vehicles). A public or ride-sharing autonomous
vehicle can provide transportation services for a large number of passengers per day. Thus, as autonomous vehicle
s become more popular, autonomous vehicle operating safety with regard to other autonomous vehicles and non-
autonomous vehicles is a critical matter that should be considered.
IBM’s patent application US20200010080 illustrates a vehicle to vehicle communication management system to
facilitate operational safety via risk assessment. Vehicle to vehicle communication can be used to facilitate
operational safety of autonomous vehicles via risk assessment and context analysis. Forexample, operational
safety focuses on safe lane passing in which one vehicle overtakes another vehicle on a given roadway can be
considered. Autonomous vehicle’s IoT sensors (e.g., cameras, light sensors, audio sensors, LIDAR, radar,
ultrasonic sensor, etc.) and vehicle to vehicle communications (e.g., 5G based V2X) can be used to aid in
facilitating safe lane passing by detecting signals of a nearby autonomous vehicle or non- autonomous vehicle (e.g.,

48
alerting a danger ahead, requesting to overtake, allowing a vehicle to merge, driver signals, etc.) and contextual
interpreting of signals for situational risk assessment.
The assessmentof situational risks and contextual situations comprises analysis of road conditions, vehicle types,
vehicle payload (e.g., number of passenger, weight of transported goods), road types, weather, need to pass (e.g.,
emergency), daytime versus nighttime driving, and road topology (e.g., curves, curve radii, accident history, hills,
valleys, etc.). The risk assessmentcan employ real-time visual analytics and deep learning algorithms by scanning
the road ahead or back of the given vehicle to attempt to ensure operational safety.
The data exchanged between the given vehicle and the other vehicles can be managed via a blockchain-based
system for data security measure. The autonomous vehicle can receive rating, points, or rewards based on how well
the autonomous vehicle helped to reduce risks, gives appropriate signals for alerting accident location ahead,
and/or nature/value of human interactions. The system can facilitate a mechanism for requesting an autonomous
vehicle to pay to another autonomous vehicle for facilitating a pass, yielding to another vehicle via a blockchain-
based cryptocurrency payment. Blockchain-based DID (decentralized identifier) for an autonomous vehicle can be
used to form a decentralized IoT network, where items (or things) in the network are “smart devices” that are
connected to the blockchain through their corresponding DID. This allows for institutional wide tracking of
vehicles.

49
Figure 14 illustrates a methodology 800 for providing vehicle to vehicle communication to facilitate operational
safety via risk assessment. In step 802, in a given computing node in a given vehicle, the method obtains at the
given computing node data from at one of: (i) sensors on the given vehicle; (ii) computing nodes associated with
other vehicles via at vehicle to vehicle communication protocol;and (iii) data sources along the path of the given
vehicle.
In step 804, the method utilizes at the given computing node a portion of the obtained data to compute a risk
assessment for the given vehicle with respect to an operational safety level of a proposed vehicle action. In step
806, the method initiates or prevents at the given computing node the proposed vehicle action for the given vehicle
based on the computed risk assessment.

50
Figure 14. Methodology for providing vehicle to vehicle communication.

51
8. 5G-basedAI+Blockchain+IoTEdge Computing System
In the coming years, it is expected that there will be a greater need for wirelessly accessible, relatively low-latency,
relatively high-powered computing placed near the edge of networks. It is expected that various AI algorithms will
need to process relatively high-bandwidth streams of data to output results in real-time after that data is acquired.
Examples include processingvideos and other sensordata gathered by self-driving cars, autonomous drones,
wearable computing devices, and other IoT sensors. In many cases, uploading this data to a traditional public cloud
data center to process the data and to generate actionable commands or results is too slow, in part due to the
amount of time taken to convey the data over relatively large geographic distances. This is due, in part, to the time
consumed transmitting the data and results from the speed of light imposing limits on how fast information can be
conveyed over large geographic distances. Additional delays arise from switching and routing equipment along the
path and potential congestion. Accordingly, it is expected that there will be a need to distribute relatively high-
performance computing facilities, such as data centers, over distributed geographic areas. Forexample, distributing
the data centers every few miles in a metropolitan area, county, state, or country, rather than relying exclusively
upon data centers that are geographically concentrated and serve, for example, a continent from a single geographic
location.
Edge computing refers to the transition of compute and storage resources closer to endpoint devices in order to
reduce application latency, improve service capabilities, and improve compliance with security or data privacy

52
requirements. Edge computing provides a cloud-like distributed service that offers orchestration and management
for applications among many types of storage and compute resources. As a result, some implementations of edge
computing have been referred to as the edge cloud. Edge computing use cases in 5G mobile network settings have
been developed for integration with multi-access edge computing (MEC) approaches, also known as mobile edge
computing.
MEC approaches are designed to allow application developers and content providers to access computing
capabilities and an IT service environment in dynamic mobile network settings at the edge of the network.
MEC technology has some advantages when compared to traditional centralized could computing environments.
For example, MEC technology provides a service by service providers to user agent or a user equipment with a
lower latency, a lower cost, a higher bandwidth, a closer proximity, and/or an exposure to real-time radio network
and context information.
Intel’s patent application US20200195495 illtstrates technological solutions for implementing a MEC-based
system to realize 5G network slicing with blockchain traceability. The technological solutions integrates MEC with
various types of IoT or Fog networking implementations as well as dynamic network slicing and resource
utilization management. The technological solutions benefit a variety of use cases, such as 5G network
communications for automotive devices, smart factory sensors, and smart healthcare devices.

53
The technological solutions include providing blockchain quality traceability of 5G network slice instances and
virtual network services necessary for Service Provider Service Level Agreements (SLAs). The technological
solutions techniques can be used for 5G network slicing to enable multiple owners, users, and applications to
utilize a communication network in slice instances through the transfer of computing and communication resources.
Blockchain traceability can be used to provide traceability and tracking (e.g., of resource usage and dynamic
resource allocation during dynamic slice usage) to meet SLAs associated with services delivered in the 5G
communication environment.
The technological solutions include on-demand 5G network slice instance deployments with blockchain
traceability for an informed 5G service supply chain that includes service providers, regulators, and certain 5G
fixed and mobile subscribers. In this regard, the 5G network slice and virtual network service traceability
requirements can be met while leveraging public key encryption hardware acceleration to perform and allow for
on-demand 5G network slice instance creation, deployment, and re-configuration. Additionally, The technological
solutions use AI-based network inferencing functions to perform resourcemanagement in connection with the 5G
network slice instance configuration, deployment, and re-configuration. For example, AI-based network
inferencing functions can be used to dynamically monitor and predict network resource utilization as well as detect
changes in SLAs used in connection with 5G network slice instance management to trigger initial resource
allocation as well as re-allocation of the network resources within a specific network slice instance or among a
group of network slice instances.

54
Figure 15 illustrates a block diagram showing an overview of a configuration for edge computing, which includes a
layer of processing referenced in many of examples as an edge cloud. This network topology can be extended
through the use of 5G network slice instance management using blockchain traceability and AI-based resource
management techniques.
Figure 15. Block diagram showing an overview of a configuration for edge computing.

55
The edge cloud 110B is co-located at an edge location, such as the base station 140B, a local processinghub 150B,
and a central office 120B. The edge cloud is located much closer to the endpoint data sources 160B (e.g.,
autonomous vehicles 161B, user equipment 162B, business, and industrial equipment 163B, video capture devices
164B, drones 165B, smart cities and building devices 166B, sensors and IoT devices 167B, etc.) than the cloud
data center 130B. Compute, memory, and storage resources which are offered at the edges in the edge cloud are
critical to providing ultra-low latency responsetimes for services and functions used by the endpoint data sources
as well as reduce network backhaul traffic from the edge cloud toward cloud data center 130.
Edge computing is performed at or closer to the edge of a network, typically through the use of a computing
platform implemented at base stations, gateways, network routers, or other devices which are much closer to end
point devices producing and consuming the data. Depending on the real-time requirements in a communications
context, a hierarchical structure of data processing and storage nodes can be defined in an edge computing
deployment.
Figure 16 illustrates deployment and orchestration for virtual edge configurations across an edge-computing
system operated among multiple edge nodes and multiple tenants. Specifically, Figure 16 depicts coordination of
the first edge node 122C and the second edge node 124C in an edge-computing system 100C, to fulfill requests and
responses for various client endpoints 110C from various virtual edge instances. The virtual edge instances provide

56
edge compute capabilities and processing in an edge cloud, with access to a cloud/data center 140C for higher-
latency requests for websites, applications, database servers, etc. Thus, the edge cloud enables coordination of
processing among multiple edge nodes for multiple tenants or entities using 5G network slice instance management
using blockchain traceability and AI-based resource management.
Figure 16. Deployment and orchestrationfor virtual edge configurations across anedge-computing system.

57
In the figure, these virtual edge instances include the first virtual edge 132C, offered to a first tenant (Tenant 1),
which offers the first combination of edge storage, computing, and services; and the second virtual edge 134C,
offering a second combination of edge storage, computing, and services, to a second tenant (Tenant 2). The virtual
edge instances 132C, 134C are distributed among the edge nodes 122C, 124C, and include scenarios in which a
request and responseare fulfilled from the same or different edge nodes. The configuration of each edge node
122C, 124C to operate in a distributed yet coordinated fashion with shared memory access occursbased on edge
provisioning functions 150C and resource, blockchain, and slice management (RBSM) functions 170C. The
functionality of the edge nodes 122C, 124C to provide coordinated operation for applications and services, among
multiple tenants, occurs based on orchestration functions 160C.
The RBSM functions performs the functionalities of the AI-based resource management module 160, the
blockchain traceability management module 162, and the slice management module 164. The RBSM functions are
used to determine (or predict) network resource utilization using AI-based network inferencing functions, and
configure the network slice instances based on the network resource utilization.
The edge computing system can be extended to provide orchestration of multiple applications through the use of
(Docker) containers, in a multi-owner, multi-tenant environment. A multi-tenant orchestrator can be used to
perform key management, trust anchor management, and other security functions related to the provisioning and

58
lifecycle of the trusted slice. Accordingly, the edge computing system can fulfill requests and responses for various
client endpoints from multiple virtual edge instances from a cloud or remote data center. The use of these virtual
edge instances supports multiple tenants and multiple applications (e.g., AR/VR, enterprise applications, content
delivery, gaming, compute offload) simultaneously. Further, there can be multiple types of applications within the
virtual edge instances (e.g., normal applications, latency-sensitive applications, latency-critical applications, user
plane applications, networking applications, etc.). The virtual edge instances also can be spanned across systems of
multiple owners at different geographic locations.

59

AI, Blockchain, IoT System Implementation Insights

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to AI, Blockchain, IoT System Implementation Insights

Similar to AI, Blockchain, IoT System Implementation Insights (20)

More from Alex G. Lee, Ph.D. Esq. CLP

More from Alex G. Lee, Ph.D. Esq. CLP (20)

Recently uploaded

Recently uploaded (20)

AI, Blockchain, IoT System Implementation Insights