Talk no. 8 about big data given by Mohamed Seif

1. Big Data Talks Nile University
Talk 8
Big Data Analysis with Signal Processing on
Graphs
Introduction
Fundamentals of Graph Theory
DSP on Graphs
Graph Products
Applications
Mohamed Seif m.seif@nu.edu.eg 8-1

2. Contents
Introduction
Fundamentals of Graph Theory
DSP on Graphs
Graph Products
Applications
Big Data Analysis with Signal Processing on Graphs 8-2

3. Introduction
How to relate the Signal Processing
Definitions with Graphs?
Big Data Analysis with Signal Processing on Graphs 8-3

4. Introduction
How to relate the Signal Processing
Definitions with Graphs?
Big Data Analysis with Signal Processing on Graphs 8-4

5. Contents
Introduction
Fundamentals of Graph Theory
DSP on Graphs
Graph Products
Applications
Big Data Analysis with Signal Processing on Graphs 8-5

6. What Is Graph?
In graph theory, the graph G is defined as a tuple G = (V, E) where
V = {v0, v1, . . . , vN−1} is the set of N nodes and
E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links
between the nodes.
Example:
V = {1, 2, 3, 4}
E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}}
1 2
3 4
Big Data Analysis with Signal Processing on Graphs 8-6

7. What Is Graph?
In graph theory, the graph G is defined as a tuple G = (V, E) where
V = {v0, v1, . . . , vN−1} is the set of N nodes and
E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links
between the nodes.
Example:
V = {1, 2, 3, 4}
E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}}
1 2
3 4
Big Data Analysis with Signal Processing on Graphs 8-7

8. What Is Graph? (cont’d)
In graph theory, the graph G is defined as a tuple G = (V, E) where
V = {v0, v1, . . . , vN−1} is the set of N nodes and
E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links
between the nodes.
Big Data Analysis with Signal Processing on Graphs 8-8

9. What Is Graph?
In graph theory, the graph G is defined as a tuple G = (V, E) where
V = {v0, v1, . . . , vN−1} is the set of N nodes and
E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links
between the nodes.
Big Data Analysis with Signal Processing on Graphs 8-9

10. Alternative Representation of Graphs
1. Adjacency matrix AN×N , defined as
Ai,j =



1 if vi & vj are connected
0 o.w.
2. Laplacian graph LN×N , defined as
L = D − A, where D is the degree matrix
Li,j =



deg(vi) if i = j
−1 if i = j & vi is adjacent to vj
0 o.w.
Big Data Analysis with Signal Processing on Graphs 8-10

11. Contents
Introduction
Fundamentals of Graph Theory
DSP on Graphs
Graph Products
Applications
Big Data Analysis with Signal Processing on Graphs 8-11

12. Graph Signals
Given the graph, the data set forms a graph signal, defined as a map
s : V → C, vn → sn (1)
It is convenient to write graph signals as vectors
s = [s0, s1, . . . , sN−1]
T
∈ CN×1
(2)
Big Data Analysis with Signal Processing on Graphs 8-12

13. Graph Shift
In DSP, a signal shift, implemented as a time delay
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= Cs (3)
where C is the N × N cyclic shift matrix.
DSP on Graphs extends the concept of shift to general graphs by
defining the graph shift as a local operation that replaces a signal value
sn at node vn by a linear combination of the values at neighbors of vn
weighted by their edge weights:
˜sn =
m∈Nn
An,msm (4)
Big Data Analysis with Signal Processing on Graphs 8-13

14. Graph Shift (cont’d)
It can be interpreted as a first-order interpolation, weighted
averaging, or regression on graphs, which is a widely used operation
in graph regression, distributed consensus, telecommunications.
Then, the graph shift is written as
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= As (5)
Big Data Analysis with Signal Processing on Graphs 8-14

15. Graph Filters and Z-Transform
In signal processing, a filter is a system H(.) that takes an input
signal s and outputs a signal:
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= H(s) (6)
Among the most widely used filters are linear shift-ivariant (LSI) ones.
The z-transform provides a convenient representation for signals and
filters in DSP. (In short)
An alternative representation for the output signal is given by
˜s = h(C)s (7)
where h(c) =
N−1
n=0 hnCn
(Resultant is a circulant matrix)
Big Data Analysis with Signal Processing on Graphs 8-15

16. Graph Filters and Z-Transform
In signal processing, a filter is a system H(.) that takes an input
signal s and outputs a signal:
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= H(s) (6)
Among the most widely used filters are linear shift-ivariant (LSI) ones.
The z-transform provides a convenient representation for signals and
filters in DSP. (In short)
An alternative representation for the output signal is given by
˜s = h(C)s (7)
where h(c) =
N−1
n=0 hnCn
(Resultant is a circulant matrix)
Big Data Analysis with Signal Processing on Graphs 8-16

17. Graph Filters and Z-Transform
In signal processing, a filter is a system H(.) that takes an input
signal s and outputs a signal:
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= H(s) (6)
Among the most widely used filters are linear shift-ivariant (LSI) ones.
The z-transform provides a convenient representation for signals and
filters in DSP. (In short)
An alternative representation for the output signal is given by
˜s = h(C)s (7)
where h(c) =
N−1
n=0 hnCn
(Resultant is a circulant matrix)
Big Data Analysis with Signal Processing on Graphs 8-17

18. Graph Filters and Z-Transform
In signal processing, a filter is a system H(.) that takes an input
signal s and outputs a signal:
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= H(s) (6)
Among the most widely used filters are linear shift-ivariant (LSI) ones.
The z-transform provides a convenient representation for signals and
filters in DSP. (In short)
An alternative representation for the output signal is given by
˜s = h(C)s (7)
where h(c) =
N−1
n=0 hnCn
(Resultant is a circulant matrix)
Big Data Analysis with Signal Processing on Graphs 8-18

19. Graph Filters and Z-Transform (cont’d)
DSP on Graphs extends the concept of filters to general graphs.
Similarly to the extension of the time shift to the graph shift, filters
are generalized to graph filters as polynomials in the graph shift , and
all LSI graph filters have the form
h(A) =
L−1
l=0
hlAl
(8)
In analogy with signal filters, the graph filter output is given by
˜s = h(A)s (9)
Big Data Analysis with Signal Processing on Graphs 8-19

20. Graph Fourier Transform
Mathematically, a Fourier transform with respect to a set of operators
is the expansion of a signal into a basis of the operators eigen
functions.
Since in signal processing the operators of interest are filters, DSPG
defines the Fourier transform with respect to the graph filters.
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= GFT{s} (10)
Big Data Analysis with Signal Processing on Graphs 8-20

21. Graph Fourier Transform
Mathematically, a Fourier transform with respect to a set of operators
is the expansion of a signal into a basis of the operators eigen
functions.
Since in signal processing the operators of interest are filters, DSPG
defines the Fourier transform with respect to the graph filters.
˜s = [ ˜s0, ˜s1, . . . , ˜sN−1]
T
= GFT{s} (10)
Big Data Analysis with Signal Processing on Graphs 8-21

22. Graph Fourier Transform (cont’d)
For simplicity, assume that A is diagonalizable and its decomposition
is
A = V ΛV −1
(11)
where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N
are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are
eigenvalues of A
In general A can be diagonalized using Jordan decomposition.
Big Data Analysis with Signal Processing on Graphs 8-22

23. Graph Fourier Transform (cont’d)
For simplicity, assume that A is diagonalizable and its decomposition
is
A = V ΛV −1
(11)
where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N
are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are
eigenvalues of A
In general A can be diagonalized using Jordan decomposition.
Big Data Analysis with Signal Processing on Graphs 8-23

24. Graph Fourier Transform (cont’d)
The eigenfunctions of graph filters h(A) are given by the eigenvectors
of the graph shift matrix A
Since the expansion into the eigenbasis is given by the multiplication
with the inverse eigenvector matrix, which always exists, the graph
Fourier transform is well defined and computed as
ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1]
T
= V −1
s (12)
= Fs (13)
where F = V −1
is the graph Fourier transform matrix.
Big Data Analysis with Signal Processing on Graphs 8-24

25. Graph Fourier Transform (cont’d)
The eigenfunctions of graph filters h(A) are given by the eigenvectors
of the graph shift matrix A
Since the expansion into the eigenbasis is given by the multiplication
with the inverse eigenvector matrix, which always exists, the graph
Fourier transform is well defined and computed as
ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1]
T
= V −1
s (12)
= Fs (13)
where F = V −1
is the graph Fourier transform matrix.
Big Data Analysis with Signal Processing on Graphs 8-25

26. Graph Fourier Transform (cont’d)
The eigenfunctions of graph filters h(A) are given by the eigenvectors
of the graph shift matrix A
Since the expansion into the eigenbasis is given by the multiplication
with the inverse eigenvector matrix, which always exists, the graph
Fourier transform is well defined and computed as
ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1]
T
= V −1
s (12)
= Fs (13)
where F = V −1
is the graph Fourier transform matrix.
Big Data Analysis with Signal Processing on Graphs 8-26

27. Graph Fourier Transform (cont’d)
The inverse graph Fourier transform reconstructs the graph signal
from is frequency content by combining graph frequency components
weighted by the coefficients of the signal’s graph Fourier transform:
s = ˆs0v0 + ˆs1v1 + · · · + ˆsN−1vN−1 (14)
= F−1
s = V s (15)
Big Data Analysis with Signal Processing on Graphs 8-27

28. Low and High Frequencies on Graphs
The values ˆsn in (12) are the signal’s expansion in the eigenvector
basis and represent the frequency content of the signal s.
The eigenvalues λn of the shift matrix A represent graph frequency
content, and the eigenvectors vn represent the corresponding graph
frequency component.
To conclude, the higher λn, the higher frequency content and vice
versa.
Big Data Analysis with Signal Processing on Graphs 8-28

29. Low and High Frequencies on Graphs
The values ˆsn in (12) are the signal’s expansion in the eigenvector
basis and represent the frequency content of the signal s.
The eigenvalues λn of the shift matrix A represent graph frequency
content, and the eigenvectors vn represent the corresponding graph
frequency component.
To conclude, the higher λn, the higher frequency content and vice
versa.
Big Data Analysis with Signal Processing on Graphs 8-29

30. Low and High Frequencies on Graphs
The values ˆsn in (12) are the signal’s expansion in the eigenvector
basis and represent the frequency content of the signal s.
The eigenvalues λn of the shift matrix A represent graph frequency
content, and the eigenvectors vn represent the corresponding graph
frequency component.
To conclude, the higher λn, the higher frequency content and vice
versa.
Big Data Analysis with Signal Processing on Graphs 8-30

31. Low and High Frequencies on Graphs
The values ˆsn in (12) are the signal’s expansion in the eigenvector
basis and represent the frequency content of the signal s.
The eigenvalues λn of the shift matrix A represent graph frequency
content, and the eigenvectors vn represent the corresponding graph
frequency component.
To conclude, the higher λn, the higher frequency content and vice
versa.
Big Data Analysis with Signal Processing on Graphs 8-31

32. Frequency Response of Graph Filters
In addition to expressing the frequency content of graph signals, the
graph Fourier transform also characterizes the effect of filters on the
frequency content of signals.
The filtering operation ˜s = h(A)s can be written using
h(A) =
L−1
l=0 hlAl
and ˆs = V −1
s as follows
˜s = h(A)s = h(F−1
AF)s = F−1
h(Λ)Fs (16)
where h(Λ) is a diagonal matrix with values h(λn) =
L−1
l=0 hlλl
n
As a result,
˜s = h(A)s ⇔ Fs = h(Λ)s (17)
Big Data Analysis with Signal Processing on Graphs 8-32

33. Frequency Response of Graph Filters
In addition to expressing the frequency content of graph signals, the
graph Fourier transform also characterizes the effect of filters on the
frequency content of signals.
The filtering operation ˜s = h(A)s can be written using
h(A) =
L−1
l=0 hlAl
and ˆs = V −1
s as follows
˜s = h(A)s = h(F−1
AF)s = F−1
h(Λ)Fs (16)
where h(Λ) is a diagonal matrix with values h(λn) =
L−1
l=0 hlλl
n
As a result,
˜s = h(A)s ⇔ Fs = h(Λ)s (17)
Big Data Analysis with Signal Processing on Graphs 8-33

34. Frequency Response of Graph Filters
In addition to expressing the frequency content of graph signals, the
graph Fourier transform also characterizes the effect of filters on the
frequency content of signals.
The filtering operation ˜s = h(A)s can be written using
h(A) =
L−1
l=0 hlAl
and ˆs = V −1
s as follows
˜s = h(A)s = h(F−1
AF)s = F−1
h(Λ)Fs (16)
where h(Λ) is a diagonal matrix with values h(λn) =
L−1
l=0 hlλl
n
As a result,
˜s = h(A)s ⇔ Fs = h(Λ)s (17)
Big Data Analysis with Signal Processing on Graphs 8-34

35. Frequency Response of Graph Filters (cont’d)
That is, the frequency content of a filtered signal is modified by
multiplying its frequency content element-wise by h(λn) . These
values represent the graph frequency response of the graph
filter.
The relation is a generalization of the classical convolution theorem
to graphs: filtering a graph signal in the graph domain is equivalent in
the frequency domain to multiplying the signal spectrum by the
frequency response of the graph filter. ˜s = h(A)s ⇔ Fs = h(Λ)s
Big Data Analysis with Signal Processing on Graphs 8-35

36. Frequency Response of Graph Filters (cont’d)
That is, the frequency content of a filtered signal is modified by
multiplying its frequency content element-wise by h(λn) . These
values represent the graph frequency response of the graph
filter.
The relation is a generalization of the classical convolution theorem
to graphs: filtering a graph signal in the graph domain is equivalent in
the frequency domain to multiplying the signal spectrum by the
frequency response of the graph filter. ˜s = h(A)s ⇔ Fs = h(Λ)s
Big Data Analysis with Signal Processing on Graphs 8-36

37. Contents
Introduction
Fundamentals of Graph Theory
DSP on Graphs
Graph Products
Applications
Big Data Analysis with Signal Processing on Graphs 8-37

38. Product Graphs
Consider two graphs G1 = (V1, A1) and G2 = (V2, A2) with |V1| = N1
and |V2| = N2 nodes, respectively. The product graph, denoted by , of
G1 and G2 is the graph
G = G1 G2 = (V, A ) (18)
with |V| = N1N2 and dim(A ) = N1N2 × N1N2.
Big Data Analysis with Signal Processing on Graphs 8-38

39. Common Product Graphs Types
1. Kronecker product
A⊗ = A1 ⊗ A2 (19)
Example:
If we have two matrices B ∈ CM×N
and C ∈ CK×L
, then the
Kronecker product is defined as follows
B ⊗ C =




b1,1C
...
bM,1C
· · ·
...
· · ·
b1,M C
...
bM,M C



 ∈ CMK×NL
(20)
Big Data Analysis with Signal Processing on Graphs 8-39

40. Common Product Graphs Types
1. Cartesian product
A× = A1 ⊗ IN2
+ IN1
⊗ A2 (21)
2. Strong product
A = A1 ⊗ A2 + A1 ⊗ IN2 + IN1 ⊗ A2 (22)
Big Data Analysis with Signal Processing on Graphs 8-40

41. Examples on Product Graphs
Big Data Analysis with Signal Processing on Graphs 8-41

42. Examples on Product Graphs
Big Data Analysis with Signal Processing on Graphs 8-42

43. Examples on Product Graphs
Big Data Analysis with Signal Processing on Graphs 8-43

44. Notes on Product Graphs
Big Data Analysis with Signal Processing on Graphs 8-44

45. Signal Processing on Product Graphs
The computation of filtering and Fourier transform on graphs and
improve algorithms, data storage, and memory access for large data sets
can modularized thanks to graph products. Such as
Filtering
Computation complexity: O(N2
) =⇒ O(N(N1 + N2))
Fourier transform
Computation complexity: O(N3
) =⇒ O(N3
1 + N3
2 )
Big Data Analysis with Signal Processing on Graphs 8-45

46. Signal Processing on Product Graphs
The computation of filtering and Fourier transform on graphs and
improve algorithms, data storage, and memory access for large data sets
can modularized thanks to graph products. Such as
Filtering
Computation complexity: O(N2
) =⇒ O(N(N1 + N2))
Fourier transform
Computation complexity: O(N3
) =⇒ O(N3
1 + N3
2 )
Big Data Analysis with Signal Processing on Graphs 8-46

47. Signal Processing on Product Graphs
The computation of filtering and Fourier transform on graphs and
improve algorithms, data storage, and memory access for large data sets
can modularized thanks to graph products. Such as
Filtering
Computation complexity: O(N2
) =⇒ O(N(N1 + N2))
Fourier transform
Computation complexity: O(N3
) =⇒ O(N3
1 + N3
2 )
Big Data Analysis with Signal Processing on Graphs 8-47

48. Contents
Introduction
Fundamentals of Graph Theory
DSP on Graphs
Graph Products
Applications
Big Data Analysis with Signal Processing on Graphs 8-48

49. Applications
Like-wise traditional DSP problems:
Data compression
Fourier transform or through wavelet expansions, or adaptive filter design
Detection of corrupted data
High pass filter
Big Data Analysis with Signal Processing on Graphs 8-49

50. Applications
Like-wise traditional DSP problems:
Data compression
Fourier transform or through wavelet expansions, or adaptive filter design
Detection of corrupted data
High pass filter
Big Data Analysis with Signal Processing on Graphs 8-50

51. Applications
Like-wise traditional DSP problems:
Data compression
Fourier transform or through wavelet expansions, or adaptive filter design
Detection of corrupted data
High pass filter
Big Data Analysis with Signal Processing on Graphs 8-51

52. Challenges of Big Data
While there is no single, universally agreed upon set of properties that
define big data.
Some of the commonly mentioned ones are volume, velocity, and
variety of data.
Big Data Analysis with Signal Processing on Graphs 8-52

53. Challenges of Big Data
While there is no single, universally agreed upon set of properties that
define big data.
Some of the commonly mentioned ones are volume, velocity, and
variety of data.
Big Data Analysis with Signal Processing on Graphs 8-53

54. Challenges of Big Data
Some of the commonly mentioned ones are volume, velocity, and
variety of data.
First of all, the sheer volume of data to be processed requires efficient
distributed and scalable storage, access, and processing.
High velocity of new data arrival demands fast algorithms to prevent
bottlenecks and explosion of the data volume and to extract valuable
information from the data and incorporate it into the decision-making
process in real time. (FFT in DSP)
Finally, collected data sets contain information in all varieties and forms,
including numerical, textual, and visual data. To generalize data analysis
techniques to diverse data sets, we need a common representation
framework for data sets and their structure.
Big Data Analysis with Signal Processing on Graphs 8-54

55. Challenges of Big Data
Some of the commonly mentioned ones are volume, velocity, and
variety of data.
First of all, the sheer volume of data to be processed requires efficient
distributed and scalable storage, access, and processing.
High velocity of new data arrival demands fast algorithms to prevent
bottlenecks and explosion of the data volume and to extract valuable
information from the data and incorporate it into the decision-making
process in real time. (FFT in DSP)
Finally, collected data sets contain information in all varieties and forms,
including numerical, textual, and visual data. To generalize data analysis
techniques to diverse data sets, we need a common representation
framework for data sets and their structure.
Big Data Analysis with Signal Processing on Graphs 8-55

56. Challenges of Big Data
Some of the commonly mentioned ones are volume, velocity, and
variety of data.
First of all, the sheer volume of data to be processed requires efficient
distributed and scalable storage, access, and processing.
High velocity of new data arrival demands fast algorithms to prevent
bottlenecks and explosion of the data volume and to extract valuable
information from the data and incorporate it into the decision-making
process in real time. (FFT in DSP)
Finally, collected data sets contain information in all varieties and forms,
including numerical, textual, and visual data. To generalize data analysis
techniques to diverse data sets, we need a common representation
framework for data sets and their structure.
Big Data Analysis with Signal Processing on Graphs 8-56

57. Challenges of Big Data
Some of the commonly mentioned ones are volume, velocity, and
variety of data.
First of all, the sheer volume of data to be processed requires efficient
distributed and scalable storage, access, and processing.
High velocity of new data arrival demands fast algorithms to prevent
bottlenecks and explosion of the data volume and to extract valuable
information from the data and incorporate it into the decision-making
process in real time. (FFT in DSP)
Finally, collected data sets contain information in all varieties and forms,
including numerical, textual, and visual data. To generalize data analysis
techniques to diverse data sets, we need a common representation
framework for data sets and their structure.
Big Data Analysis with Signal Processing on Graphs 8-57

58. Thank You!
Big Data Analysis with Signal Processing on Graphs 8-58