20190413 Shared in pytorch hsinchu

1. Introduction to Julia
Julia Taiwan發起人 杜岳華
1

2. 2

3. 3

4. python
★ 22496
golang
★ 53368
nodejs
★ 58292
rust
★ 34079

5. Why Julia?
5

6. In scientific computing and data
science…
6

7. Other users
7

8. Avoid two language problem
8
Rapid development Performance

9. 速度與可讀性的取捨
 itertools的效能
 「一般來說，我們不會去優化所有的程式碼，因為優化有很
大的代價:一般性與可讀性。 通常跑得快與寫的快，是要做
取捨的。 這裡的例子很好想像，大家只要比較R的程式碼與
Rcpp
http://wush.ghost.io/itertools-performance/
9

10. 使用Julia就不用做取捨了阿!!
10

11. Julia的特色
 Write like Python, run like C.
 擁有 python 的可讀性 (readibility)
 擁有 C 的效能
 Easy to parallelism
 內建套件管理器
 ...
11

12. Julia code
a = [1, 2, 3, 4, 5]
function square(x)
return x^2
end
for x in a
println(square(x))
end
12

13. https://julialang.org/benchmarks/
Julia performance
13

14. Who use Julia?
14

15.  Nobel prize in economic sciences
 The founder of QuantEcon
 “His team at NYU uses Julia for macroeconomic modeling and contributes
to the Julia ecosystem.”
https://juliacomputing.com/case-studies/thomas-sargent.html
15

16.  In 2015, economists at the Federal Reserve Bank of New York (FRBNY)
published FRBNY’s most comprehensive and complex macroeconomic
models, known as Dynamic Stochastic General Equilibrium, or DSGE
models, in Julia.
https://juliacomputing.com/case-studies/ny-fed.html
16

17.  UK cancer researchers turned to Julia to run simulations of tumor growth.
Nature Genetics, 2016
 Approximate Bayesian Computation (ABC) algorithms require potentially millions of
simulations - must be fast
 BioJulia project for analyzing biological data in Julia
 Bayesian MCMC methods Lora.jl and Mamba.jl
https://juliacomputing.com/case-studies/nature.html
17

18.  IBM and Julia Computing analyzed eye fundus images provided by Drishti
Eye Hospitals.
 Timely screening for changes in the retina can help get them to treatment
and prevent vision loss. Julia Computing’s work using deep learning
makes retinal screening an activity that can be performed by a trained
technician using a low cost fundus camera.
https://juliacomputing.com/case-studies/ibm.html
18

19.  Path BioAnalytics is a computational biotech company developing novel
precision medicine assays to support drug discovery and development,
and treatment of disease.
https://juliacomputing.com/case-studies/pathbio.html
19

20.  The Sloan Digital Sky Survey contains nearly 5 million telescopic images
of 12 megabytes each – a dataset of 55 terabytes.
 In order to analyze this massive dataset, researchers at UC Berkeley and
Lawrence Berkeley National Laboratory created a new code named
Celeste.
https://juliacomputing.com/case-studies/intel-astro.html
20

21. http://pkg.julialang.org/pulse.html
Julia Package Ecosystem Pulse
21

22. 22

23. 23

24. Julia CI/CD
24

25. 25

26. 26

27. 27

28. 28

29. 29

30. 30

31. IDE
31

32. Juno
32

33. 33

34. 34

35. Introduction to Julia
35

36. 數字
 Julia 中支援的數字型別
 整數
 浮點數
 有理數
 複數
36

37. 整數跟浮點數
Integer
Int8
Int16
Int32
Int64
Int128
Unsigned
Uint8
Uint16
Uint32
Uint64
Uint128
Float
Float16
Float32
Float64
37

38. 有理數
 有理數表示
 自動約分
 自動調整負號
 接受分母為0
2//3 # 2//3
-6//12 # -1//2
5//-20 # -1//4
5//0 # 1//0
numerator(2//10) # 1
denominator(7//14) # 2
2//4 + 1//7 # 9//14
3//10 * 6//9 # 1//5
10//15 == 8//12 # true
float(3//4) # 0.75 38

39. 複數
1 + 2im
(1 + 2im) + (3 - 4im) # 4 - 2im
(1 + 2im)*(3 - 4im) # 11 + 2im
(-4 + 3im)^(2 + 1im) # 1.950 + 0.651im
real(1 + 2im) # 1
imag(3 + 4im) # 4
conj(1 + 2im) # 1 - 2im
abs(3 + 4im) # 5.0
angle(3 + 3im)/pi*180 # 45.0
39

40. 宣告變數
 指定或不指定型別
x = 5
y = 4
z = x + y
println(z) # 9
40

41. 動態型別
 Value is immutable
x = 5
println(x) # 5
println(typeof(x)) # Int64
x = 6.0
println(x) # 6.0
println(typeof(x)) # Float64
41

42. 算術運算子
 +x： 就是x本身
 -x： 變號
 x + y, x - y, x * y, x / y： 一般四則運算
 div(x, y)： 商
 x % y： 餘數，也可以用rem(x, y)
 x y： 反除，等價於y / x
 x ^ y： 次方
42

43. 位元運算子
 ~x： bitwise not
 x & y： bitwise and
 x | y： bitwise or
 x ⊻ y: bitwise xor
 x >>> y：無正負號，將x的位元右移y個位數
 x >> y：保留正負號，將x的位元右移y個位數
 x << y： 將x的位元左移y個位數
https://www.technologyuk.net/mathematics/number-systems/images/binary_number.gif
43

44. 更新運算子
 +=
 -=
 *=
 /=
 =
 %=
 ^=
 &=
 |=
 ⊻=
 >>>=
 >>=
 <<=
x += 5
等價於
x = x + 5
44

45. 比較運算子
 x == y：等於
 x != y, x ≠ y：不等於
 x < y：小於
 x > y：大於
 x <= y, x ≤ y：小於或等於
 x >= y, x ≥ y：大於或等於
a, b, c = (1, 3, 5)
a < b < c # true
45

46. 強型別
 算術運算會自動轉換
3.14 * 4 # 12.56
parse(“5”) # 5
convert(String, 5) # “5”
46

47. If 判斷式
 短路邏輯
if <判斷式>
<程式碼>
end
if 3 > 5 && 10 > 0
…
end
47
if <判斷式1>
<程式碼1>
elseif <判斷式2>
<程式碼2>
else
<程式碼3>
end

48. 迴圈
while <持續條件判斷式>
<程式碼>
end
48
for i = 1:5 # for迴圈，有限的迴圈次數
println(i)
end

49. 函式
function add(a, b)
c = a + b
return c
end
49
add(a, b) = a + b

50. 匿名函式
 執行效能與一般函式相同，不會有效能損失
50
add = (a, b) -> a + b
p = () -> println(“hello”)
plusone = x -> x + 1

51. 函式 – 參數傳遞
 pass-by-sharing
5x
function foo(a)
end
a
51

52. 陣列
 homogenous
 start from 1
 mutable
[ ]2 3 5
A = [2, 3, 5]
A[2] # 3
52

53. 多維陣列
A = [0, -1, 1;
1, 0, -1;
-1, 1, 0]
A[1, 2]
53

54. 陣列搭配迴圈
strings = ["foo","bar","baz"]
for s in strings
println(s)
end
54

55. 數值運算
 介紹各種 Array 函式
zeros(Float64, 2, 2) # 2-by-2 matrix with 0
ones(Float64, 3, 3) # 3-by-3 matrix with 1
trues(2, 2) # 2-by-2 matrix with true
eye(3) # 3-by-3 diagnal matrix
rand(2, 2) # 2-by-2 matrix with random number
55

56. Comprehension
[x for x = 1:3]
[x for x = 1:20 if x % 2 == 0]
["$x * $y = $(x*y)" for x=1:9, y=1:9]
[1, 2, 3]
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
[“1 * 1 = 1“, “1 * 2 = 2“, “1 * 3 = 3“ ...]
56

57. Tuple
 Immutable
tup = (1, 2, 3)
tup[1] # 1
tup[1:2] # (1, 2)
(a, b, c) = (1, 2, 3)
57

58. Set
 Mutable
filled = Set([1, 2, 2, 3, 4])
push!(filled, 5)
intersect(filled, other)
union(filled, other)
setdiff(Set([1, 2, 3, 4]), Set([2, 3, 5]))
Set([i for i=1:10])
58

59. Dict
 Mutable
filled = Dict("one"=> 1, "two"=> 2, "three"=> 3)
keys(filled)
values(filled)
Dict(x=> i for (i, x) in enumerate(["one", "two", "three", "four"]))
59

60. Julia special features
60

61. 支援 Unicode符號及 latex
 打`alpha<tab>` => α
 α = 1 # 作為變數名稱
 打`alpha<tab>^2<tab>` => α2
 α2 = () -> 4 # 作為函式名稱
 μ = 0
 σ = 1
 normal = Normal(μ, σ)
61

62. Easy to optimize
 Allow generalization and flexibility, and enable to optimize.
 Hints:
 Avoid global variables
 Add type declarations
 Measure performance with @time and pay attention to memory
allocation
 ……
62

63. Easy to profile
 Use @time
 ProfileView.view()
63

64. Easy to parallelize
for i = 1:100000
do_something()
end
@parallel for i = 1:100000
do_something()
end
64

65. Package manager
Julia mode:
julia> using Pkg
julia> Pkg.update()
julia> Pkg.add(“Foo”)
julia> Pkg.rm(“Foo”)
65
Pkg mode:
v(1.1) pkg> update
v(1.1) pkg> add Foo
v(1.1) pkg> rm Foo

66. @code_native
julia> @code_native add(1, 2)
.text
Filename: REPL[2]
pushq %rbp
movq %rsp, %rbp
Source line: 2
leaq (%rcx,%rdx), %rax
popq %rbp
retq
nopw (%rax,%rax)
function add(a, b)
return a+b
end
66

67. @code_llvm
julia> @code_llvm add(1, 2.0)
; Function Attrs: uwtable
define double @julia_add_71636(i64, double) #0 {
top:
%2 = sitofp i64 %0 to double
%3 = fadd double %2, %1
ret double %3
}
function add(a, b)
return a+b
end
67

68. Type system
68

69. Type system
 Use type, not class
 Define methods out of type
 Multiple dispatch on types
 Type hierarchy
 Traits for method interface
69

70. Use type, not class
 Type!
struct Dog
name::String
color::String
end
dog = Dog(“Tom”, “brown”)
Name: Tom
Color: brown

71. Define methods out of type
function color(a::Animal)
return a.color
end
function voice(d::Dog)
return "bark"
end
function voice(c::Cat)
return "meow"
end

72. Multiple dispatch on types
function double(obj::Foo, x)
return 2*x
end
function double(obj::Bar, x)
return string(x)*2
end
double
double
args
(obj::Foo, x::Any)
(obj::Bar, x::Any)

73. Type Hierarchy
Ref:https://en.wikibooks.org/wiki/Introducing_Julia/Types

74. Traits for method interface
 Traits define a set of functions
 Implement a trait with types
 Independent of type hierarchy
74
https://github.com/mauro3/SimpleTraits.jl

75. 泛型
 支援參數化型別及參數化方法
 參數化型別
 像 C++ 的 template
 參數化方法
 Magic!
75

76. Metaprogramming
 把程式碼當成資料結構般操作
 Code as data structure.
 Macro
 Generated function
76

77. Data science in Julia
77

78. 78

79. 79

80. 80

81. 81

82. 82

83. DataFrames.jl
julia> using DataFrames
julia> df = DataFrame(A = 1:4, B = ["M", "F", "F", "M"])
4×2 DataFrame
│ Row │ A │ B │
├─────┼───┼───┤
│ 1 │ 1 │ M │
│ 2 │ 2 │ F │
│ 3 │ 3 │ F │
│ 4 │ 4 │ M │
83

84. DataFrames.jl
julia> df[:A]
4-element Array{Int64,1}:
1
2
3
4
julia> df[2, :A]
2
84

85. CSV.jl
julia> df = CSV.read("data.csv")
julia> df = DataFrame(A = 1:10);
julia> CSV.write("output.csv", df)
85

86. DataFrames.jl
julia> names = DataFrame(ID = [1, 2], Name = ["John Doe",
"Jane Doe"])
julia> jobs = DataFrame(ID = [1, 2], Job = ["Lawyer",
"Doctor"])
julia> full = join(names, jobs, on = :ID)
2×3 DataFrame
│ Row │ ID │ Name │ Job │
├─────┼────┼──────────┼────────┤
│ 1 │ 1 │ John Doe │ Lawyer │
│ 2 │ 2 │ Jane Doe │ Doctor │ 86

87. Query.jl
julia> q1 = @from i in df begin
@where i.age > 40
@select {number_of_children=i.children, i.name}
@collect DataFrame
end
87

88. StatsBase.jl
 Mean Functions
 mean(x, w)
 geomean(x)
 harmmean(x)
 Scalar Statistics
 var(x, wv[; mean=...])
 std(x, wv[; mean=...])
 mean_and_var(x[, wv][, dim])
 mean_and_std(x[, wv][, dim])
 zscore(X, μ, σ)
 entropy(p)
 crossentropy(p, q)
 kldivergence(p, q)
 percentile(x, p)
 nquantile(x, n)
 quantile(x)
 median(x, w)
 mode(x)
88

89. StatsBase.jl
 Sampling from Population
 sample(a)
 Correlation Analysis of Signals
 autocov(x, lags[; demean=true])
 autocor(x, lags[; demean=true])
 corspearman(x, y)
 corkendall(x, y)
89

90. Distributions.jl
 Continuous Distributions
 Beta(α, β)
 Chisq(ν)
 Exponential(θ)
 Gamma(α, θ)
 LogNormal(μ, σ)
 Normal(μ, σ)
 Uniform(a, b)
 Discrete Distributions
 Bernoulli(p)
 Binomial(n, p)
 DiscreteUniform(a, b)
 Geometric(p)
 Hypergeometric(s, f, n)
 NegativeBinomial(r, p)
 Poisson(λ)
90

91. GLM.jl
91
julia> data = DataFrame(X=[1,2,3], Y=[2,4,7])
3x2 DataFrame
|-------|---|---|
| Row # | X | Y |
| 1 | 1 | 2 |
| 2 | 2 | 4 |
| 3 | 3 | 7 |

92. GLM.jl
92
julia> OLS = glm(@formula(Y ~ X), data, Normal(),
IdentityLink())
DataFrameRegressionModel{GeneralizedLinearModel,Float64}:
Coefficients:
Estimate Std.Error z value Pr(>|z|)
(Intercept) -0.666667 0.62361 -1.06904 0.2850
X 2.5 0.288675 8.66025 <1e-17

93. GLM.jl
93
julia> newX = DataFrame(X=[2,3,4]);
julia> predict(OLS, newX, :confint)
3×3 Array{Float64,2}:
4.33333 1.33845 7.32821
6.83333 2.09801 11.5687
9.33333 1.40962 17.257
# The columns of the matrix are prediction, 95% lower and
upper confidence bounds

94. Gadfly.jl
94

95. Plots.jl
95
mutable struct Lorenz
dt; σ; ρ; β; x; y; z
end
function step!(l::Lorenz)
dx = l.σ*(l.y - l.x) ; l.x += l.dt * dx
dy = l.x*(l.ρ - l.z) - l.y ; l.y += l.dt * dy
dz = l.x*l.y - l.β*l.z ; l.z += l.dt * dz
end
attractor = Lorenz(dt = 0.02, σ = 10., ρ = 28., β = 8//3, x = 1., y = 1., z = 1.)
plt = plot3d(1, xlim=(-25,25), ylim=(-25,25), zlim=(0,50),
title = "Lorenz Attractor", marker = 2)
@gif for i=1:1500
step!(attractor)
push!(plt, attractor.x, attractor.y, attractor.z)
end every 10

96. Data
 JuliaData
 DataFrames.jl
 CSV.jl
 Tables.jl
 CategoricalArrays.jl
 JuliaDB
96

97. Machine learning in Julia
97

98. 98
https://julialang.org/blog/2017/12/ml&pl-zh_tw

99. Yann LeCun: 深度學習需要新的語言
 Facebook 的首席 AI 科學家
 CNN發明人
99Ref: https://venturebeat.com/2019/02/18/facebooks-chief-ai-scientist-deep-learning-may-need-a-new-programming-language/
Pic: https://xconomy.com/boston/2017/11/01/as-facebook-fights-fake-news-lecun-sees-bigger-role-for-a-i/

100. 2019.2.20
10 a.m.

101. Flux.jl
 100% 純 Julia
 支援自動微分
 支援高階抽象以及低階 API
 套件直接相容語言
 支援CUDA，但不依賴 CUDA C 函式
庫
101

102. Flux.jl
 100% 純 Julia
 直接是 Julia 的效能
 支援自動微分
 支援各式函數微分
 支援CUDA，但不依賴 CUDA C 函式
庫
 由 Julia 提供與 CUDA C 函式庫匹敵的效能
102

103. Flux.jl - 支援高階抽象以及低階 API
103
Pic: https://blog.algorithmia.com/introduction-to-loss-functions/
Loss function
Pic: http://dsdeepdive.blogspot.com/2016/03/optimizations-of-gradient-descent.html
Gradient

104. Flux.jl - 套件直接相容語言
 TORCH.TENSOR
 32-bit floating point
 torch.FloatTensor
 torch.cuda.FloatTensor
 64-bit floating point
 torch.DoubleTensor
 torch.cuda.DoubleTensor
104
 Flux.jl
 Array{T, N}
 cpu() or gpu()
 直接用 for-loop, while-loop

105. 105
https://julialang.org/blog/2019/04/fluxdiffeq-zh_tw

106. Knet.jl
 100%純Julia
 支援自動微分
 支援低階 API
 套件直接相容語言
106

107. Turing.jl
 Universal probabilistic programming with an intuitive
modelling interface
 Hamiltonian Monte Carlo (HMC) sampling
 Gibbs sampling that combines particle MCMC, HMC and
many other MCMC algorithms
107

108. Learn.jl
 General abstractions and algorithms for modeling and
optimization
 Implementations of common models
 Tools for working with datasets
108

109. Others
 TensorFlow.jl
 MXNet.jl
 Mocha.jl
 Klara.jl: MCMC inference in Julia
 Mamba.jl: Markov chain Monte Carlo (MCMC) for Bayesian
analysis in julia
109

110. Science in Julia
110

111. Differential equation
 JuliaDiff
 ForwardDiff.jl: 支援前向自動微分
 ReverseDiff.jl: 支援後向自動微分
 TaylorSeries.jl
 JuliaDiffEq
 DifferentialEquations.jl
 Discrete Equations (function maps, discrete stochastic (Gillespie/Markov) simulations)
 常微分方程 Ordinary Differential Equations (ODEs)
 隨機微分方程 Stochastic Differential Equations (SDEs)
 代數微分方程 Algebraic Differential Equations (DAEs)
 延遲微分方程 Delay Differential Equations (DDEs)
111

112. DifferentialEquations.jl
 地表最強大的微分方程套件！
 比較 MATLAB, R, Julia, Python, C, Mathematica, Maple 及
Fortran 的微分方程套件
112
http://www.stochasticlifestyle.com/co
mparison-differential-equation-solver-
suites-matlab-r-julia-python-c-fortran/

113. Optimization
 JuliaOpt
 JuMP.jl
 Convex.jl
113
Objective types
• Linear
• Convex Quadratic
• Nonlinear (convex and
nonconvex)
Constraint types
• Linear
• Convex Quadratic
• Second-order Conic
• Semidefinite
• Nonlinear (convex and
nonconvex)
Variable types
• Continuous
• Integer-valued
• Semicontinuous
• Semi-integer

114. Graph / Network
 JuliaGraphs
 LightGraphs.jl
 GraphPlot.jl
114

115. Glue language of Julia
115

116. Glue
 JuliaPy
 JuliaInterop
116

117. Web stack in Julia
117

118. Genie – full-stack MVC framework
118

119. Escher
119

120. Web
 JuliaWeb
 Http.jl
 HTTP parser
 RESTful API
 WebSockets.jl
 SearchLight.jl: ORM for Genie
120

121. HPC from Intel Labs
Announced in JuliaCon 2016
121

122. 122
https://www.slideshare.net/EhsanTotoni/hpat-presentation-at-juliacon-2016

123. 123
https://www.slideshare.net/EhsanTotoni/hpat-presentation-at-juliacon-2016

124. 124
https://www.slideshare.net/EhsanTotoni/hpat-presentation-at-juliacon-2016

125. HPC from Intel Labs
 Video
 https://www.youtube.com/watch?v=Qa7nfaDacII
 Slide
 https://www.slideshare.net/EhsanTotoni/hpat-presentation-at-juliacon-2016
 Github
 2015: IntelLabs/ParallelAccelerator.jl
 2016: IntelLabs/HPAT.jl
 High Performance Analytics Toolkit (HPAT) is a Julia-based framework for big data
analytics on clusters.
 2018: IntelLabs/Latte.jl
 A high-performance DSL for deep neural networks in Julia
125

126. 126

127. 127

128. The downside
 Lack of users and developers
 Lack of internet resources

129.  Julia Taiwan 社群: https://www.facebook.com/groups/JuliaTaiwan/
 新知發布平台: https://www.facebook.com/juliannewstw/
 Slack [julialang] invitation: https://slackinvite.malmaud.com/
 論壇: https://discourse.julialang.org/
129

130. https://www.books.com.tw/products/0010808607

131. Backup
131

132. File
 JuliaIO
 FileIO.jl
 JSON.jl
 LightXML.jl
 HDF5.jl
 GZip.jl
132

133. Programming
 JuliaCollections
 DataStructures.jl
 SortingAlgorithms.jl
 FunctionalCollections.jl
 Combinatorics.jl
133

134. Type Hierarchy

135. Type System
 動態的，但擁有一些靜態型別系統的優點
 函式參數不加上型別，參數會被設成Any，加上型別可以增
加效能跟系統的穩健性
 參數化型別可以實現generic type
 型別系統是以階層式（hierarchical

136. Type System
Tuple
Union

137. OOP in Julia