20190424 Shared in PyHUG

1. Metaprogramming in
Julia
2019.04.24 Julia Taiwan發起人 杜岳華

2. Outline
• The language
• AST
• Expr
• Macro
• Generated function
• Staged programming

3. 想必大家應該都沒聽過metaprogramming

4. 甚麼是metaprogramming呢

5. 什麼是meta?
• meta-
• pref.字首
• 1. 表示變化，變換
• 2. 表示繼，在……之後
• 3. 表示超越
• 4. 表示在……之間，介於
• 翻譯
• 後設、元、統合、總體

6. Metadata
• data的data

7. Meta-algorithm
• Example: random forest

8. Metaphysics
• from Ancient Greece
• Physics – “nature” in Greek
• Exploring the fundamental nature of reality
• Ultimately, what is there?
• What is it like?

9. Meta-analysis
• 後設分析，或統合/整合分析
• 結合以前研究的結果來達到與研究主題有關的整合性結論

10. Metacognition
• 描述
• 認知的認知
• 知識的知識
• 就是對自己的認知過程（包括：記憶、感知、計算、聯想等各項）
的思考

11. metaprogramming
• Program the program
• Code as data - LISP

12. How to program the program?

13. From program to execution
• Pre-processor
• processes its input data to produce output that
is used as input to another program
• Compiler
• transforms source code written in a
programming language into another computer
language
• Assembler
• Translates instructions into machine code
• From Wikipedia
pre-processor
compiler
assembler
linker
memory
loader

14. Runtime
Compile time
From program to execution
• Linker
• takes one or more object files generated
by a compiler and combines them into a
single executable file, library file, or
another object file
• Loader
• part of an operating system that is
responsible for loading programs and
libraries
• From Wikipedia
pre-processor
compiler
assembler
linker
memory
loader

15. Compile time
From program to execution
pre-processor
compiler
assembler
linker
Macro processing, augmentation,
file inclusion, language extension
Generate target assembly code
Generate relocatable machine code
(object file)
Links object files and generate
executable machine code (binary file)

16. Runtime
From program to execution
memory
loader
Loader (part of OS) loads
programs and libraries
Execution

17. How compiler works?
Compile time
pre-processer
compiler
assembler
linker
Lexical analysis
Syntax analysis
Semantic analysis
Intermediate code
generation
Optimization
Code generation

18. 各種metaprogramming
• Macro in C, Julia
• Template in C++
• Reflection in Java……
• Introspection
• Generated function only in Julia

19. Compiler
Compiler in brief
Lexical analysis
Syntax analysis
Semantic analysis
scannerIdentify tokens
parser
Identify the grammar and
establish the parsing tree
checkerType checking, declaration

20. Compiler in brief
Compiler
Intermediate code
generation
Optimization
Code generation
Generate machine independent code for
common optimization
Optimization! Some magics……
Generate assembly code
Interpreter

21. Julia’s compiler
Source code
Tokens
AST
Julia IR
Typed Julia IR
LLVM IR
LLVM IR
Instructions
lexing
parsing
lowering
type inference
LLVM codegen
optimize
native codegen
LLVM IRLLVM IRMacro
expansion
Generated
function
JIT

22. While execution…
sum(12, 3.0)
Function table
-----------
sum
add
reduce
length
sum
-----------
(Number, Number)
(Integer, Integer)
(Float64, Float64)
AST
(Int64, Float64)
sum
Multiple dispatch
Parsing
Function call
Compile
Execute
Cached?

23. 用julia建構julia

24. Symbol
• Similar to token in compiler
• Symbol is the way for compiler to identify it
julia> :foo
:foo
julia> :bar
:bar

25. Symbol
julia> :foo == Symbol("foo")
true
julia> Symbol("func",10)
:func10
julia>
Symbol(:var,'_',"sym")
:var_sym

26. Expr
• Expr包含了3個部份：
• head：以一個Symbol來表示這個Expr的種類，而Symbol你可
以把他當成電腦讀的懂的字串
• args：代表著expression的參數，你可以想成是這些元素組成
這個expression的

27. Expr
julia> ex1 = Meta.parse(“1 + 1”)
:(1 + 1)
julia> typeof(ex1)
Expr

28. Expr
julia> ex1.head
:call
julia> ex1.args
3-element Array{Any,1}:
:+
1
1

29. Abstract Syntax Tree
Julia的程式碼可以內在表示為資料結構
而資料結構可以被語言本身所接受

30. Abstract Syntax Tree
julia> dump(ex1)
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Int64 1

31. Abstract Syntax Tree
1
+
1

32. AST julia> dump(ex)
Expr
head: Symbol call
args: Array{Any}((4,))
1: Symbol +
2: Symbol a
3: Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol *
2: Symbol b
3: Symbol c
4: Int64 1
julia> ex = :(a + b * c + 1)
:(a + b * c + 1)

33. AST
julia> dump(ex)
Expr
head: Symbol call
args: Array{Any}((4,))
1: Symbol +
2: Symbol a
3: Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol *
2: Symbol b
3: Symbol c
4: Int64 1
a
+
1
b
*
c

34. eval
• 執行Expr (AST)
julia> eval(:(1 + 2))
3

35. eval
julia> a = 1
1
julia> ex = :a
:a
julia> eval(ex)
1

36. Interpolation
julia> a = 1;
julia> ex = :($a + b)
:(1 + b)

37. Macro
• 他像一個函式一樣可以接受參數，他會回傳Expr並執行
julia> macro sayhello()
return :( println("Hello, world!") )
end
@sayhello
julia> @sayhello
Hello, world!

38. Macro
julia> macro sayhello(name)
return :( println("Hello, ", $name) )
end
@sayhello
julia> @sayhello("Bob")
Hello, Bob
julia> @sayhello "Bob"
Hello, Bob

39. Macro
parse → expressions → macro → new expr. → compile

40. Code generation
• 如此一來就可以來定義語言了
for op = (:+, :*, :&, :|, :$)
eval(quote
($op)(a,b,c) = ($op)(($op)(a,b),c)
end)
end

41. Implement decorator
• Python decorator
julia> macro decorator(dec, func)
name = func.args[1].args[1]
hiddenname = gensym()
func.args[1].args[1] = hiddenname
quote
$func
const $(esc(name)) = $dec($hiddenname)
end
end
http://julia-programming-language.2336112.n4.nabble.com/Macro-as-decorators-td40521.html

42. Implement decorator
julia> foo(f) = x -> 2*f(x+10);
julia> @decorator foo function bar(x)
return x+1
end
julia> bar(0)
22

43. Generated function
• Generate specialized function depending on the types of their
arguments
• Macro?
• Act on parsing time, and not accept type of arguments
• Once generated function knows type of arguments, it expand
and remain un-compiled

44. Generated function
julia> @generated function foo(x)
println(x)
return :(x*x)
end
foo (generic function with 1 method)

45. Generated function
julia> x = foo(2);
Int64
julia> x
4

46. Generated function
julia> y = foo("bar");
String
julia> y
"barbar"

47. Generated function
• How to code a generated function?
1. Add `@generated`
2. In the function body, only type is accessible, not value
3. Return quoted expression, instead of calculation result or actions

48. Generated function
• Purpose: To specialize for performance
julia> @generated function bar(x)
if x <: Integer
return :(x^2)
else
return :(x)
end
end
bar (generic function with 1 method)
julia> bar(4)
16
julia> bar("baz")
"baz"

49. How Julia’s compiler work?
Source code
Tokens
AST
Julia IR
Typed Julia IR
LLVM IR
LLVM IR
Instructions
lexing
parsing
lowering
type inference
LLVM codegen
optimize
native codegen
LLVM IRLLVM IR
Expr
Symbol
@code_lowered
@code_typed
@code_llvm
@code_native
@code_warntype

50. JIT compiler in Julia
AST
Julia IR
Typed Julia IR
lowering
type inference
LLVM IR
LLVM codegen
Function call
Multiple dispatch
Execution
specialize
specialize
optimize
codegen
https://www.youtube.com/watch?v=XWIZ_dCO6X8

51. 為什麼Julia可以這麼快?
• 不提供龐雜的語言 API，增加最佳化可能性
• 型別系統與多重分派的設計提供多樣而可特化的語言
• 在定義語言的時候大量使用macro跟generated function
• 將多數動態語言在runtime執行的事情搬到compile time
• 盡力提供fully-typed LLVM IR，交由LLVM做最佳化

52. Reflection
• module_name(m::Module) → Symbol
• nfields(x::DataType) → Int
• fieldnames(x::DataType)
• Base.function_name(f::Function) → Symbol

53. Thank you for attention