Talk given at Scala Symposium 2016, Amsterdam

1. Implementing Higher-
Kinded Types in Dotty
Martin Odersky
Scala Symposium, October 2016, Amsterdam
joint work with
Guillaume Martres, Dmitry Petrashko

2. Dotty
• The new Scala compiler developed at LAMP
• Based on DOT as a theoretical foundation
• Long-standing question: How to implement
higher-kinded types on these foundations?
• The paper and talk investigate 4 (!) different
solutions to do so.

3. Timeline
Simple
Encoding
Projection
Encoding
Refinement
Encoding
Direct
Repr.

4. Foundations: DOT
The DOT calculus is intended to be a minimal
foundation of Scala.
Its type structure is a blueprint for the types used
internally in the compiler.

5. DOT Terms
• Translated to Scala notation, the language
covered by DOT is:
Value v = (x: T) => t Function
new { x: T => d } Object
Definition d = def a = t Method definition
type A = T Type
Term t = v Value
x Variable
t1(t2) Application
t.a Selection
{ val x = t1; t2 } Local definition.

6. DOT Types
The Types covered by DOT are:
Type T = Any Top type
Nothing Bottom type
x.A Selection
(x: T1) => T2 Function
{ def a: T } Method declaration
{ type T >: T1 <: T2 } Type declaration
T1 & T2 Intersection
{ x => T } Recursion

7. DOT Types
The Types covered by DOT are:
Type T = Any Top type
Nothing Bottom type
x.A Selection
(x: T1) => T2 Function
{ def a: T } Method declaration
{ type T >: T1 <: T2 } Type declaration
T1 & T2 Intersection
{ x => T } Recursion
Should Scala have these?

8. DOT Types
The Types covered by DOT are:
Type T = Any Top type
Nothing Bottom type
x.A Selection
(x: T1) => T2 Function
{ def a: T } Method declaration
{ type T >: T1 <: T2 } Type declaration
T1 & T2 Intersection
{ x => T } RecursionScala has only refinements
T { d1 … dn}
with this as self reference.

9. Encoding Type Parameters

10. Lists, Encoded

11. A Variant of Variance

12. Variance, Encoded
=

13. Why Encode Parameterization?
• One fundamental concept (type member
selection) instead of two.
• Clarifies interaction between parameterization
and membership.
E.g.
List[_ <: Number] = List { type T <: Number }

14. The Simple Encoding
Simple
Encoding

15. Higher-Kinded Types
Problem: How to encode
type C[X] = List[X]
type C[+X <: Number] <: List[X]
?
Idea: Generalize definition of type parameters:
• of a class: designated member types
• of an alias type: type params of its alias
• of an abstract type: type params of upper bound

16. The Simple Encoding
Problem: How to encode
type C[X] <: List[X]
type D[X <: Number] = Cons[X]
?
This leads to:
type C <: List
type D = List { type T <: Number }

17. The Simple Encoding
Problem: How to encode
type C[X] <: List[X]
type D[X <: Number] = Cons[X]
?
This leads to:
type C <: List List[_]
type D = List { type T <: Number } List[_ <: Number]
Note that these encodings can also be seen as
wildcard (existential) types!

18. Limitations of the Simple Encoding
Unfortunately, there are things that can’t be
encoded that way:
type Rep[T] = T
type LL[T] = List[List[T]]
type RMap[V, K] = Map[K, V]
What To Do?

19. The Projection Encoding
Projection
Encoding

20. The Projection Encoding
Idea (originally due to Adriaan Moors):
Simulate type application using type selection.
E.g.
type C[X] <: List[X]
becomes
type C <: Lambda$P {
type $hk0
type $Apply <: List { type T = $hk0 }
}

21. The Projection Encoding
Idea (originally due to Adriaan Moors):
Simulate type application using type selection.
Then
C[String]
becomes
C { type $hk0 = String } # $Apply

22. The Projection Encoding
Idea (originally due to Adriaan Moors):
Simulate type application using type selection.
If we instantiate C to List, say, we get:
type C = Lambda$P {
type $hk0
type $Apply = List { type T = $hk0 }
}

23. The Projection Encoding
Hence,
C[String]
= C { type $hk0 = String } # $Apply
= Lambda$P {
type $hk0; type $Apply = List { type T = $hk0 }
} { type $hk0 = String } # $Apply
=:=
List { type $hk0 = String }
= List[String]
Where “=:=” is interpreted as “being mutual subtype of”.

24. Problems with the Projection Enc.
• There were several, but the killer was:
It relies on type projection T#U, which is unsound in
general and will therefore be phased out.
• See the paper for a discussion in detail.

25. The Refinement Encoding
Refinement
Encoding

26. The Refinement Encoding
Idea:
Encode
type C[X] <: List[X]
C[String]
as
type C <: { z => List { type T = $hk0 }
type $hk0 }
C { type $hk0 = String }
This also had several problems which are described in
the paper.

27. In the end…
… we represented hk types directly, using special
type forms for
• Type lambda: [X] -> T TypeLambda
• Higher-kinded
type application: C[T] HKApply
• A beta reduction rule:
([X] -> T)[U] à [X := U]T
• Rules for subtyping, type inference …

28. Implementation Data
Simple
Encoding
Projection
Encoding
Refinement
Encoding
Direct
Repr.
Total type-
checker + core:
28’000 cloc

29. Evaluation
• The simple encoding is a modest extension for a
language with wildcards or variance that wants to
support basic concepts associated with hk types.
• Cheap way to get a lot of the advantages of hk.
• Difficulty: Explain what’s legal and what is not.
• Other encodings were less successful than they
looked at first.
• A direct representation is needed for robust
support of full higher kinded types.

30. Still Missing
• A theory that explains what the compiler did
in terms of DOT.
• Hopefully, we can find something that’s small
and fits well.

31. A Take Away
• Compiler hacking is a possible way to validate
ideas
• But it may not be the most efficient one.
• If we would have a better theory how higher-
kinded types fit into DOT, we could have avoided
some costly experiments.