Deep learning has renewed interest in computational creativity. Can machines be creative? In which sense? And why this would be useful? We argue current creative AI systems are stuck: they explore combination, analogy or random, but the value of the objects are provided by the system designer. The only way to creative AI is to develop agents building their own value. We also argue: the generative potential of deep learning is understudied. Current focus is on likelihood - whereas creativity is unlikely. We present an implementation of these ideas on the MNIST handwritten digits dataset - to create symbols that could have been digits (e.g. in an imaginary culture) but that are not.

1. Digits that are not
Generating new types through deep neural nets
Mehdi Cherti, Balázs Kégl
LAL/LRI, CNRS
Université Paris-Saclay
Akın Kazakçı
MINES ParisTech,
PSL Research University

2. The Short Story
Why are we doing this?

3. How to determine the value of novelty?
The value of novelty is the blindspot of creativity research
(Kazakçı, 2014).

4. Computational creativity
Deep learning to the rescue?
• has enabled great progress in machine
learning
• Also, several promising work renewed
interest in computational creativity.
• Google created Magenta (nobody needs us
anymore)
Gatys et al. 2015
Deep learning
Yet
• main emphasis in DL research remains
learning to predict
• models are based on likelihood whereas
creativity is unlikely
• the initial breakthrough came from a
generative model
But the generative potential of
deep nets is largely unexplored

5. Fitness function barrier
• For most computational creativity systems, the value
function is fixed and predetermined
• This is a paradox. - And an obstacle for progress.
• Evolutionary approaches to computational creativity
bears these inherent limitations:
• Explicit fitness functions reflects system designer’s
preference for novelty - not the machine’s.
• We call this the fitness function barrier.

6. A program to go beyond the barrier
Ultimate objective:Try & get rid of hard-coded
value functions; let the system develop its own
A genuinely creative system needs to be able to develop its
own notion of value
Current work: Build a system
1. that can study & learn a referential set of objects (RS)
2. that can generate new objects that keep essential
features of RS but generate unseen types
3. that provides an experimental bench for developing
& testing various ways an agent can develop a value
function

7. Auto-associative neural nets
a.k.a auto-encoders
Learning to
disassemble
Learning to
build
- Auto-encoders have existed
for long time (Kramer 1991)
- Deep variants are more recent
(Hinton, Salakhutdinov, 2006;
Bengio 2009)
- A deep auto-encoder learns
successive transformations
that decompose and then
recompose a set of training
objects
- The depth allows learning a
hierarchy of transformations

8. The experimental setup
- Training data : MNIST, 70000 images of
handwritten digits of size 28x28
- We use a sparse convolutional auto-
encoder (3c1d) trained to:
- Encode : take an image and
transform it to a sparse code
- Decode : take the sparse code and
reconstruct the image
- Training objective is to minimize the
reconstruction error

9. Generating new symbols
- We use an iterative method to build symbols the has never seen:
- Start with a random image x0 = r,
- and force the network to construct (i.e. interpret)
- xk = f(xk-1), until convergence
- Our method is inspired by Bengio et al. (2013)
- By contrast to them, we do not constrain the net to generate only
known types (we do not consider unknown symbols as spurious)

10. Visualising the structure of generated images
Coloured clusters are original
MNIST digits (classes from 0
to 9)
The gray dots are newly
generated objects
New objects form new clusters
Using a clustering algorithm, we
recover coherent sets of new
symbols
A distance preserving projection of
digits to a two-dimensional space
(van der Maaten and Hinton 2008)

11. Creativity by fixations
LearnForceFixate

12. Summary
A genuinely creative system needs to be able to develop its
own notion of value.
Our system is a first step:
• It can effectively create new types of objects preserving abstract
and semantic properties of a domain.
• It provides an experimental setup that enables testing various
hypothesis.
• It provides a bridge between current research on machine
learning and creativity research.

13. Thank you
Mehdi Cherti, Balázs Kégl
{mehdi.cherti, balazskegl}@gmail.com
Akın Kazakçı
akin.kazakci@mines-paristech.fr