SlideShare une entreprise Scribd logo
1  sur  295
HOW RESEARCH INTO
EPISTEMIC NETWORK
     ANALYSIS
   MIGHT INFORM
DISCOURSE-CENTRIC
LEARNING ANALYTICS
      David Williamson Shaffer
ENA DCLA



 David Williamson Shaffer
A CANDLE



 David Williamson Shaffer
Students   Teacher
Students   Teacher




           Lesson
            Plan
Students   Teacher




           Lesson
 Tests
            Plan
Students   Teacher




           Lesson
 Tests
            Plan
Students   Teacher




           Lesson
 Tests
            Plan
Students                          Teacher



             How do learning
               analytics and
            discourse analysis
           change this picture?


                                  Lesson
 Tests
                                   Plan
I would set up a experimental ultrafiltration
membrane and test the surfactants.
First you have to set up a controlled
experiment with no carbon nanotubes.
Record how membrane fouling acts upon the
membrane with no carbon nanotubes. Then,
slowly increase the carbon nanotube
percentage with each experiment and
compare them to the controlled experiment
along with all the experiments before it. You
should then be able to figure out what
percentage is most effective.
First you have to set up a controlled
experiment with no carbon nanotubes.
Record how membrane fouling acts upon the
membrane with no carbon nanotubes. Then,
slowly increase the carbon nanotube
percentage with each experiment and
compare them to the controlled experiment
along with all the experiments before it. You
should then be able to figure out what
percentage is most effective.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
 0%
       Year 1   Year 2   Year 3
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
 0%
       Year 1   Year 2   Year 3
d = + 0.28
d = + 0.28
d = + 0.91
d = + 0.91
Students   Teacher




           Lesson
 Tests
            Plan
Students   Mentor




           Lesson
 Tests
            Plan
Student


      Student   Mentor

Student




                Lesson
  Tests
                 Plan
Student

                Chat
      Student          Mentor
                Hub

Student




                       Lesson
  Tests
                        Plan
Student

                Chat
      Student          Mentor
                Hub

Student




  Tests                Playbook
Student

                Chat
      Student          Mentor
                Hub

Student




  Tests                Playbook
Student

                  Chat
      Student               Mentor
                  Hub

Student




                Utterance
                  Table

  Tests                     Playbook
Data Mining
GIGO
Garbage In, Garbage Out
We value what we can
      measure
             —Don Norman
We can only
meaningfully measure
  based on theory
Data Mining
Data Mining
Data Geology
An example to think with
Jean Lave
Community of Practice
culture
culture
culture
culture
culture
culture
What are the necessary and sufficient
conditions of knowledge?
What makes justified beliefs justified?
What are the necessary and sufficient
conditions of knowledge?
What makes justified beliefs justified?
How do people
    make decisions
           and justify actions?
Epistemic Frame
Complex thinking happens in
   cultures of practice
1. Apply virtual communication      4. Improve the visibility of both
tools and facilitation techniques   formal and informal skills.
to more effectively connect
people from around the world.       5. Use dashboards linked to
                                    collaborative tools to capture
2. Use collaborative spaces to      key realtime information about
gather knowledge, express           people, activities and outcomes.
ideas and concerns and share
passions.                           6. Provide more frequent
                                    guidance and link performance
3. Spend more time on setting       to recognition.
organizational context and
communicating where the
organization needs to go.
The environmental factors
      within MMORPGs can be


75%   applied to enhance
      leadership effectiveness
      for the globally integrated
      enterprise
Game v Corporate Skills

                   100



                    80
Corporate Skills




                    60



                    40



                    20



                     0
                         0   20         40                 60   80   100
                                             Game Skills
The skills and techniques



61%
      of the gaming leader
      cannot be transferred
      directly to the leadership
      of the virtual corporation
Complex thinking happens in
           cultures of practice

Cultures of practice are characterized by their
            patterns of Discourse
R
eflection-in-action
T
thinking that reshapes
  what we are doing
 while we are doing it
thinking that reshapes
  what we are doing
 while we are doing it




                         K
                         ways of
                         knowing
D
                         ways of
                          doing



thinking that reshapes
  what we are doing
 while we are doing it


                         ways of
                         knowing
ways of
             doing




P
practicum



            ways of
            knowing
A   action




practicum




            R
            reflection-
            on-action
action



practicum


            reflection-
            on-action
action


practicum

            reflection-
            on-action
practicum
            R
            eflection-in-action
practicum
Skills   Knowledge   Identity   Values   Epistemology
Skills   Knowledge   Identity   Values   Epistemology
Complex thinking happens in
            cultures of practice

 Cultures of practice are characterized by their
             patterns of Discourse

Patterns of Discourse can be described in terms
             of an epistemic frame
Epistemic Network Analysis
          (ENA)
©2005, 2008 David Williamson Shaffer
©2005, 2008 David Williamson Shaffer
©2005, 2008 David Williamson Shaffer
B



A


        ©2005, 2008 David Williamson Shaffer
B



A


        ©2005, 2008 David Williamson Shaffer
B

        D
    C
A


            ©2005, 2008 David Williamson Shaffer
B       F
    E
            D
        C
A


                    ©2005, 2008 David Williamson Shaffer
E
        B
    A
                C          F

            D

                    ©2005, 2008 David Williamson Shaffer
B
C
    A           D   F

            E



                        ©2005, 2008 David Williamson Shaffer
B
F
    A       D C

        E



                  ©2005, 2008 David Williamson Shaffer
©2005, 2008 David Williamson Shaffer
B
A


        ©2005, 2008 David Williamson Shaffer
C




        B
    A


            ©2005, 2008 David Williamson Shaffer
C
            F


        B
    A


                ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A


                ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A

                E
                    ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A

                E
                    ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A

                E
                    ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A

                E
                    ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A

                E
                    ©2005, 2008 David Williamson Shaffer
C
            F


D
        B
    A

                E
                    ©2005, 2008 David Williamson Shaffer
Social Network Analysis
Skills   Knowledge   Identity   Values   Epistemology
Semantic Coding
Semantic Coding
AI Complete Problem
Structural Functional Linguistic Codes
Skills   Knowledge   Identity   Values   Epistemology
Structural Functional Linguistic Codes
           Skills   Knowledge   Identity   Values   Epistemology
Domain codes
Structural Functional Linguistic Codes
Domain codes
Structural Functional Linguistic Codes
           Skills   Knowledge   Identity   Values   Epistemology
Domain codes
Sentences
Sentences   Verbs
Sentences   Verbs   Nouns (Adj)
Sentences   Verbs   Nouns (Adj)   People
Structural Functional Linguistic Codes
Structural Functional Linguistic Codes
Domain codes
Structural Functional Linguistic Codes
Domain codes




                        Conjunctive Coding
Structural Functional Linguistic Codes
Domain codes
Structural Functional Linguistic Codes


                                           Value of Surfactants
Domain codes
Structural Functional Linguistic Codes


                                           Value of Surfactants
Domain codes
Structural Functional Linguistic Codes
Domain codes




                                               Engineering Identity
Structural Functional Linguistic Codes
Domain codes




                                               Engineering Identity
Human Agreement with AutoCoder



                                                       E and AC   N1 and N2 E1 and E2
                                             E/Data    0.521463   0.321308   0.697609
                                           E/Design     0.64044   0.479389    0.54518
                                           E/Clients   0.746846   0.796759   0.746846
                                     E/Consultants     0.764168   0.566491   0.853813
                                           V/Clients   0.538768   0.663687   0.663311
                                    V/Consultants      0.493692   0.663311   0.663311
                                             S/Data     0.76369   0.615417   0.940922
                                           S/Design    0.801079   0.466942   0.728728
                                 S/Professionalism     0.645465   0.456452    0.72107
                                   S/Collaboration     0.694157   0.666465   0.888954
                                         I/Engineer    0.479239   0.824869   0.560584
                                            I/Intern   0.921389   0.816862   0.853813
                                       K/Nanotech             1   0.796759          1
                                     K/Surfactants      0.95827   0.965993   0.919605
                                       K/Attributes    0.981197   0.900837   0.990646
                                             K/Data     0.63988   0.784384   0.869369
                                           K/Design    0.728741   0.771417   0.774949
                                           K/Clients          1   0.945658          1
                                       K/Materials     0.637786   0.916048   0.984033
                                       K/Processes     0.364711   0.811285    0.95827
Human Agreement with AutoCoder



                                                       E and AC   N1 and N2 E1 and E2
                                             E/Data    0.521463   0.321308   0.697609
                                           E/Design     0.64044   0.479389    0.54518
                                           E/Clients   0.746846   0.796759   0.746846
                                     E/Consultants     0.764168   0.566491   0.853813
                                           V/Clients   0.538768   0.663687   0.663311
                                    V/Consultants      0.493692   0.663311   0.663311
                                             S/Data     0.76369   0.615417   0.940922
                                           S/Design    0.801079   0.466942   0.728728
                                 S/Professionalism     0.645465   0.456452    0.72107
                                   S/Collaboration     0.694157   0.666465   0.888954
                                         I/Engineer    0.479239   0.824869   0.560584
                                            I/Intern   0.921389   0.816862   0.853813
                                       K/Nanotech             1   0.796759          1
                                     K/Surfactants      0.95827   0.965993   0.919605
                                       K/Attributes    0.981197   0.900837   0.990646
                                             K/Data     0.63988   0.784384   0.869369
                                           K/Design    0.728741   0.771417   0.774949
                                           K/Clients          1   0.945658          1
                                       K/Materials     0.637786   0.916048   0.984033
                                       K/Processes     0.364711   0.811285    0.95827
Human Agreement with AutoCoder



                                                       E and AC   N1 and N2 E1 and E2
                                             E/Data    0.521463   0.321308   0.697609
                                           E/Design     0.64044   0.479389    0.54518
                                           E/Clients   0.746846   0.796759   0.746846
                                     E/Consultants     0.764168   0.566491   0.853813
                                           V/Clients   0.538768   0.663687   0.663311
                                    V/Consultants      0.493692   0.663311   0.663311
                                             S/Data     0.76369   0.615417   0.940922
                                           S/Design    0.801079   0.466942   0.728728
                                 S/Professionalism     0.645465   0.456452    0.72107
                                   S/Collaboration     0.694157   0.666465   0.888954
                                         I/Engineer    0.479239   0.824869   0.560584
                                            I/Intern   0.921389   0.816862   0.853813
                                       K/Nanotech             1   0.796759          1
                                     K/Surfactants      0.95827   0.965993   0.919605
                                       K/Attributes    0.981197   0.900837   0.990646
                                             K/Data     0.63988   0.784384   0.869369
                                           K/Design    0.728741   0.771417   0.774949
                                           K/Clients          1   0.945658          1
                                       K/Materials     0.637786   0.916048   0.984033
                                       K/Processes     0.364711   0.811285    0.95827
Human Agreement with AutoCoder



                                                       E and AC   N1 and N2 E1 and E2
                                             E/Data    0.521463   0.321308   0.697609
                                           E/Design     0.64044   0.479389    0.54518
                                           E/Clients   0.746846   0.796759   0.746846
                                     E/Consultants     0.764168   0.566491   0.853813
                                           V/Clients   0.538768   0.663687   0.663311
                                    V/Consultants      0.493692   0.663311   0.663311
                                             S/Data     0.76369   0.615417   0.940922
                                           S/Design    0.801079   0.466942   0.728728
                                 S/Professionalism     0.645465   0.456452    0.72107
                                   S/Collaboration     0.694157   0.666465   0.888954
                                         I/Engineer    0.479239   0.824869   0.560584
                                            I/Intern   0.921389   0.816862   0.853813
                                       K/Nanotech             1   0.796759          1
                                     K/Surfactants      0.95827   0.965993   0.919605
                                       K/Attributes    0.981197   0.900837   0.990646
                                             K/Data     0.63988   0.784384   0.869369
                                           K/Design    0.728741   0.771417   0.774949
                                           K/Clients          1   0.945658          1
                                       K/Materials     0.637786   0.916048   0.984033
                                       K/Processes     0.364711   0.811285    0.95827
Human Agreement with AutoCoder



                                                       E and AC   N1 and N2 E1 and E2
                                             E/Data    0.521463   0.321308   0.697609
                                           E/Design     0.64044   0.479389    0.54518
                                           E/Clients   0.746846   0.796759   0.746846
                                     E/Consultants     0.764168   0.566491   0.853813
                                           V/Clients   0.538768   0.663687   0.663311
                                    V/Consultants      0.493692   0.663311   0.663311
                                             S/Data     0.76369   0.615417   0.940922
                                           S/Design    0.801079   0.466942   0.728728
                                 S/Professionalism     0.645465   0.456452    0.72107
                                   S/Collaboration     0.694157   0.666465   0.888954
                                         I/Engineer    0.479239   0.824869   0.560584
                                            I/Intern   0.921389   0.816862   0.853813
                                       K/Nanotech             1   0.796759          1
                                     K/Surfactants      0.95827   0.965993   0.919605
                                       K/Attributes    0.981197   0.900837   0.990646
                                             K/Data     0.63988   0.784384   0.869369
                                           K/Design    0.728741   0.771417   0.774949
                                           K/Clients          1   0.945658          1
                                       K/Materials     0.637786   0.916048   0.984033
                                       K/Processes     0.364711   0.811285    0.95827
Skills     Knowledge   Identity     Values     Epistemology
    1   2    3   1   2   3    1    2    1   2    3    1    2   3
3   0   0    0   0   1   0    1    0    0   1    0    0    0   0
Skills     Knowledge   Identity     Values     Epistemology
    1   2    3   1   2   3    1    2    1   2    3    1    2   3
3   0   0    0   0   1   0    1    0    0   1    0    0    0   0
Skills         Knowledge               Identity         Values           Epistemology
                     1   2    3       1   2   3                1    2        1   2    3          1    2   3
   3                 0   0    0       0   1   0                1    0        0   1    0          0    0   0


                             Skills           Knowledge           Identity         Values             Epistemology
                         1    2       3   1      2        3       1      2     1     2      3     1        2         3
                     1
Skills




                     2
                     3
                     1
Identity Knowledge




                     2                                            1
                     3
                     1                           1
                     2
                     1
Values




                     2
                     3
Epistemology




                     1
                     2
                     3
Skills         Knowledge               Identity         Values           Epistemology
                     1   2    3       1   2   3                1    2        1   2    3          1    2   3
   3                 0   0    0       0   1   0                1    0        0   1    0          0    0   0


                             Skills           Knowledge           Identity         Values             Epistemology
                         1    2       3   1      2        3       1      2     1     2      3     1        2         3
                     1   0    0       0   0      0        0       0      0     0     0      0     0        0         0
Skills




                     2   0    0       0   0      0        0       0      0     0     0      0     0        0         0
                     3   0    0       0   0      0        0       0      0     0     0      0     0        0         0
                     1   0    0       0   0      0        0       0      0     0     0      0     0        0         0
Identity Knowledge




                     2   0    0       0   0      0        0       1      0     0     1      0     0        0         0
                     3   0    0       0   0      0        0       0      0     0     0      0     0        0         0
                     1   0    0       0   0      1        0       0      0     0     1      0     0        0         0
                     2   0    0       0   0      0        0       0      0     0     0      0     0        0         0
                     1   0    0       0   0      0        0       0      0     0     0      0     0        0         0
Values




                     2   0    0       0   0      1        0       1      0     0     0      0     0        0         0
                     3   0    0       0   0      0        0       0      0     0     0      0     0        0         0
Epistemology




                     1   0    0       0   0      0        0       0      0     0     0      0     0        0         0
                     2   0    0       0   0      0        0       0      0     0     0      0     0        0         0
                     3   0    0       0   0      0        0       0      0     0     0      0     0        0         0
Skills           Knowledge       Identity       Values           Epistemology
                         1    2       3   1      2        3   1      2   1     2      3   1        2         3
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Skills




                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      1        0   0      0   0     0      0   0        0         0
Identity Knowledge




                     2   0    0       0   1      0        0   1      0   0     1      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      1        0   0      0   0     1      0   0        0         0
                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Values




                     2   0    0       0   0      1        0   1      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Epistemology




                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Skills           Knowledge       Identity       Values           Epistemology
                         1    2       3   1      2        3   1      2   1     2      3   1        2         3
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         1
Skills




                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      1        0   0      1   0     0      0   0        0         0
Identity Knowledge




                     2   0    0       0   1      0        0   1      0   0     1      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      1        0   0      0   0     1      0   0        0         0
                     2   0    0       0   1      0        0   0      0   0     0      1   0        0         0
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Values




                     2   0    0       0   0      1        0   1      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      1   0     0      0   0        0         0
Epistemology




                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   1    0       0   0      0        0   0      0   0     0      0   0        0         0
Skills           Knowledge       Identity       Values           Epistemology
                         1    2       3   1      2        3   1      2   1     2      3   1        2         3
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         1
Skills




                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      1        0   0      1   0     0      0   0        0         0
Identity Knowledge




                     2   0    0       0   1      0        0   2      0   0     1      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      2        0   0      0   0     1      0   0        0         0
                     2   0    0       0   1      0        0   0      0   0     0      1   0        0         0
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Values




                     2   0    0       0   0      1        0   1      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      1   0     0      0   0        0         0
Epistemology




                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   1    0       0   0      0        0   0      0   0     0      0   0        0         0
Skills           Knowledge       Identity       Values           Epistemology
                         1    2       3   1      2        3   1      2   1     2      3   1        2         3
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         2
Skills




                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      1        0   0      1   0     0      0   0        0         0
Identity Knowledge




                     2   0    0       0   1      0        0   4      0   0     2      0   0        0         0
                     3   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     1   0    0       0   0      4        0   0      0   0     1      0   0        0         0
                     2   0    0       0   1      0        0   0      0   0     0      1   0        0         0
                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
Values




                     2   0    0       0   0      1        0   2      0   0     0      0   0        0         0
                     3   0    0       0   0      0        0   0      1   0     0      0   0        0         0
Epistemology




                     1   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     2   0    0       0   0      0        0   0      0   0     0      0   0        0         0
                     3   2    0       0   0      0        0   0      0   0     0      0   0        0         0
0                        0       0       0         0       0       0       0       0       0       0       0         0       2




                                             Skills                Knowledge           Identity                Values                Epistemology
                                     1         2       3       1       2       3       1       2       1         2       3       1        2         3
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         2
    Skills




                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       1       0       0       1       0         0       0       0        0         0
    Identity Knowledge




                             2       0         0       0       1       0       0       4       0       0         2       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       4       0       0       0       0         1       0       0        0         0
                             2       0         0       0       1       0       0       0       0       0         0       1       0        0         0
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
    Values




                             2       0         0       0       0       1       0       2       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       1       0         0       0       0        0         0
    Epistemology




                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       2         0       0       0       0       0       0       0       0         0       0       0        0         0
0                        0       0       0         0       0       0       0       0       0       0       0         0       2       0        0         0   0   0   0   0   0   0   0   0   0   0   0




                                             Skills                Knowledge           Identity                Values                Epistemology
                                     1         2       3       1       2       3       1       2       1         2       3       1        2         3
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         2
    Skills




                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       1       0       0       1       0         0       0       0        0         0
    Identity Knowledge




                             2       0         0       0       1       0       0       4       0       0         2       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       4       0       0       0       0         1       0       0        0         0
                             2       0         0       0       1       0       0       0       0       0         0       1       0        0         0
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
    Values




                             2       0         0       0       0       1       0       2       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       1       0         0       0       0        0         0
    Epistemology




                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       2         0       0       0       0       0       0       0       0         0       0       0        0         0
0                        0       0       0         0       0       0       0       0       0       0       0         0       2       0        0         0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0




                                             Skills                Knowledge           Identity                Values                Epistemology
                                     1         2       3       1       2       3       1       2       1         2       3       1        2         3
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         2
    Skills




                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       1       0       0       1       0         0       0       0        0         0
    Identity Knowledge




                             2       0         0       0       1       0       0       4       0       0         2       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       4       0       0       0       0         1       0       0        0         0
                             2       0         0       0       1       0       0       0       0       0         0       1       0        0         0
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
    Values




                             2       0         0       0       0       1       0       2       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       1       0         0       0       0        0         0
    Epistemology




                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       2         0       0       0       0       0       0       0       0         0       0       0        0         0
0                        0       0       0         0       0       0       0       0       0       0       0         0       2       0        0         0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0




                                             Skills                Knowledge           Identity                Values                Epistemology
                                     1         2       3       1       2       3       1       2       1         2       3       1        2         3
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         2
    Skills




                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       1       0       0       1       0         0       0       0        0         0
    Identity Knowledge




                             2       0         0       0       1       0       0       4       0       0         2       0       0        0         0
                             3       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             1       0         0       0       0       4       0       0       0       0         1       0       0        0         0
                             2       0         0       0       1       0       0       0       0       0         0       1       0        0         0
                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
    Values




                             2       0         0       0       0       1       0       2       0       0         0       0       0        0         0
                             3       0         0       0       0       0       0       0       1       0         0       0       0        0         0
    Epistemology




                             1       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             2       0         0       0       0       0       0       0       0       0         0       0       0        0         0
                             3       2         0       0       0       0       0       0       0       0         0       0       0        0         0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0   2   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0
                                                                           Skills   0   0      0
                                                                                            Knowledge   0   0 Identity
                                                                                                                     0    0    0
                                                                                                                              Values   0   0   Epistemology 0
                                                                                                                                                    0           0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    21    2
                                                                            0       3
                                                                                    0   1
                                                                                        0      2
                                                                                               0        3
                                                                                                        0   1
                                                                                                            0        02   1
                                                                                                                          0    02      3
                                                                                                                                       0   1
                                                                                                                                           0        02      3
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
                                                                  1    0    0       0   0      0        0   0         0   0     0      0   0         0      2
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0        0    0    0       0   0        0       0   0   0   0   0   0   0   0   0   0   0   0
                                            Skills




                                                                  2    0    0       0   0      0        0   0         0   0     0      0   0         0      0
0   0   0   0   0   0   0   0   0   0   0   0                    03   20    0
                                                                            0       0
                                                                                    0   0
                                                                                        0      0
                                                                                               0        0
                                                                                                        0   0
                                                                                                            0        00   0
                                                                                                                          0    00      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    01   20    0
                                                                            0       0
                                                                                    0   0
                                                                                        0      0
                                                                                               1        0
                                                                                                        0   0
                                                                                                            0        01   0
                                                                                                                          0    00      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
                                            Identity Knowledge




0   0   0   0   0   0   0   0   0   0   0   0                    02   20    0
                                                                            0       0
                                                                                    0   1
                                                                                        0      0
                                                                                               0        0
                                                                                                        0   4
                                                                                                            0        00   0
                                                                                                                          0    02      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
                                                                  3    0    0       0   0      0        0   0         0   0     0      0   0         0      0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0        0    0    0       0   0        0       0   0   0   0   0   0   0   0   0   0   0   0
                                                                  1    0    0       0   0      4        0   0         0   0     1      0   0         0      0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0        0    0    0       0   0        0       0   0   0   0   0   0   0   0   0   0   0   0
                                                                  2    0    0       0   1      0        0   0         0   0     0      1   0         0      0
0   0   0   0   0   0   0   0   0   0   0   0                    01   20    0
                                                                            0       0
                                                                                    0   0
                                                                                        0      0
                                                                                               0        0
                                                                                                        0   0
                                                                                                            0        00   0
                                                                                                                          0    00      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
                                            Values




0   0   0   0   0   0   0   0   0   0   0   0                    02   20    0
                                                                            0       0
                                                                                    0   0
                                                                                        0      1
                                                                                               0        0
                                                                                                        0   2
                                                                                                            0        00   0
                                                                                                                          0    00      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    03   20    0
                                                                            0       0
                                                                                    0   0
                                                                                        0      0
                                                                                               0        0
                                                                                                        0   0
                                                                                                            0        01   0
                                                                                                                          0    00      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
                                            Epistemology




                                                                  1    0    0       0   0      0        0   0         0   0     0      0   0         0      0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0        0    0    0       0   0        0       0   0   0   0   0   0   0   0   0   0   0   0
                                                                  2    0    0       0   0      0        0   0         0   0     0      0   0         0      0
0   0   0   0   0   0   0   0   0   0   0   0                    03   22    0
                                                                            0       0
                                                                                    0   0
                                                                                        0      0
                                                                                               0        0
                                                                                                        0   0
                                                                                                            0        00   0
                                                                                                                          0    00      0
                                                                                                                                       0   0
                                                                                                                                           0        00      0
                                                                                                                                                            0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0        0    0    0       0   0        0       0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0                    0    2     0       0   0      0        0   0       0     0    0       0   0       0       0    0   0   0   0   0   0   0   0   0   0   0
Complex thinking happens in
            cultures of practice

 Cultures of practice are characterized by their
             patterns of Discourse

Patterns of Discourse can be described in terms
             of an epistemic frame

 An epistemic frame can be quantified using a
            network model (ENA)
S1 x K2




S1 x V2

          S1 x K1
S1 x K2




S1 x V2
          S1

               S1 x K1
K2


S1 x K2




S1 x V2
          S1
V2
               S1 x K1
                         K1
K2


S1 x K2




S1 x V2
          S1
V2
               S1 x K1
                         K1
K2
                Equiload
S1 x K2         Projection




S1 x V2
          S1
V2
               S1 x K1
                         K1
Player 1: Good Morning
Player 2: Hello everyone, is this
          chat with our new
          group now?
Player 1: I think so
Player 3: Probably
Player 1: Good morning new
          team. I excited to work
          with with everyone.
Player 4: hey
Player 1: We are meeting and
          currently discussing
          our various materials.
Player 1: Good Morning
Player 2: Hello everyone, is this
          chat with our new
          group now?
Player 1: I think so
Player 3: Probably
Player 1: Good morning new
          team. I excited to work
          with with everyone.
Player 4: hey
Player 1: We are meeting and
          currently discussing
          our various materials.
Player 1:   We ranked the different
            materials based on the
            device specifications and
            our experimental data.
Player 2:   And we just need to
            decide on the 5th team
            experimental device.
Player 3:   We looked at… how the
            cost changed with each
            matieral while controling
            everything else
Player 4:   hey team, I am going to
            post the rankings of the
            membranes as a public
            document so everyone
            can see it.
Product
Product
Professionalism




   Product
Professionalism




   Product
Professionalism   Design




   Product
Professionalism   Design




   Product
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Professionalism    Design




   Product        Materials
Trajectories of Mastery
Professionalism    Design




   Product        Materials
d = + 0.28
Complex thinking happens in
            cultures of practice

 Cultures of practice are characterized by their
             patterns of Discourse

Patterns of Discourse can be described in terms
             of an epistemic frame

 An epistemic frame can be quantified using a
            network model (ENA)

       ENA can model complex thinking
Student

                  Chat
      Student               Mentor
                  Hub

Student




                Utterance
                  Table

  Tests                     Playbook
Student

                        Chat
      Student                     Mentor
                        Hub

Student




                      Utterance
                        Table
                ENA
  Tests                           Playbook
Student

                        Chat
      Student                     Mentor
                        Hub

Student




                      Utterance
                        Table
                ENA
                                  Playbook
Student
Models
Student

                        Chat
      Student                     Mentor
                        Hub

Student




                      Utterance
                        Table
                ENA
                                  Playbook
Student
Models
Student

                           Chat
      Student                        Mentor
                           Hub

Student




                Speech
                  Act
                         Utterance
                           Table
                 ENA
                                     Playbook
Student
Models
Student

                           Chat
      Student                        Mentor
                           Hub

Student




  Auto          Speech
 Mentor           Act
                         Utterance
                           Table
                 ENA
                                     Playbook
Student
Models
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Student
Models
Scope   Trigger
         Scope    Action
                  Scope
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Student
Models
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Student
Models
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Student
Models
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Student
Models
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Student
Models
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Summative
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest



  Auto          Speech
 Mentor           Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Summative
Student

                            Chat
      Student                         Mentor
                            Hub

Student
                Suggest




                Speech
Formative
                  Act
                          Utterance
                            Table
                 ENA
                                      Playbook
Summative
Student

                            Chat               Auto
      Student
                            Hub               Mentor

Student
                Suggest              Using ENA in games
                                          to create
                                   fummative assessments
                Speech
Formative
                  Act
                          Utterance
                            Table
                 ENA
                                             Playbook
Summative
Student

                            Chat               Auto
      Student
                            Hub               Mentor

Student
                Suggest             Using ENA in games
                                         to create
                                   sormative assessments
                Speech
Formative
                  Act
                          Utterance
                            Table
                 ENA
                                             Playbook
Summative
Auto
           Mentor


 Using ENA in games
      to create
sormative assessments
Art
Graesser               Auto
                      Mentor


            Using ENA in games
                 to create
           sormative assessments
Art
Graesser               Auto
                      Mentor

 Andre
 Rupp       Using ENA in games
                 to create
           sormative assessments
Art
Graesser                Auto
                       Mentor

 Andre
 Rupp        Using ENA in games
                  to create
            sormative assessments
  Kris
Scopinich
Art
Graesser                Auto
                       Mentor

 Andre
 Rupp        Using ENA in games
                  to create
            sormative assessments
  Kris
Scopinich



 Asligul
Goḉmen
Skills   Knowledge   Identity   Values   Epistemology
Skills   Knowledge   Identity   Values   Epistemology
Skills   Knowledge   Identity   Values   Epistemology
Based on the sample codes and comparison metrics provided, it appears the autoencoder is performing fairly well at identifying concepts and semantic relationships in the data, with agreement scores ranging from 0.52 to 0.85 compared to human coding. Some areas like clients and consultants show very strong agreement over 0.7. Names and design show some room for improvement around 0.5 agreement. Overall the autoencoder seems to be capturing important semantics at a level that could be useful for initial analysis or clustering, while still allowing for human validation and refinement of lower agreement areas

Contenu connexe

En vedette

Flipped classroom - Workplace learning - Transfer
Flipped classroom - Workplace learning - TransferFlipped classroom - Workplace learning - Transfer
Flipped classroom - Workplace learning - TransferPoh-Sun Goh
 
XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...
XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...
XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...Simon Buckingham Shum
 
Learning Analytics - George Siemens
Learning Analytics - George SiemensLearning Analytics - George Siemens
Learning Analytics - George SiemensSungjin Nam
 
Applied learning analytics to class and program learning activities
Applied learning analytics to class and program learning activitiesApplied learning analytics to class and program learning activities
Applied learning analytics to class and program learning activitiesPoh-Sun Goh
 
Role of digital analytics to guide learning (updated)
Role of digital analytics to guide learning (updated)Role of digital analytics to guide learning (updated)
Role of digital analytics to guide learning (updated)Poh-Sun Goh
 
Everything I have learnt about eLearning
Everything I have learnt about eLearningEverything I have learnt about eLearning
Everything I have learnt about eLearningPoh-Sun Goh
 
Applied learning analytics for modular content (updated)
Applied learning analytics for modular content (updated)Applied learning analytics for modular content (updated)
Applied learning analytics for modular content (updated)Poh-Sun Goh
 
One (more) representation of the continuum of learning
One (more) representation of the continuum of learningOne (more) representation of the continuum of learning
One (more) representation of the continuum of learningPoh-Sun Goh
 
Learning and Training for Transfer
Learning and Training for TransferLearning and Training for Transfer
Learning and Training for TransferPoh-Sun Goh
 
Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...
Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...
Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...Poh-Sun Goh
 
Applying learning analytics to the flipped classroom (updated)
Applying learning analytics to the flipped classroom (updated)Applying learning analytics to the flipped classroom (updated)
Applying learning analytics to the flipped classroom (updated)Poh-Sun Goh
 
The Value of Competency-based Medical Education Across the Continuum
The Value of Competency-based Medical Education Across the ContinuumThe Value of Competency-based Medical Education Across the Continuum
The Value of Competency-based Medical Education Across the ContinuumMedCouncilCan
 
Using data to guide (e)teaching and (e)learning
Using data to guide (e)teaching and (e)learningUsing data to guide (e)teaching and (e)learning
Using data to guide (e)teaching and (e)learningPoh-Sun Goh
 
edmedia2014-learning-analytics-keynote
edmedia2014-learning-analytics-keynoteedmedia2014-learning-analytics-keynote
edmedia2014-learning-analytics-keynoteSimon Buckingham Shum
 
Developing a multiple-document-processing performance assessment for epistem...
 Developing a multiple-document-processing performance assessment for epistem... Developing a multiple-document-processing performance assessment for epistem...
Developing a multiple-document-processing performance assessment for epistem...Simon Knight
 
Dcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dclaDcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dclaSimon Knight
 
SoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessmentSoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessmentSimon Knight
 
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talkWriting Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talkSimon Knight
 
CBME and Assessment
CBME and Assessment CBME and Assessment
CBME and Assessment jakinyi
 
Personal and personalised learning and teaching (updated)
Personal and personalised learning and teaching (updated)Personal and personalised learning and teaching (updated)
Personal and personalised learning and teaching (updated)Poh-Sun Goh
 

En vedette (20)

Flipped classroom - Workplace learning - Transfer
Flipped classroom - Workplace learning - TransferFlipped classroom - Workplace learning - Transfer
Flipped classroom - Workplace learning - Transfer
 
XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...
XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...
XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scient...
 
Learning Analytics - George Siemens
Learning Analytics - George SiemensLearning Analytics - George Siemens
Learning Analytics - George Siemens
 
Applied learning analytics to class and program learning activities
Applied learning analytics to class and program learning activitiesApplied learning analytics to class and program learning activities
Applied learning analytics to class and program learning activities
 
Role of digital analytics to guide learning (updated)
Role of digital analytics to guide learning (updated)Role of digital analytics to guide learning (updated)
Role of digital analytics to guide learning (updated)
 
Everything I have learnt about eLearning
Everything I have learnt about eLearningEverything I have learnt about eLearning
Everything I have learnt about eLearning
 
Applied learning analytics for modular content (updated)
Applied learning analytics for modular content (updated)Applied learning analytics for modular content (updated)
Applied learning analytics for modular content (updated)
 
One (more) representation of the continuum of learning
One (more) representation of the continuum of learningOne (more) representation of the continuum of learning
One (more) representation of the continuum of learning
 
Learning and Training for Transfer
Learning and Training for TransferLearning and Training for Transfer
Learning and Training for Transfer
 
Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...
Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...
Using data (analytics and analysis) to guide (e)teaching and (e)learning (upd...
 
Applying learning analytics to the flipped classroom (updated)
Applying learning analytics to the flipped classroom (updated)Applying learning analytics to the flipped classroom (updated)
Applying learning analytics to the flipped classroom (updated)
 
The Value of Competency-based Medical Education Across the Continuum
The Value of Competency-based Medical Education Across the ContinuumThe Value of Competency-based Medical Education Across the Continuum
The Value of Competency-based Medical Education Across the Continuum
 
Using data to guide (e)teaching and (e)learning
Using data to guide (e)teaching and (e)learningUsing data to guide (e)teaching and (e)learning
Using data to guide (e)teaching and (e)learning
 
edmedia2014-learning-analytics-keynote
edmedia2014-learning-analytics-keynoteedmedia2014-learning-analytics-keynote
edmedia2014-learning-analytics-keynote
 
Developing a multiple-document-processing performance assessment for epistem...
 Developing a multiple-document-processing performance assessment for epistem... Developing a multiple-document-processing performance assessment for epistem...
Developing a multiple-document-processing performance assessment for epistem...
 
Dcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dclaDcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dcla
 
SoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessmentSoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessment
 
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talkWriting Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
 
CBME and Assessment
CBME and Assessment CBME and Assessment
CBME and Assessment
 
Personal and personalised learning and teaching (updated)
Personal and personalised learning and teaching (updated)Personal and personalised learning and teaching (updated)
Personal and personalised learning and teaching (updated)
 

Similaire à Based on the sample codes and comparison metrics provided, it appears the autoencoder is performing fairly well at identifying concepts and semantic relationships in the data, with agreement scores ranging from 0.52 to 0.85 compared to human coding. Some areas like clients and consultants show very strong agreement over 0.7. Names and design show some room for improvement around 0.5 agreement. Overall the autoencoder seems to be capturing important semantics at a level that could be useful for initial analysis or clustering, while still allowing for human validation and refinement of lower agreement areas

Andy Kirk Malofiej 20 Presentation
Andy Kirk Malofiej 20 PresentationAndy Kirk Malofiej 20 Presentation
Andy Kirk Malofiej 20 PresentationAndy Kirk
 
Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...
Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...
Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...Sebastian Deterding
 
Understanding Content: The Stuff We Design For
Understanding Content: The Stuff We Design ForUnderstanding Content: The Stuff We Design For
Understanding Content: The Stuff We Design ForKaren McGrane
 
Before and after1online
Before and after1onlineBefore and after1online
Before and after1onlinecreativesvs
 
Cultural Intelligence PowerPoint Presentation Slides
Cultural Intelligence PowerPoint Presentation Slides Cultural Intelligence PowerPoint Presentation Slides
Cultural Intelligence PowerPoint Presentation Slides SlideTeam
 
Video Planning Presentation
Video Planning PresentationVideo Planning Presentation
Video Planning PresentationMr. Walters
 
Information Literacy Scope and Sequence
Information Literacy Scope and SequenceInformation Literacy Scope and Sequence
Information Literacy Scope and SequenceACob
 
Psycholinguistics : The Future of Behavioral Competency Matching
Psycholinguistics : The Future of Behavioral Competency MatchingPsycholinguistics : The Future of Behavioral Competency Matching
Psycholinguistics : The Future of Behavioral Competency MatchingTalview
 
Communication Skills Learning Styles In A Justice Environment
Communication Skills Learning Styles In A Justice Environment Communication Skills Learning Styles In A Justice Environment
Communication Skills Learning Styles In A Justice Environment Bradley W. Deacon
 
Master Workshop Business Modeling - The Institute Mexico
Master Workshop Business Modeling - The Institute MexicoMaster Workshop Business Modeling - The Institute Mexico
Master Workshop Business Modeling - The Institute MexicoTheInstituteMexico
 
Br visual research
Br   visual researchBr   visual research
Br visual researchbedfordgrp
 
Storytelling for research software engineers
Storytelling for research software engineersStorytelling for research software engineers
Storytelling for research software engineersAlbanLevy
 
AKSI 2 PART 1 materi bahasa indonesia.pptx
AKSI 2 PART 1 materi bahasa indonesia.pptxAKSI 2 PART 1 materi bahasa indonesia.pptx
AKSI 2 PART 1 materi bahasa indonesia.pptxammarsaputra23
 
Webquest
WebquestWebquest
Webquesttrevas
 

Similaire à Based on the sample codes and comparison metrics provided, it appears the autoencoder is performing fairly well at identifying concepts and semantic relationships in the data, with agreement scores ranging from 0.52 to 0.85 compared to human coding. Some areas like clients and consultants show very strong agreement over 0.7. Names and design show some room for improvement around 0.5 agreement. Overall the autoencoder seems to be capturing important semantics at a level that could be useful for initial analysis or clustering, while still allowing for human validation and refinement of lower agreement areas (20)

Andy Kirk Malofiej 20 Presentation
Andy Kirk Malofiej 20 PresentationAndy Kirk Malofiej 20 Presentation
Andy Kirk Malofiej 20 Presentation
 
Let the iPad CREATE
Let the iPad CREATELet the iPad CREATE
Let the iPad CREATE
 
Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...
Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...
Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/A...
 
Designer As Catalyst
Designer As CatalystDesigner As Catalyst
Designer As Catalyst
 
Understanding Content: The Stuff We Design For
Understanding Content: The Stuff We Design ForUnderstanding Content: The Stuff We Design For
Understanding Content: The Stuff We Design For
 
Before and after1online
Before and after1onlineBefore and after1online
Before and after1online
 
Cultural Intelligence PowerPoint Presentation Slides
Cultural Intelligence PowerPoint Presentation Slides Cultural Intelligence PowerPoint Presentation Slides
Cultural Intelligence PowerPoint Presentation Slides
 
Video Planning Presentation
Video Planning PresentationVideo Planning Presentation
Video Planning Presentation
 
Video Planning
Video PlanningVideo Planning
Video Planning
 
Information Literacy Scope and Sequence
Information Literacy Scope and SequenceInformation Literacy Scope and Sequence
Information Literacy Scope and Sequence
 
Psycholinguistics : The Future of Behavioral Competency Matching
Psycholinguistics : The Future of Behavioral Competency MatchingPsycholinguistics : The Future of Behavioral Competency Matching
Psycholinguistics : The Future of Behavioral Competency Matching
 
Tec314
Tec314Tec314
Tec314
 
Communication Skills Learning Styles In A Justice Environment
Communication Skills Learning Styles In A Justice Environment Communication Skills Learning Styles In A Justice Environment
Communication Skills Learning Styles In A Justice Environment
 
Master Workshop Business Modeling - The Institute Mexico
Master Workshop Business Modeling - The Institute MexicoMaster Workshop Business Modeling - The Institute Mexico
Master Workshop Business Modeling - The Institute Mexico
 
Br visual research
Br   visual researchBr   visual research
Br visual research
 
Storytelling for research software engineers
Storytelling for research software engineersStorytelling for research software engineers
Storytelling for research software engineers
 
"Educate" Presentation Handout for DECA Ontario Presidents' Summit
"Educate" Presentation Handout for DECA Ontario Presidents' Summit"Educate" Presentation Handout for DECA Ontario Presidents' Summit
"Educate" Presentation Handout for DECA Ontario Presidents' Summit
 
UDL
UDLUDL
UDL
 
AKSI 2 PART 1 materi bahasa indonesia.pptx
AKSI 2 PART 1 materi bahasa indonesia.pptxAKSI 2 PART 1 materi bahasa indonesia.pptx
AKSI 2 PART 1 materi bahasa indonesia.pptx
 
Webquest
WebquestWebquest
Webquest
 

Based on the sample codes and comparison metrics provided, it appears the autoencoder is performing fairly well at identifying concepts and semantic relationships in the data, with agreement scores ranging from 0.52 to 0.85 compared to human coding. Some areas like clients and consultants show very strong agreement over 0.7. Names and design show some room for improvement around 0.5 agreement. Overall the autoencoder seems to be capturing important semantics at a level that could be useful for initial analysis or clustering, while still allowing for human validation and refinement of lower agreement areas

  • 1.
  • 2.
  • 3.
  • 4. HOW RESEARCH INTO EPISTEMIC NETWORK ANALYSIS MIGHT INFORM DISCOURSE-CENTRIC LEARNING ANALYTICS David Williamson Shaffer
  • 5. ENA DCLA David Williamson Shaffer
  • 6. A CANDLE David Williamson Shaffer
  • 7.
  • 8.
  • 9. Students Teacher
  • 10. Students Teacher Lesson Plan
  • 11. Students Teacher Lesson Tests Plan
  • 12. Students Teacher Lesson Tests Plan
  • 13. Students Teacher Lesson Tests Plan
  • 14. Students Teacher How do learning analytics and discourse analysis change this picture? Lesson Tests Plan
  • 15.
  • 16.
  • 17.
  • 18. I would set up a experimental ultrafiltration membrane and test the surfactants.
  • 19. First you have to set up a controlled experiment with no carbon nanotubes. Record how membrane fouling acts upon the membrane with no carbon nanotubes. Then, slowly increase the carbon nanotube percentage with each experiment and compare them to the controlled experiment along with all the experiments before it. You should then be able to figure out what percentage is most effective.
  • 20. First you have to set up a controlled experiment with no carbon nanotubes. Record how membrane fouling acts upon the membrane with no carbon nanotubes. Then, slowly increase the carbon nanotube percentage with each experiment and compare them to the controlled experiment along with all the experiments before it. You should then be able to figure out what percentage is most effective.
  • 21.
  • 22.
  • 23.
  • 24.
  • 27. d = + 0.28
  • 28. d = + 0.28
  • 29. d = + 0.91
  • 30. d = + 0.91
  • 31. Students Teacher Lesson Tests Plan
  • 32. Students Mentor Lesson Tests Plan
  • 33. Student Student Mentor Student Lesson Tests Plan
  • 34. Student Chat Student Mentor Hub Student Lesson Tests Plan
  • 35.
  • 36.
  • 37. Student Chat Student Mentor Hub Student Tests Playbook
  • 38. Student Chat Student Mentor Hub Student Tests Playbook
  • 39. Student Chat Student Mentor Hub Student Utterance Table Tests Playbook
  • 40.
  • 41.
  • 43. GIGO
  • 45.
  • 46.
  • 47. We value what we can measure —Don Norman
  • 48. We can only meaningfully measure based on theory
  • 52. An example to think with
  • 53.
  • 54.
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
  • 61.
  • 68. What are the necessary and sufficient conditions of knowledge? What makes justified beliefs justified?
  • 69. What are the necessary and sufficient conditions of knowledge? What makes justified beliefs justified? How do people make decisions and justify actions?
  • 71.
  • 72.
  • 73.
  • 74.
  • 75. Complex thinking happens in cultures of practice
  • 76.
  • 77.
  • 78.
  • 79. 1. Apply virtual communication 4. Improve the visibility of both tools and facilitation techniques formal and informal skills. to more effectively connect people from around the world. 5. Use dashboards linked to collaborative tools to capture 2. Use collaborative spaces to key realtime information about gather knowledge, express people, activities and outcomes. ideas and concerns and share passions. 6. Provide more frequent guidance and link performance 3. Spend more time on setting to recognition. organizational context and communicating where the organization needs to go.
  • 80. The environmental factors within MMORPGs can be 75% applied to enhance leadership effectiveness for the globally integrated enterprise
  • 81.
  • 82.
  • 83.
  • 84.
  • 85.
  • 86.
  • 87. Game v Corporate Skills 100 80 Corporate Skills 60 40 20 0 0 20 40 60 80 100 Game Skills
  • 88. The skills and techniques 61% of the gaming leader cannot be transferred directly to the leadership of the virtual corporation
  • 89. Complex thinking happens in cultures of practice Cultures of practice are characterized by their patterns of Discourse
  • 90.
  • 92. T thinking that reshapes what we are doing while we are doing it
  • 93. thinking that reshapes what we are doing while we are doing it K ways of knowing
  • 94. D ways of doing thinking that reshapes what we are doing while we are doing it ways of knowing
  • 95. ways of doing P practicum ways of knowing
  • 96. A action practicum R reflection- on-action
  • 97. action practicum reflection- on-action
  • 98. action practicum reflection- on-action
  • 99. practicum R eflection-in-action
  • 101. Skills Knowledge Identity Values Epistemology
  • 102. Skills Knowledge Identity Values Epistemology
  • 103. Complex thinking happens in cultures of practice Cultures of practice are characterized by their patterns of Discourse Patterns of Discourse can be described in terms of an epistemic frame
  • 105. ©2005, 2008 David Williamson Shaffer
  • 106. ©2005, 2008 David Williamson Shaffer
  • 107. ©2005, 2008 David Williamson Shaffer
  • 108. B A ©2005, 2008 David Williamson Shaffer
  • 109. B A ©2005, 2008 David Williamson Shaffer
  • 110. B D C A ©2005, 2008 David Williamson Shaffer
  • 111. B F E D C A ©2005, 2008 David Williamson Shaffer
  • 112. E B A C F D ©2005, 2008 David Williamson Shaffer
  • 113. B C A D F E ©2005, 2008 David Williamson Shaffer
  • 114. B F A D C E ©2005, 2008 David Williamson Shaffer
  • 115. ©2005, 2008 David Williamson Shaffer
  • 116. B A ©2005, 2008 David Williamson Shaffer
  • 117. C B A ©2005, 2008 David Williamson Shaffer
  • 118. C F B A ©2005, 2008 David Williamson Shaffer
  • 119. C F D B A ©2005, 2008 David Williamson Shaffer
  • 120. C F D B A E ©2005, 2008 David Williamson Shaffer
  • 121. C F D B A E ©2005, 2008 David Williamson Shaffer
  • 122. C F D B A E ©2005, 2008 David Williamson Shaffer
  • 123. C F D B A E ©2005, 2008 David Williamson Shaffer
  • 124. C F D B A E ©2005, 2008 David Williamson Shaffer
  • 125. C F D B A E ©2005, 2008 David Williamson Shaffer
  • 127. Skills Knowledge Identity Values Epistemology
  • 128.
  • 129.
  • 130.
  • 131.
  • 132.
  • 133.
  • 134.
  • 135.
  • 136.
  • 137.
  • 138.
  • 141. Structural Functional Linguistic Codes Skills Knowledge Identity Values Epistemology
  • 142. Structural Functional Linguistic Codes Skills Knowledge Identity Values Epistemology Domain codes
  • 143. Structural Functional Linguistic Codes Domain codes
  • 144.
  • 145.
  • 146.
  • 147. Structural Functional Linguistic Codes Skills Knowledge Identity Values Epistemology Domain codes
  • 148.
  • 149.
  • 150.
  • 151.
  • 152.
  • 154. Sentences Verbs
  • 155. Sentences Verbs Nouns (Adj)
  • 156. Sentences Verbs Nouns (Adj) People
  • 158. Structural Functional Linguistic Codes Domain codes
  • 159. Structural Functional Linguistic Codes Domain codes Conjunctive Coding
  • 160. Structural Functional Linguistic Codes Domain codes
  • 161. Structural Functional Linguistic Codes Value of Surfactants Domain codes
  • 162. Structural Functional Linguistic Codes Value of Surfactants Domain codes
  • 163. Structural Functional Linguistic Codes Domain codes Engineering Identity
  • 164. Structural Functional Linguistic Codes Domain codes Engineering Identity
  • 165. Human Agreement with AutoCoder E and AC N1 and N2 E1 and E2 E/Data 0.521463 0.321308 0.697609 E/Design 0.64044 0.479389 0.54518 E/Clients 0.746846 0.796759 0.746846 E/Consultants 0.764168 0.566491 0.853813 V/Clients 0.538768 0.663687 0.663311 V/Consultants 0.493692 0.663311 0.663311 S/Data 0.76369 0.615417 0.940922 S/Design 0.801079 0.466942 0.728728 S/Professionalism 0.645465 0.456452 0.72107 S/Collaboration 0.694157 0.666465 0.888954 I/Engineer 0.479239 0.824869 0.560584 I/Intern 0.921389 0.816862 0.853813 K/Nanotech 1 0.796759 1 K/Surfactants 0.95827 0.965993 0.919605 K/Attributes 0.981197 0.900837 0.990646 K/Data 0.63988 0.784384 0.869369 K/Design 0.728741 0.771417 0.774949 K/Clients 1 0.945658 1 K/Materials 0.637786 0.916048 0.984033 K/Processes 0.364711 0.811285 0.95827
  • 166. Human Agreement with AutoCoder E and AC N1 and N2 E1 and E2 E/Data 0.521463 0.321308 0.697609 E/Design 0.64044 0.479389 0.54518 E/Clients 0.746846 0.796759 0.746846 E/Consultants 0.764168 0.566491 0.853813 V/Clients 0.538768 0.663687 0.663311 V/Consultants 0.493692 0.663311 0.663311 S/Data 0.76369 0.615417 0.940922 S/Design 0.801079 0.466942 0.728728 S/Professionalism 0.645465 0.456452 0.72107 S/Collaboration 0.694157 0.666465 0.888954 I/Engineer 0.479239 0.824869 0.560584 I/Intern 0.921389 0.816862 0.853813 K/Nanotech 1 0.796759 1 K/Surfactants 0.95827 0.965993 0.919605 K/Attributes 0.981197 0.900837 0.990646 K/Data 0.63988 0.784384 0.869369 K/Design 0.728741 0.771417 0.774949 K/Clients 1 0.945658 1 K/Materials 0.637786 0.916048 0.984033 K/Processes 0.364711 0.811285 0.95827
  • 167. Human Agreement with AutoCoder E and AC N1 and N2 E1 and E2 E/Data 0.521463 0.321308 0.697609 E/Design 0.64044 0.479389 0.54518 E/Clients 0.746846 0.796759 0.746846 E/Consultants 0.764168 0.566491 0.853813 V/Clients 0.538768 0.663687 0.663311 V/Consultants 0.493692 0.663311 0.663311 S/Data 0.76369 0.615417 0.940922 S/Design 0.801079 0.466942 0.728728 S/Professionalism 0.645465 0.456452 0.72107 S/Collaboration 0.694157 0.666465 0.888954 I/Engineer 0.479239 0.824869 0.560584 I/Intern 0.921389 0.816862 0.853813 K/Nanotech 1 0.796759 1 K/Surfactants 0.95827 0.965993 0.919605 K/Attributes 0.981197 0.900837 0.990646 K/Data 0.63988 0.784384 0.869369 K/Design 0.728741 0.771417 0.774949 K/Clients 1 0.945658 1 K/Materials 0.637786 0.916048 0.984033 K/Processes 0.364711 0.811285 0.95827
  • 168. Human Agreement with AutoCoder E and AC N1 and N2 E1 and E2 E/Data 0.521463 0.321308 0.697609 E/Design 0.64044 0.479389 0.54518 E/Clients 0.746846 0.796759 0.746846 E/Consultants 0.764168 0.566491 0.853813 V/Clients 0.538768 0.663687 0.663311 V/Consultants 0.493692 0.663311 0.663311 S/Data 0.76369 0.615417 0.940922 S/Design 0.801079 0.466942 0.728728 S/Professionalism 0.645465 0.456452 0.72107 S/Collaboration 0.694157 0.666465 0.888954 I/Engineer 0.479239 0.824869 0.560584 I/Intern 0.921389 0.816862 0.853813 K/Nanotech 1 0.796759 1 K/Surfactants 0.95827 0.965993 0.919605 K/Attributes 0.981197 0.900837 0.990646 K/Data 0.63988 0.784384 0.869369 K/Design 0.728741 0.771417 0.774949 K/Clients 1 0.945658 1 K/Materials 0.637786 0.916048 0.984033 K/Processes 0.364711 0.811285 0.95827
  • 169. Human Agreement with AutoCoder E and AC N1 and N2 E1 and E2 E/Data 0.521463 0.321308 0.697609 E/Design 0.64044 0.479389 0.54518 E/Clients 0.746846 0.796759 0.746846 E/Consultants 0.764168 0.566491 0.853813 V/Clients 0.538768 0.663687 0.663311 V/Consultants 0.493692 0.663311 0.663311 S/Data 0.76369 0.615417 0.940922 S/Design 0.801079 0.466942 0.728728 S/Professionalism 0.645465 0.456452 0.72107 S/Collaboration 0.694157 0.666465 0.888954 I/Engineer 0.479239 0.824869 0.560584 I/Intern 0.921389 0.816862 0.853813 K/Nanotech 1 0.796759 1 K/Surfactants 0.95827 0.965993 0.919605 K/Attributes 0.981197 0.900837 0.990646 K/Data 0.63988 0.784384 0.869369 K/Design 0.728741 0.771417 0.774949 K/Clients 1 0.945658 1 K/Materials 0.637786 0.916048 0.984033 K/Processes 0.364711 0.811285 0.95827
  • 170.
  • 171.
  • 172.
  • 173.
  • 174. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 3 0 0 0 0 1 0 1 0 0 1 0 0 0 0
  • 175. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 3 0 0 0 0 1 0 1 0 0 1 0 0 0 0
  • 176. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 3 0 0 0 0 1 0 1 0 0 1 0 0 0 0 Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 Skills 2 3 1 Identity Knowledge 2 1 3 1 1 2 1 Values 2 3 Epistemology 1 2 3
  • 177. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 3 0 0 0 0 1 0 1 0 0 1 0 0 0 0 Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 0 0 0 1 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 178. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 1 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 179. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 1 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 180. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 2 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 181. Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 4 0 0 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 4 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 182. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 4 0 0 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 4 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 183. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 4 0 0 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 4 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 184. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 4 0 0 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 4 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 185. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Skills Knowledge Identity Values Epistemology 1 2 3 1 2 3 1 2 1 2 3 1 2 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Identity Knowledge 2 0 0 0 1 0 0 4 0 0 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 4 0 0 0 0 1 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 2 0 0 0 0 1 0 2 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 186. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 187. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 Skills 0 0 0 Knowledge 0 0 Identity 0 0 0 Values 0 0 Epistemology 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 2 0 3 0 1 0 2 0 3 0 1 0 02 1 0 02 3 0 1 0 02 3 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Skills 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 03 20 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 20 0 0 0 0 0 0 0 1 0 0 0 0 01 0 0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 Identity Knowledge 0 0 0 0 0 0 0 0 0 0 0 0 02 20 0 0 0 0 1 0 0 0 0 0 4 0 00 0 0 02 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 4 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 20 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 Values 0 0 0 0 0 0 0 0 0 0 0 0 02 20 0 0 0 0 0 0 1 0 0 0 2 0 00 0 0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 03 20 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 Epistemology 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 03 22 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  • 188.
  • 189.
  • 190.
  • 191.
  • 192.
  • 193.
  • 194.
  • 195.
  • 196.
  • 197.
  • 198.
  • 199.
  • 200.
  • 201.
  • 202.
  • 203.
  • 204.
  • 205.
  • 206.
  • 207.
  • 208.
  • 209.
  • 210.
  • 211.
  • 212.
  • 213.
  • 214.
  • 215.
  • 216.
  • 217.
  • 218. Complex thinking happens in cultures of practice Cultures of practice are characterized by their patterns of Discourse Patterns of Discourse can be described in terms of an epistemic frame An epistemic frame can be quantified using a network model (ENA)
  • 219.
  • 220.
  • 221.
  • 222.
  • 223.
  • 224.
  • 225.
  • 226. S1 x K2 S1 x V2 S1 x K1
  • 227. S1 x K2 S1 x V2 S1 S1 x K1
  • 228. K2 S1 x K2 S1 x V2 S1 V2 S1 x K1 K1
  • 229. K2 S1 x K2 S1 x V2 S1 V2 S1 x K1 K1
  • 230. K2 Equiload S1 x K2 Projection S1 x V2 S1 V2 S1 x K1 K1
  • 231.
  • 232. Player 1: Good Morning Player 2: Hello everyone, is this chat with our new group now? Player 1: I think so Player 3: Probably Player 1: Good morning new team. I excited to work with with everyone. Player 4: hey Player 1: We are meeting and currently discussing our various materials.
  • 233. Player 1: Good Morning Player 2: Hello everyone, is this chat with our new group now? Player 1: I think so Player 3: Probably Player 1: Good morning new team. I excited to work with with everyone. Player 4: hey Player 1: We are meeting and currently discussing our various materials.
  • 234.
  • 235. Player 1: We ranked the different materials based on the device specifications and our experimental data. Player 2: And we just need to decide on the 5th team experimental device. Player 3: We looked at… how the cost changed with each matieral while controling everything else Player 4: hey team, I am going to post the rankings of the membranes as a public document so everyone can see it.
  • 236.
  • 237.
  • 238.
  • 241. Professionalism Product
  • 242. Professionalism Product
  • 243. Professionalism Design Product
  • 244. Professionalism Design Product
  • 245. Professionalism Design Product Materials
  • 246. Professionalism Design Product Materials
  • 247. Professionalism Design Product Materials
  • 248. Professionalism Design Product Materials
  • 249. Professionalism Design Product Materials
  • 250. Professionalism Design Product Materials
  • 251. Professionalism Design Product Materials
  • 252. Professionalism Design Product Materials
  • 253. Professionalism Design Product Materials
  • 254. Professionalism Design Product Materials
  • 255. Trajectories of Mastery Professionalism Design Product Materials
  • 256. d = + 0.28
  • 257.
  • 258.
  • 259.
  • 260.
  • 261.
  • 262.
  • 263.
  • 264.
  • 265. Complex thinking happens in cultures of practice Cultures of practice are characterized by their patterns of Discourse Patterns of Discourse can be described in terms of an epistemic frame An epistemic frame can be quantified using a network model (ENA) ENA can model complex thinking
  • 266. Student Chat Student Mentor Hub Student Utterance Table Tests Playbook
  • 267. Student Chat Student Mentor Hub Student Utterance Table ENA Tests Playbook
  • 268. Student Chat Student Mentor Hub Student Utterance Table ENA Playbook Student Models
  • 269. Student Chat Student Mentor Hub Student Utterance Table ENA Playbook Student Models
  • 270. Student Chat Student Mentor Hub Student Speech Act Utterance Table ENA Playbook Student Models
  • 271. Student Chat Student Mentor Hub Student Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 272. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 273. Scope Trigger Scope Action Scope
  • 274.
  • 275.
  • 276.
  • 277. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 278. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 279. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 280. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 281. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Student Models
  • 282. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Summative
  • 283. Student Chat Student Mentor Hub Student Suggest Auto Speech Mentor Act Utterance Table ENA Playbook Summative
  • 284. Student Chat Student Mentor Hub Student Suggest Speech Formative Act Utterance Table ENA Playbook Summative
  • 285. Student Chat Auto Student Hub Mentor Student Suggest Using ENA in games to create fummative assessments Speech Formative Act Utterance Table ENA Playbook Summative
  • 286. Student Chat Auto Student Hub Mentor Student Suggest Using ENA in games to create sormative assessments Speech Formative Act Utterance Table ENA Playbook Summative
  • 287. Auto Mentor Using ENA in games to create sormative assessments
  • 288. Art Graesser Auto Mentor Using ENA in games to create sormative assessments
  • 289. Art Graesser Auto Mentor Andre Rupp Using ENA in games to create sormative assessments
  • 290. Art Graesser Auto Mentor Andre Rupp Using ENA in games to create sormative assessments Kris Scopinich
  • 291. Art Graesser Auto Mentor Andre Rupp Using ENA in games to create sormative assessments Kris Scopinich Asligul Goḉmen
  • 292. Skills Knowledge Identity Values Epistemology
  • 293. Skills Knowledge Identity Values Epistemology
  • 294. Skills Knowledge Identity Values Epistemology