Presentation for DICE Coder's Day (2010 November) by Daniel Collin

1. Introduction to Data-Oriented Design

2. So what is this Data-Oriented Design?

3. It’s about on shifting focus to how data is read and written

4. Why should we care?

5. Performance

6. A read from memory takes ~600 cycles at 3.2 GHz

7. A read from memory takes 40 cycles at 300 MHz

8. Performance Disks (Blu-ray/DVD/HDD) Main Memory L2 Cache L1 Cache CPU / Registers Latency :( 600 cycles 40 cycles 1 – 2 cycles

12. Example - OOD class Bot { ... Vec3 m_position; ... float m_mod; ... float m_aimDirection; ... void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } }

13. Example - OOD class Bot { ... Vec3 m_position; ... float m_mod; ... float m_aimDirection; ... void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } } icache-miss

14. Example - OOD class Bot { ... Vec3 m_position; ... float m_mod; ... float m_aimDirection; ... void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } } icache-miss data-miss

15. Example - OOD class Bot { ... Vec3 m_position; ... float m_mod; ... float m_aimDirection; ... void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } } icache-miss data-miss Unused cached data

16. Example - OOD class Bot { ... Vec3 m_position; ... float m_mod; ... float m_aimDirection; ... void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } } icache-miss data-miss Unused cached data Very hard to optimize!

17. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots)

18. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600

19. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600

20. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600

21. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600 ~20 cycles

22. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 ~20 cycles

23. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 ~20 cycles iCache – 600 m_position – 600 m_mod - 600 aimDir – 100

24. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 ~20 cycles iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 iCache – 600 m_position – 600 m_mod - 600 aimDir – 100

25. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 ~20 cycles iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 iCache – 600 m_position – 600 m_mod - 600 aimDir – 100

26. Example - OOD void updateAim(Vec3 target) { m_aimDirection = dot3(m_position, target) * m_mod; } Lets say we call this code 4 times (4 diffrent Bots) iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 ~20 cycles iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 iCache – 600 m_position – 600 m_mod - 600 aimDir – 100 7680

27. Example - DOD

30. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } }

31. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } } What has changed?

32. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } } Only read needed inputs What has changed?

33. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } } Only read needed inputs Write to linear array What has changed?

34. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } } Only read needed inputs Write to linear array Loop over all the data What has changed?

35. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } } Only read needed inputs Write to linear array Loop over all the data Actual code unchanged What has changed?

36. Example - DOD void updateAims(float* aimDir,const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i],target) * aim->mod[i]; } } Only read needed inputs Write to linear array Loop over all the data Actual code unchanged What has changed? Code separated

37. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } }

38. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600

39. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600

40. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600

41. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600 ~20 cycles

42. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600 aimDir – 100 ~20 cycles

43. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600 aimDir – 100 ~20 cycles

44. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600 aimDir – 100 ~20 cycles

45. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600 aimDir – 100 ~20 cycles

46. Example - DOD void updateAims(float* aimDir, const AimingData* aim, Vec3 target, uint count) { for (uint i = 0; i < count; ++i) { aimDir[i] = dot3(aim->positions[i], target) * aim->mod[i]; } } iCache – 600 positions – 600 mod - 600 aimDir – 100 ~20 cycles 1980

47. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0

48. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

49. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

50. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

51. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

52. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

53. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

54. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

55. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

56. Data layout OOD vs DOD pos0 mod0 aimDir0 pos0 Pos1 mod1 aimDir1 pos0 pos0 pos0 pos1 pos1 pos1 pos1 pos2 pos2 pos2 pos2 pos3 pos3 pos3 pos3 mod0 mod1 mod2 mod3 aimDir0 aimDir1 aimDir2 aimDir3 pos0 pos0 pos0 Each color block is one 128 byte cache line

57. Its all about memory

61. Remember

64. Example: Area Triggers

65. Example: Area Triggers position position position position next position position position position next position position position position next Source data (Linked List)

66. Example: Area Triggers position position position position next position position position position next position position position position next Source data (Linked List) Native Data (Array) position position position position position position position position position position position position position position count

67. Example: Culling System

68. Example: Culling System Old System

69. Example: Culling System Old System New System (Linear arrays and brute force)

70. Example: Culling System Old System New System (Linear arrays and brute force) 3x faster, 1/5 code size, simpler

71. Data Oriented Design Delivers:

72. Better Performance

73. Often simpler code

74. More parallelizable code

75. Questions?