Results of my JavaScript SEO experiments and our latest research of how to make any JS framework indexable!

1. Javascript Indexing
and SEO:
The Naked Truth
@bart_goralewicz
Bartosz Góralewicz
linkedin.com/in/goralewicz

2. Why is JavaScript SEO such a complex topic?

3. DATANO
Why is JavaScript SEO such a complex topic?

4. DATANO
Making JavaScript SEO easy

5. JS SEO

6. JS SEO

7. What is the problem with JavaScript?
HTML
JS

8. What is the problem with JavaScript?
HTML
JS

9. What is the problem with JavaScript?
HTML
HTML

10. What is the problem with JavaScript?
HTML
HTML

11. JavaScript

12. and JavaScript

13. JavaScript & SEO

14. 50 shades of JavaScript

15. *more about it later
not as “forgiving” as HTML*

16. It has all with…

18. Reverse
engineering
algorithms is
no longer an
option.
Times have changed. Today, as
long as you're not blocking
Googlebot from crawling your
JavaScript or CSS files, we are
generally able to render and
understand your web pages like
modern browsers. To reflect this
improvement, we recently updated
our technical Webmaster
Guidelines to recommend against
disallowing Googlebot from
crawling your site's CSS or JS files.
ele.ph/ajax

19. ele.ph/scotch

20. ele.ph/scotch

21. Can this backfire?

22. Hulu.com
JavaScript & SEO backfire

23. -37%

26. ele.ph/hulu
Hulu case study

27. Google and JavaScript – difficult relationship

28. Google and JavaScript – difficult relationship

29. The only solution…
Java Script Experiment

30. Kamil
Grymuza

31. http://jsseo.expert

37. Content!
OK.

38. ele.ph/AI

40. Inspect Element
vs.
View Source

41. Inspect element

42. Inspect element

43. View source

44. First JavaScript rendering issues

45. Timeout, GSC bug or real ?

46. /test/

48. JavaScript links were
not parsed by Googlebot
Google
kept us
waiting…

50. Fail?

51. Re-fetched all URLs via GSC

55. /html/test/ (duh!)
/plain-inline/test/
/jquery-inline/test/
/jquery-ajax/test/
/vue/test/
/react-inline/test/
And… that’s it! /test/

56. Let’s start with JS frameworks created by

57. Version 1

58. Checklist
Check if ”framework/test/” URL was crawled
Search for unique content from framework’s page
Is URL indexed by Google?
Fetch and render test (GSC)

59. Fetch and render test (GSC)

60. Is URL indexed by Google?

61. Search for unique content from framework’s page

62. Check if ”framework/test/” URL was crawled

63. V2 Inline
GSC
fetch
URL
Indexed?
Content
Indexed?
Test
URL
Inline Inline

65. GSC
fetch
URL
Indexed?
Content
Indexed?
Test
URL
External Ajax callInline

67. GSC
fetch
URL
Indexed?
Content
Indexed?
Test
URL
Inline External

69. GSC
fetch
URL
Indexed?
Content
Indexed?
Test
URL
Inline External

70. Inline vs. External JS makes a huge difference
Not all JavaScript frameworks are crawled
and indexed in the same way
Mindblowing findings
JS generated links aren’t always crawled

71. ele.ph/angularU
Also…
Regardless of what your
developers say, Angular JS 2
always has to be server
rendered…
according to
Angular JS 2 community.

72. If you search for any competitive
keyword terms, it’s always going
to be server rendered sites. And
the reason is because although
Google does index client-side
rendered HTML, it’s not perfect
yet and other search engines
don’t do it as well. So if you care
about SEO, you still need to have
server-rendered content.
Jeff Whelpley
Angular U conference, June 22-25, 2015, Hyatt Regency, San Francisco Airport
“Angular 2 Server Rendering”
ele.ph/angularU

73. ?How about Search Engines other than

75. 63,4%
22,8%
11,7%
1,3% 0,8%
Google Bing Yahoo! Ask AOL
Search Engine market share
Source: https://www.statista.com/statistics/267161/market-share-of-search-engines-in-the-united-states/

76. 83,5%
11,1%
3,8% 0,8% 0,6%
Google Bing Yahoo! MSN DuckDuckGo
Search Engine market share
Source: https://www.statista.com/statistics/280269/market-share-held-by-search-engines-in-the-united-kingdom/

77. ele.ph/beyondgoogle

78. ele.ph/jsforum

79. Tell me
why I
can’t…
Clear your
mind of
questions
*not the actual conversation

81. GSC
fetch
URL
Indexed?
Content
Indexed?
Test
URL
External Ajax callInline

83. http://jsseo.expert/plain-external/test/
http://jsseo.expert/react-external/test/
http://jsseo.expert/jquery-external/test/
we found this URL from a link ca 3 weeks ago, but we haven't crawled it
we picked this one up ~19 days ago, and similar to above,
our systems just didn't feel like crawling it yet
picked up ~21 days ago, also not crawled yet.

84. Thanks,
John!
You’re the
best,
Bartosz!*
*he didn’t say that
John Mueller’s feedback got me thinking…

85. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit
amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada
fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce
posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus.
Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros
quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin
pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar
ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi
tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra
imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy
pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus
malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et
netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor
congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce
est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem
ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet
commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac
turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere,
magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus.
Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros
quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin
pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar
ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi
tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra
imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy
pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus
malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et
netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor
congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce
JavaScript vs. crawler budget
Some JavaScript frameworks are indexable,
but there is a fine print…

86. JavaScript

87. BUT…
HTML

88. Do you already know where I am going with this?

90. Kamil
Grymuza

91. Crawler budget experiment (early beta)

92. We repeated this experiment a few times and
the results were always the same.

93. JavaScript,
even when indexable,
can kill your crawler budget…

94. JS is less
effective than
HTML
JavaScript is not
bad for SEO, if
done right*
*he said exactly that
Ilya Grigorik

95. “I think if you actually look at
why, and fix the underlying
JavaScript errors... you won't see
any practical difference. HTML
is very forgiving, JS less so.
JavaScript is not bad for SEO, if
done right.”
Ilya Grigorik
Is there a way
to improve JS
crawling?

96. Debugging JavaScript – Ilya’s style

97. “It's throwing JS errors on ES6
syntax, which is not supported
in M41. You can use the debug
snippet we provided in the blog
post to log out the error into the
DOM to see it.”
Ilya Grigorik
Let’s start
from decoding
Ilya’s words

98. How to make ANY* JS indexable?
*I’m not 100% sure about it, but it sounds cool. Right?

99. JS is less
effective than
HTML
JavaScript is not
bad for SEO, if
done right*
*he said exactly that
Ilya Grigorik
After talking to Ilya

100. and a lot of
investigation

101. We’ve managed to get
Angular 2 indexed by Google

102. How?

103. There was an error in
Angular 2’s source code
in official repository

104. team fucked up
Search
Angular
’s

105. April 26th: error was fixed
ele.ph/ang-fix

110. For other frameworks
Polyfills
Transpilers

111. What are
Transpilers
ele.ph/transpiler

112. What are
ele.ph/polyfill
Polyfills
(n) 1. A shim that mimics a future API,
providing fallback functionality to older
browers.
e.g. geolocation polyfill

113. So…
Every JS framework is indexable…
HTML is very
forgiving,
JS less so.
Ilya Grigorik
But

114. Is there a perfect JavaScript SEO scenario?

115. YES

116. The setup for that is called
Isomorphic JavaScript on
React & Universal on
Angular. That's what I'd
recommend doing in general,
if you can.
Isomorphic JavaScript
ele.ph/Iso-John

117. ele.ph/iso

118. What if it’s too late?

119. Prerender.io

120. user
server
middleware
Prerendering
crawlers
JS
HTML
JS
?
?

121. Understanding technology behind

122. ele.ph/ilya

123. ele.ph/chrome41

124. ele.ph/chrome41
Download Chrome 41 here:

125. Using Chrome 41
for debugging JS indexing

126. http://jsseo.expert/angular2/
in Chrome 41

127. http://jsseo.expert/angular2/
in up-to-date Chrome

128. Chrome 41
1. Open:

129. JSSEO.EXPERT
2. Open:

130. /
3. Open:
Chrome Developer Tools
cmd ctrl shift+ + i

132. 4. Show this sh*t to your developers

134. Polymer
No ES5 to ES6 compilation
Wrong application build
Polymer-cli related issues (whatever that means)
anyways… it works now.

135. Polymer
No ES5 to ES6 compilation
Wrong application build
Polymer-cli related issues (whatever that means)
anyways… it works now.

137. Experiments continue…

138. JS
Only Google and Ask
can handle JavaScript
Key takeways

139. JS
Only Google and Ask
can handle JavaScript
All JavaScript
frameworks are indexable
Key takeways

140. All JavaScript
frameworks are indexable
Indexing JavaScript
is only half the battle
Key takeways

141. Indexing JavaScript
is only half the battle
JavaScript even when
rendered and indexed,
can kill crawler budget
Key takeways

142. JavaScript even when
rendered and indexed,
can kill crawler budget
Use Chrome 41
to debug JS
Key takeways

143. Use Chrome 41
to debug JS
If too late,
use prerendering
Key takeways

144. If too late,
use prerendering
Key takeways
Isomorphic JS
=
perfect scenario

145. All JavaScript
frameworks are indexable
Indexing JavaScript
is only half the battle
JavaScript even when
rendered and indexed,
can kill crawler budget
JS
Only Google and Ask
can handle JavaScript
Use Chrome 41
to debug JS
If too late,
use prerendering
Isomorphic JS
=
perfect scenario
Key takeways

146. @bart_goralewicz
/in/goralewicz/
Thanks! :*