SlideShare une entreprise Scribd logo
1  sur  50
Télécharger pour lire hors ligne
A Small Talk on Getting Big.
Scaling a Rails App & all that Jazz.
Good Dog Picture.
Medium Dog.
Bad Dog Picture.
How did this Happen?
Mongrel Picture.
Too Much Time in the Application.
 Too Much Time in the Database.
A Really Cool Trick
class ApplicationController < ActionController:Base
    include ResponseManagementAdditions
    after_filter :show_runtimes

   around_filter { |controller, action| controller.total_runtime =
   Benchmark.measure(&action).real }
end
A Really Cool Trick
module ResponseManagementAdditions
   def show_runtimes
    return if response.headers['Status'] == '304 Not Modified' || !
(response.body && response.body.respond_to?(:sub!))

   db_runtime = ((0 + (@db_rt_before_render || 0) +
(@db_rt_after_render || 0)) * 1000).truncate
   rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate
   total_runtime = ((@total_runtime || 0) * 1000).truncate

    response.body.gsub!(/</body></html>$/, quot;<!-- served to you
through a copper wire by #{HOSTNAME} at
#{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r
#{rendering_runtime}). thank you, come again. -->
n</body></html>quot;)
   end
end
Two Ways People Design Sites.

       Over-architect.
       Under-architect.
Move Fast. Scale Quickly.
Too Much Time in the Application.
Abstract Long Running Processes
          to Daemons.
An Ugly but Amazingly Simple Queuing
              System.
class FooDaemon < TwitterDaemon::Base
 before_startup :set_fugly_dist_idx

 def process
  unprocessed_content do |c|
   increment_counter(:total)

       # Do work ...

   break unless running?
  end
 end

 ...
...

def unprocessed_content(&block)
   loop do
     content = ContentModel.pending_content(quot;substring(truncate(id,
0),-2,1) = #{@fugly_dist_idx}quot;)
     messages.each { |message| yield message }
     sleep 1 if messages.nil? || messages.empty?
   end
 end

 def set_fugly_dist_idx
  @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) }
  raise quot;You need to specify a dist idx between 0 and 9.quot; unless
@fugly_dist_idx
  @fugly_dist_idx = @fugly_dist_idx.to_i
 end
end
A Better Queuing System.
Starling.
Distributed Queuing.
   Transactional Playback.
            Fast.
           Simple.
Speaks Memcache's Language.
      100% Pure Ruby.
Not Open Source.
(yet ...)
Too Much Time in the Database.
The Basics. Database 101.
(I shouldn't need this slide in here.)
Index everything you will query on.
            Avoid complex joins.
Use joint indices when you must join tables.
     Avoid scanning large sets of data.
Cache.
Cache.
Cache.
CACHE!
But How?
That Sounds Hard.
Turns Out it Isn't.
Serialize, Denormalize.
class User < ActiveRecord::Base
 serialize :following_ids

 def following_ids
  # this accessor is overwritten because we want to lazily set the
  # friends_ids column, rather than running a gigantic slow migration.
  RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot;
  ids = read_attribute(:following_ids)

  if ids.nil? || !ids.kind_of?(Array)
    ids = connection.select_values(quot;SELECT DISTINCT followed_user_id
FROM followed_users WHERE user_id =
#{self.id}quot;).map(&:to_i).compact
    update_attribute(:following_ids, ids)
  end

  ids
 end

 ...
def following_ids_add(the_id)
  ids = self.following_ids.dup
  ids << the_id
  write_attribute(:following_ids, ids)
 end

 def following_ids_delete(the_id)
  ids = self.following_ids.dup
  ids.delete(the_id)
  write_attribute(:following_ids, ids)
 end

end # End Class
Oh yeah, and Cheat.
     (It's ok!)
Thing about your application.
How can you cheat and get away
           with it?
Is your data delivered in real time?
    Is your data static content?
      How do users interact?
Interestingness.
(Little things that don't deserve other space.)
It's OK to use Monit to kill processes
          if they get too big.
Ensure you can deploy frequently.
Ensure you can roll back easily.
Scale where it matters.
Some code is ugly. It's OK.
  (who needs a hug?)
Ensure your users can give feedback
              easily.
Use the Community.
Make an API.
(Scale your Developer-base.)
We run on Edge (but with Piston).
A Cool Trick. Gems in Vendor.
Rails::Initializer.run do |config|

 # Load Gems from the /vendor/gems folder first, if they exist.
 config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir|
  File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir
 end

 ...
Personal Pet Peeve.

It's 2007. Every spammer has your email address.
  Put it on your goddamn webpage so people can
      get ahold of you about interesting things.
Questions?
Britt Selvitelle

      IM & Email
anotherbritt@gmail.com

Contenu connexe

Tendances

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldChristian Melchior
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejsAmit Thakkar
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...Tom Croucher
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleTom Croucher
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebBryan Helmig
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmMartin Grider
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the futureJeff Miccolis
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015Matt Raible
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet CampPuppet
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXERBrian Moschel
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.liJoe Betz
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsGanesh Iyer
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title'"><x> '"><x>
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsDataStax Academy
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...Fwdays
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven ProgrammingKamal Hussain
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShellDale Lane
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.liKaran Parikh
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset
 

Tendances (20)

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected World
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejs
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWeb
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to Realm
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXER
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.li
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web Applications
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven Programming
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShell
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.li
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with Celery
 
AJAX Transport Layer
AJAX Transport LayerAJAX Transport Layer
AJAX Transport Layer
 

En vedette

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsAditya Rao
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at TwitterEvan Weaver
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter StackChris Aniszczyk
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisDvir Volk
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 

En vedette (6)

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessons
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at Twitter
 
OAuth
OAuthOAuth
OAuth
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter Stack
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 

Similaire à A Small Talk on Getting Big

[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...Functional Thursday
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable CodeBaidu, Inc.
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the codeWim Godden
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeWim Godden
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin Vasil Remeniuk
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkDaniel Spector
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!François-Guillaume Ribreau
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeWim Godden
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Kenneth Teh
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperfNew Relic
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best PracticesDavid Keener
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youAndrey Karpov
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..Mark Rackley
 

Similaire à A Small Talk on Getting Big (20)

The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
 
Rspec
RspecRspec
Rspec
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end Framework
 
NodeJS for Beginner
NodeJS for BeginnerNodeJS for Beginner
NodeJS for Beginner
 
Migrating legacy data
Migrating legacy dataMigrating legacy data
Migrating legacy data
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!
 
Coding Naked 2023
Coding Naked 2023Coding Naked 2023
Coding Naked 2023
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the code
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperf
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best Practices
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for you
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
 
Refactoring
RefactoringRefactoring
Refactoring
 
Refactoring
RefactoringRefactoring
Refactoring
 

Dernier

A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 

Dernier (20)

A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 

A Small Talk on Getting Big

  • 1. A Small Talk on Getting Big. Scaling a Rails App & all that Jazz.
  • 5. How did this Happen?
  • 7. Too Much Time in the Application. Too Much Time in the Database.
  • 8. A Really Cool Trick class ApplicationController < ActionController:Base include ResponseManagementAdditions after_filter :show_runtimes around_filter { |controller, action| controller.total_runtime = Benchmark.measure(&action).real } end
  • 9. A Really Cool Trick module ResponseManagementAdditions def show_runtimes return if response.headers['Status'] == '304 Not Modified' || ! (response.body && response.body.respond_to?(:sub!)) db_runtime = ((0 + (@db_rt_before_render || 0) + (@db_rt_after_render || 0)) * 1000).truncate rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate total_runtime = ((@total_runtime || 0) * 1000).truncate response.body.gsub!(/</body></html>$/, quot;<!-- served to you through a copper wire by #{HOSTNAME} at #{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r #{rendering_runtime}). thank you, come again. --> n</body></html>quot;) end end
  • 10. Two Ways People Design Sites. Over-architect. Under-architect.
  • 11. Move Fast. Scale Quickly.
  • 12. Too Much Time in the Application.
  • 13. Abstract Long Running Processes to Daemons.
  • 14. An Ugly but Amazingly Simple Queuing System.
  • 15. class FooDaemon < TwitterDaemon::Base before_startup :set_fugly_dist_idx def process unprocessed_content do |c| increment_counter(:total) # Do work ... break unless running? end end ...
  • 16. ... def unprocessed_content(&block) loop do content = ContentModel.pending_content(quot;substring(truncate(id, 0),-2,1) = #{@fugly_dist_idx}quot;) messages.each { |message| yield message } sleep 1 if messages.nil? || messages.empty? end end def set_fugly_dist_idx @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) } raise quot;You need to specify a dist idx between 0 and 9.quot; unless @fugly_dist_idx @fugly_dist_idx = @fugly_dist_idx.to_i end end
  • 19. Distributed Queuing. Transactional Playback. Fast. Simple. Speaks Memcache's Language. 100% Pure Ruby.
  • 22. Too Much Time in the Database.
  • 23. The Basics. Database 101. (I shouldn't need this slide in here.)
  • 24. Index everything you will query on. Avoid complex joins. Use joint indices when you must join tables. Avoid scanning large sets of data.
  • 30. Turns Out it Isn't.
  • 32. class User < ActiveRecord::Base serialize :following_ids def following_ids # this accessor is overwritten because we want to lazily set the # friends_ids column, rather than running a gigantic slow migration. RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot; ids = read_attribute(:following_ids) if ids.nil? || !ids.kind_of?(Array) ids = connection.select_values(quot;SELECT DISTINCT followed_user_id FROM followed_users WHERE user_id = #{self.id}quot;).map(&:to_i).compact update_attribute(:following_ids, ids) end ids end ...
  • 33. def following_ids_add(the_id) ids = self.following_ids.dup ids << the_id write_attribute(:following_ids, ids) end def following_ids_delete(the_id) ids = self.following_ids.dup ids.delete(the_id) write_attribute(:following_ids, ids) end end # End Class
  • 34. Oh yeah, and Cheat. (It's ok!)
  • 35. Thing about your application. How can you cheat and get away with it?
  • 36. Is your data delivered in real time? Is your data static content? How do users interact?
  • 37. Interestingness. (Little things that don't deserve other space.)
  • 38. It's OK to use Monit to kill processes if they get too big.
  • 39. Ensure you can deploy frequently.
  • 40. Ensure you can roll back easily.
  • 41. Scale where it matters.
  • 42. Some code is ugly. It's OK. (who needs a hug?)
  • 43. Ensure your users can give feedback easily.
  • 45. Make an API. (Scale your Developer-base.)
  • 46. We run on Edge (but with Piston).
  • 47. A Cool Trick. Gems in Vendor. Rails::Initializer.run do |config| # Load Gems from the /vendor/gems folder first, if they exist. config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir| File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir end ...
  • 48. Personal Pet Peeve. It's 2007. Every spammer has your email address. Put it on your goddamn webpage so people can get ahold of you about interesting things.
  • 50. Britt Selvitelle IM & Email anotherbritt@gmail.com