Diving in Rails - The request handling

Warning : This article was written a while ago, my views may have changed and this may not be relevant anymore.

Introduction #

There is an aspect of Rails I both like and dislike, Rails is automagical.

It does a lot of complicated things for you and makes the job super easy, however, it can be kind of frustrating, this feeling of relying on some magical piece of code.

I wanted to have a better grasp on the core problems Rails try to resolve.

You probably know it, reading the source of software, frameworks, libraries, is one of the best way to learn how they work.

In this article, I’ll talk about the stuff I learned while reading Rails’s source code, it may be the first of an undefined number of similar articles.

Today I’ll speak about how Rails handles requests, its relation to rack and more globally what is happening when you submit a request to a HTTP server backed by a Rails application.

Rack #

If you have ever worked with Ruby, you probably already know what Rack is.

If you know everything about rack and only want to read about the rails part, you can skip this part

Rack is an interface between a HTTP server, and a Ruby application. It is a standard language between those two parts.

Almost all of ruby web applications use Rack.

When the HTTP Server (for example Puma, Unicorn, Passenger, and many others) receives a HTTP request, it will create a hash that, in rack language, is referred to as the env.

The Rack environment contains all useful data needed to handle a web request. It contains the request headers, the request body, informations about the client, etc.

A Rack application is any ruby object that can respond to the call method, it can be a lambda, or an instance of a class that defines a call method.

The HTTP server will invoke the call method with one parameter, the Rack env.

To provide a HTTP response, the Rack application has to return an array composed of 3 entries.

Here is one of the simplest rack apps you could imagine (example taken on http://rack.github.io)

require 'rack'

app = Proc.new do |env|
    ['200', {'Content-Type' => 'text/html'}, ['A barebones rack app.']]
end

With the rackup tool, you can call run on your app to spawn a web server that will use this app.

One essential feature of Rack is called Middlewares.

I like to see Rack as a two-ways pipeline. Basically, your request goes through the pipeline to your app, and then goes back to the web server.

What is awesome is that you can add some code in the pipeline to modify either the request or the response.

A middleware is just like a Rack app, except that you construct it with another Rack application.

For example, In the call method of your middleware, you can then invoke call on the app you built it with, and modify the body before returning it.

Let’s say I want to build a Rack Middleware that will transform the body to only capital letters.

First, we need to define its initialize method, it receives a rack application as the only parameter :

class UpperCaseMiddleware
  def initialize(app)
    @app = app
  end
end

All we need to do here is store it for later processing.

Then, like any other Rack app, the middleware has to have a call method that takes a rack env, and to return something.

For now, our middleware will do nothing, we can just return @app.call(env) :

class UpperCaseMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    @app.call(env)
  end

You can add this middleware to your Rack stack by adding this in a config.ru file :

require 'rack'

app = Proc.new do |env|
    ['200', {'Content-Type' => 'text/html'}, ['A barebones rack app.']]
end

use UpperCaseMiddleWare

run app

If you run this app, you will see that nothing changed at all, good !

So, what is basically going on when a request is made :

Now that we know that, we can update our middleware to actually do something on the request :

def call(env)
  status, headers, body = @app.call(env)
  body.upcase!
  [status, headers, body]
end

Ok, now we know the basics of handling HTTP requests with ruby.

Back to rails ! #

As you probably guessed if you’re a Rails newbie, Rails is a Rack application like the one we just built, only it’s much much larger.

When you create a Rails application, you can see a config.ru file :

# This file is used by Rack-based servers to start the application.

require ::File.expand_path('../config/environment',  __FILE__)
run Rails.application

This file loads your config/environment.rb file then runs the Rack application in Rails.application

Rails.application is an attribute accessor defined in railties/lib/rails.rb :

module Rails
  class << self
    attr_accessor :application, :cache, :logger

    # .... many lines
  end
end

config/environment.rb will Initialize your rails application with :

YourAppName::Application.initialize!

initialize! does not exist on Rails::Application but thanks to a method_missing on the Railtie class (which is one of the ancestors of Rails::Application), calling initialize! will create an instance of the application and forward the method invokation to it.

When instanciating a rails app, Rails.application is set in railties/lib/rails/application.rb :

def initialize(initial_variable_values = {}, &block)
   super()
   # Initialization code ...
   Rails.application ||= self
   # More initialization code
end

Pfeew, now that we have a Rails.application, rack can use it to speak with our app.

Let’s have a look at the call method.

# Implements call according to the Rack API. It simply
# dispatches the request to the underlying middleware stack.
def call(env)
  env["ORIGINAL_FULLPATH"] = build_original_fullpath(env)
  env["ORIGINAL_SCRIPT_NAME"] = env["SCRIPT_NAME"]
  super(env)
end

Rails::Application Inherits from Rails::Engine, here is the call method for Rails::Engine, I’ve also included other relevant methods :

# Define the Rack API for this engine.
def call(env)
  env.merge!(env_config)
  if env['SCRIPT_NAME']
    env.merge! "ROUTES_#{routes.object_id}_SCRIPT_NAME" => env['SCRIPT_NAME'].dup
  end
  app.call(env)
end

# Defines additional Rack env configuration that is added on each call.
def env_config
  @env_config ||= {
    'action_dispatch.routes' => routes
  }
end

# Returns the underlying rack application for this engine.
def app
  @app ||= begin
    config.middleware =  config.middleware.merge_into(default_middleware_stack)
    config.middleware.build(endpoint)
  end
end

So as we can see, Rails inserts the routes in a action_dispatch.routes key into the rack env, then delegates the call to the method app.

The app method will build the middleware stack and return the actual rails app.

config.middleware.build (code here) will return a ActionDispatch::MiddlewareStack.

This is the middleware stack, this object is composed of middlewares, each middleware is instantiated with the one that follows him in the stack. The last middleware is instantiated with the actual rack app.

Invoking call on this object will go through all the middlewares and the app.

Indeed, Rails is composed of several middlewares used to handle the request.

Here is the middleware stack of a pristine rails 4.1 app : 

use Rack::Sendfile
use ActionDispatch::Static
use Rack::Lock
use #<ActiveSupport::Cache::Strategy::LocalCache::Middleware:0x007fbaa3b9d718>
use Rack::Runtime
use Rack::MethodOverride
use ActionDispatch::RequestId
use Rails::Rack::Logger
use ActionDispatch::ShowExceptions
use ActionDispatch::DebugExceptions
use ActionDispatch::RemoteIp
use ActionDispatch::Reloader
use ActionDispatch::Callbacks
use ActiveRecord::Migration::CheckPending
use ActiveRecord::ConnectionAdapters::ConnectionManagement
use ActiveRecord::QueryCache
use ActionDispatch::Cookies
use ActionDispatch::Session::CookieStore
use ActionDispatch::Flash
use ActionDispatch::ParamsParser
use Rack::Head
use Rack::ConditionalGet
use Rack::ETag
run MyApp::Application.routes

What’s important here is the last line : run MyApp::Application.routes

Let’s have a look ! It’s defined in railties/lib/rails/engine.rb :

# Defines the routes for this engine. If a block is given to
# routes, it is appended to the engine.
def routes
  @routes ||= ActionDispatch::Routing::RouteSet.new
  @routes.append(&Proc.new) if block_given?
  @routes
end

Ok, let’s look at the call method of ActionDispatch::Routing::RouteSet.new, are we done yet? I don’t think so …

Here it is

def call(env)
  @router.call(env)
end

@router is an instance of Journey::Router, here is its call method :

def call(env)
  env['PATH_INFO'] = Utils.normalize_path(env['PATH_INFO'])

  find_routes(env).each do |match, parameters, route|
    script_name, path_info, set_params = env.values_at('SCRIPT_NAME',
                                                       'PATH_INFO',
                                                       @params_key)

    unless route.path.anchored
      env['SCRIPT_NAME'] = (script_name.to_s + match.to_s).chomp('/')
      env['PATH_INFO']   = match.post_match
    end

    env[@params_key] = (set_params || {}).merge parameters

    status, headers, body = route.app.call(env)

    if 'pass' == headers['X-Cascade']
      env['SCRIPT_NAME'] = script_name
      env['PATH_INFO']   = path_info
      env[@params_key]   = set_params
      next
    end

    return [status, headers, body]
  end

  return [404, {'X-Cascade' => 'pass'}, ['Not Found']]
end

Here, we are inside the Journey engine, the rails router. Its job is to find the correct route defined in config/routes.rb for the incoming request.

It used to be a gem, it’s now merged into Rails.

You can see its synopsis on github :

Too complex right now. :(

So I’ll skip this part, partly because you probably won’t understand, but mostly because I don’t.

The router will choose the appropriate controller and action for this request, and place in the Rack env a hash containing them.

It is located in env['action_dispatch.request.path_parameters'] and will containt something like this :

{'action' => 'my_action', 'controller' => 'my_controller'}

The next app to be called is an instance of ActionDispatch::Routing::RouteSet::Dispatcher, its call method looks like this :


PARAMETERS_KEY = 'action_dispatch.request.path_parameters'

#...

def call(env)
  params = env[PARAMETERS_KEY]

  # If any of the path parameters has an invalid encoding then
  # raise since it's likely to trigger errors further on.
  params.each do |key, value|
    next unless value.respond_to?(:valid_encoding?)

    unless value.valid_encoding?
      raise ActionController::BadRequest, "Invalid parameter: #{key} => #{value}"
    end
  end

  prepare_params!(params)

  # Just raise undefined constant errors if a controller was specified as default.
  unless controller = controller(params, @defaults.key?(:controller))
    return [404, {'X-Cascade' => 'pass'}, []]
  end

  dispatch(controller, params[:action], env)
end

First, a sanity check is conducted on the request params.

Rails then normalizes the params, and tries to get a controller from them. The goal is to retrieve the controller class from params[:controller]

If the controller is not found, Rails will return an empty response with the HTTP status code 404. The X-Cascade header is a Rack convention to signify that another middleware can go ahead and tries to render a webpage. In our case it will most likely be ActionDispatch::ShowExceptions, go ahead and read its code, it’s pretty straight forward.

Once we have the controller, the fun stuff begins :

dispatch(controller, params[:action], env)

Here is the dispatch method :

def dispatch(controller, action, env)
  controller.action(action).call(env)
end

At this point, the controller variable contains a controller class, HomeController for example.

Rails controllers are instances of ActionController::Base, which itself inherits from ActionController::Metal.

ActionController::Metal Is the simplest way of using controllers in rails, you just have to create a class that inherits from it, define actions, each action can set the response body with self.response_body = 'foo', and it’s ready to roll.

The cool thing is that the action method of ActionController::Metal returns a rack endpoint for the current controller and a given action : 

# Returns a Rack endpoint for the given action name.
def self.action(name, klass = ActionDispatch::Request)
  middleware_stack.build(name.to_s) do |env|
    new.dispatch(name, klass.new(env))
  end
end

This way, the dispatch method takes an action as its parameter and has to return our well-deserved array of status code, headers, and response body !

Let’s have a look at the dispatch method

def dispatch(name, request) #:nodoc:
  @_request = request
  @_env = request.env
  @_env['action_controller.instance'] = self
  process(name)
  to_a
end

def to_a #:nodoc:
  response ? response.to_a : [status, headers, response_body]
end

As you can guess, it sets some variables about the env and the request, but nothing fun happens.

to_a will return our rack array. status, headers, and response_body are just shortcuts for the instance variables @_status, @_headers and @_response_body

All the fun actually happens in AbstractController::Base#process :

def process(action, *args)
  @_action_name = action.to_s

  unless action_name = _find_action_name(@_action_name)
    raise ActionNotFound, "The action '#{action}' could not be found for #{self.class.name}"
  end

  @_response_body = nil

  process_action(action_name, *args)
end

def process_action(method_name, *args)
  send_action(method_name, *args)
end

# Actually call the method associated with the action. Override
# this method if you wish to change how action methods are called,
# not to add additional behavior around it. For example, you would
# override #send_action if you want to inject arguments into the
# method.
alias send_action send

First, Rails will check the action name is correct, set its response body to nil, and call the method corresponding to the action (that’s what process_action does basically)

Yes !

We did it ! We followed the request from the beginning to the final action call.

Wait a minute … This won’t render anything if we don’t explicitly use self.response_body =, how come my rails app works ?

That’s because our controllers are instances of ActionController::Base, that adds tons and tons of goodies over the simple and poor ActionController::Metal.

ActionController::Base includes lots of modules, I’ll just talk about two of them that are important in our case.

The first one is ActionController::Rendering (and its sibling AbstractController::Rendering) I won’t go over the details but their roles is to provide the render method that will search the proper templates and render them.

Have a look at AbstractController::Rendering

def render(*args, &block)
  options = _normalize_render(*args, &block)
  self.response_body = render_to_body(options)
  _process_format(rendered_format, options) if rendered_format
  self.response_body
end

As you can see, render will fill the response body of the request, so, if, in your action, you call render, the response body will be set and the page will show up when requested !

But, as you remember, rails is automagical, you are not forced to call render, you can implement an empty action and a template with a corresponding name, and Rails finds it for you.

That’s why I wanted to talk about ActionController::ImplicitRender, its code is dead simple :

module ActionController
  module ImplicitRender
    def send_action(method, *args)
      ret = super
      default_render unless performed?
      ret
    end

    def default_render(*args)
      render(*args)
    end

    def method_for_action(action_name)
      super || if template_exists?(action_name.to_s, _prefixes)
        "default_render"
      end
    end
  end
end

This module overrides the send_action method, used when calling the action, to call render if it has not been called already.

This way, even if you don’t call render, Rails catches up and calls it for you.

Conclusion #

Rails is automagical, but Rails is complex as well, part of it is because each part of Rails is supposed to be usable standalone. That’s nice, but it gives headaches when reading the code. You have to expect methods are overridden in one of the dozens of modules included in the class you’re reading.

 
455
Kudos
 
455
Kudos

Now read this

Diving in Rails - Exceptions handling

Warning : This article was written a while ago, my views may have changed and this may not be relevant anymore. Ruby code, just like any other code, can be subject to failure. Fortunately, when an error occurs inside your Rails... Continue →