Under the hood: ActiveRecord::Base.find, Part 3

Posted by Jamis on November 30, 2006 @ 10:39 PM

Dynamic finders are one of my favorite bits of syntactic sugar in Rails. Suppose you have a form that people can use to log into your application. It submits the username and a password, and you then need to take those data and find the corresponding user.

Although there are several ways to do this, the best way is to use a dynamic finder, like so:

1
2
3
4
5
6
user = User.find_by_username_and_password(params[:username], params[:password])
if user.nil?
  # no such user, let them retry
else
  # log them in
end

In this next installment of “Under the Hood: ActiveRecord::Base.find”, we’ll explore the implementation of dynamic finders.

The magic, as with most DSL’s, occurs in a method_missing hook. However, you’ll find that there are actually two method_missing hooks on ActiveRecord::Base. The first is for the class, the second is for instances of the class. For dynamic finders, the class-level hook is used, since we are sending the find_by_username_and_password message to the class, not to an instance of the class.

A quick review for those unfamiliar with method_missing: any time you invoke a method on an object, and the object does not have (or “respond to”) that method, Ruby will invoke a method called method_missing on that object. The default method_missing implementation just raises a NoMethodError exception, but you can implement your own method_missing method to do some clever tricks, like ActiveRecord does to handle dynamic finders.

As of revision 5593, the class-level method_missing hook is on line 1191 of base.rb. It starts out something like this:

1
2
3
4
5
6
7
8
9
10
def method_missing(method_id, *arguments)
  if match = /find_(all_by|by)_([_a-zA-Z]\w*)/.match(method_id.to_s)
    finder, deprecated_finder = determine_finder(match), determine_deprecated_finder(match)

    attribute_names = extract_attribute_names_from_match(match)
    super unless all_attributes_exists?(attribute_names)

    attributes = construct_attributes_from_arguments(attribute_names, arguments)

    #...

Any time an ActiveRecord class receives message (method) it does not recognize, it first tries to match that method against the regular expression you see above. Note the two captures in that regex: the first captures the “all_by” or “by” text, and the second captures the “and“-delimited list of attributes to search by.

If the method name matches, we next try to determine which finder to use. The finder says whether we should find all matches (“find_all_by_xyz”) or only the first match (“find_by_xyz”). I’m going to ignore that whole “deprecated” finder part, since those will go away eventually and will no longer be relevant. But let’s look at the definition for determine_finder:

1
2
3
def determine_finder(match)
  match.captures.first == 'all_by' ? :find_every : :find_initial
end

It just looks at the first capture in the match, and returns the name of the method we need to call to satisfy the query. We’ll use that later.

Next, the method_missing hook tries to determine the names of the attributes that we are going to query by. It does that by passing the match to extract_attribute_names_from_match, which looks like this:

1
2
3
def extract_attribute_names_from_match(match)
  match.captures.last.split('_and_')
end

Basically, it just takes the last capture in the match, and splits it on the text “and”. This returns an array of strings, naming the attributes in our query.

We don’t want to allow just any attribute names, though. We need to make sure they really exist in the table in question, and we do that by calling the all_attributes_exist? method, passing in that array we just built:

1
2
3
def all_attributes_exists?(attribute_names)
  attribute_names.all? { |name| column_methods_hash.include?(name.to_sym) }
end

The Array#all? method returns true if the associated block returns true for every element of the array. In this case, we’re testing to see if every element of the array is included in the column_methods_hash.

If that’s all good, we then create a hash that maps attribute names to attribute values, merging the list of attribute names with the list of arguments to the method. This is done via construct_attributes_from_arguments:

1
2
3
4
5
def construct_attributes_from_arguments(attribute_names, arguments)
  attributes = {}
  attribute_names.each_with_index { |name, idx| attributes[name] = arguments[idx] }
  attributes
end

This looks like it could be done with Array#zip to save a few lines, but alas, it cannot. The reason is that the arguments list may have one element more than the attributes list—an options hash, such as you could pass to ActiveRecord#find. (Such little details inevitably complicate what you would think should be a trivial solution.)

So. At this point we now have a hash that maps attribute names (username and password, in our case) to values (params[:username] and params[:password]).

Before we go querying the database, we need to consider any extra options that the caller may have passed in. Those extra options will be in the argument slot immediately following the one corresponding to the last named attribute:

1
2
3
4
5
6
7
8
9
10
case extra_options = arguments[attribute_names.size]
  when nil
    # ...

  when Hash
    # ...

  else
    # ...
end

In our case, no extra options were passed in, so we’ll take the nil branch. If an option hash was passed in, the second branch would be taken, and if anything else was given, the last branch (using deprecated finders) would be taken. We get off easy with our example, though:

1
2
3
4
when nil
  options = { :conditions => attributes }
  set_readonly_option!(options)
  ActiveSupport::Deprecation.silence { send(finder, options) }

We build the options hash trivially, just setting the :conditions key to the attributes hash we constructed, setting the readonly option based on the current scope settings, and then invoking the appropriate finder (either find_initial or find_every, depending on what the name of the method was—in our specific case, it will be find_initial). Don’t ask me why Deprecation.silence is being used there…I have no clue.

The finder call will then come back with the record (or records) that matched our query. In our example at the start of this article, either the user matching the given username and password will be returned, or nil will be returned, and we can test and branch accordingly.

Pretty straight-forward! Alas, this is about the last of the “simple” aspects of ActiveRecord#Base.find that we can investigate. Future articles will delve into more complex aspects, like eager-loading and scoping.

Lastly and leastly, did you find this article helpful? These “under the hood” posts take a fair bit of time and effort to compose, and while I do enjoy doing it, any encouragement at all is appreciated. You are (of course) never under any obligation to do so, but if you wish to, a few dollars via PayPal (to jamis@jamisbuck.org) would be wonderful. Thank-you!

Posted in Under the Hood

Comments

Have something to add? Click here to leave a comment.

01 Dec 2006

1. Grant Hutchins said...

Thanks for the continued insights!

Array#zip works for me even when the arrays don’t line up, tested in Ruby 1.8.5 at least:

b

=> [“first”, “last”]

c

=> [“grant”, “hutchins”, {:contrived_example=>true}]

b.zip©

=> [[“first”, “grant”], [“last”, “hutchins”]]

Hash[*(b.zip©).flatten]

=> {“last”=>”hutchins”, “first”=>”grant”}

2. Zack said...

Jamis,

Speaking of syntactic sugar don’t you think the construct_attributes_from_arguments() method screams for a returning clause…

def construct_attributes_from_arguments(attribute_names, arguments)
  returning attributes = {} do
    attribute_names.each_with_index { |name, idx| attributes[name] = arguments[idx] }
  end
end

Zack

3. Sandro Paganotti said...

Thanks for this article, i found it very interesting ! Just one question: why rails core team used method_missing instead of reflection to generate finders ?

4. Jamis said...

Sandro,

We can’t use reflecting to generate these dynamic finders, because they could be any combination of all attributes on the table, and that could be a very large number. Rather than waste cycles building thousands of methods that will (in all likelihood) never be called, ActiveRecord just waits until they are actually called and implements them on the fly via method_missing.

5. Jamis said...

Grant, thanks for pointing that out! I wasn’t aware that zip was safe to use with arrays with unequal lengths. That would cut that method down to a single line.

Zack, you’re right, it’s a prime candidate for Kernel#returning. Though, with Array#zip, the need for it is obviated.

6. Charlie said...

Thanks for the article! One follow up on the reflection question from Sandro. It would seem that you could very easily add the method to the class the first time it is invoked. This would make subsequent calls faster. How much faster I really don’t have any idea (might not be worth it).

Just a thought, Charlie

7. Jamis said...

It does make a slight difference in speed, but not, I think, enough to warren the extra machinery involved. If you are calling something like User.find_by_username_and_password over and over, frequently, you would probably do better to use find(:all, :conditions => {}) directly. Myself, I only use dynamic finders maybe once or (at most) twice per request. The bottleneck in most cases is the database, not Rails.

04 Dec 2006

8. Matt said...

Thanks for another great post Jamis! Your under the hood series are always very useful. It’s a nice treat that you include comments about why the decisions within the code were made the way they were, as well as showing the call paths.

05 Dec 2006

9. Martin Smith said...

Hi Jamis,

Very nice series.

From a purely selfish point of view could I suggest the next step in this series?

Model creation after query, especially how the columns get turned into attributes and the difference between the normal assessors and the before_type_cast versions.

Martin

10. Jamis said...

Thanks, Martin. I’ll add your suggestion to my list. No promises, you understand, but I’ll keep it in mind!