I left my room last night headed for a night of gweeping in the free-WiFi lobby at my hotel (compared to $10/night in the room). In the elevator, I ran into Jason Hoffman from Joyent (Spotstory is a customer) and was promptly invited to a Joyent event at the Lucky Lab Brew Pub.

I can’t comment on the stronger stuff, but the root beer at the Lucky Lab was wicked good (sorry, I need to keep my provincial New England self in check)!

Thanks so much to Jason, David, Kristie, and all the other folks from Joyent for hosting a great event.

This morning, my karma has been balanced, as Jason is using my HDMI-to-VGA converter for his Scaling a Rails Application from the Bottom Up Thursday morning tutorial.

Right now, I’m in Portland, Oregon getting ready for RailsConf 2007. Even though I’ve been here for less than an hour, I’ve already had an encounter with Internet celebrities!

If you follow the Ruby on Rails community (and who doesn’t), you have probably seen the Ruby on Rails vs. Java commercial by the RailsEnvy.com guys. We STOOD TOGETHER waiting for our luggage! I was in complete, geeky fan mode.

I introduced myself and mentioned that I work on both Spotstory and a.placebetween.us. They had heard of our site! Does this make me cool? It makes me a geek.

A wicked geek.

If you have been following this weblog for a while, you have probably figured out that Spotstory is a Ruby on Rails application.

Over the past few days, I have been working to upgrade Spotstory from Rails 1.1.6 to Rails 1.2.3.

There is very little comprehensive information about some of the problems encountered when upgrading a Rails application from the 1.1.6 release to 1.2 (specifically, Rails 1.2.3).

Once You Freeze, Step Cautiously

Upgrading your Rails application is easy! Update, deploy, then go enjoy your apple pie! Right? Wrong.

% rake rake rails:freeze:edge TAG=rel_1-2-3

So far, so good. Let’s see if anything runs.

rake rails:update --trace
rake aborted!
uninitialized constant ActionController
/usr/local/gems/rake-0.7.1/lib/rake.rb:1948:in `const_missing'
config/../config/environment.rb:32

Why did this happen? Did the Rails installation get corrupted during the upgrade? No. Far, far simpler. On line 31 of config/environment.rb, I had a raw reference to ActionController. Due to major dependency changes in Rails 1.2, this no longer works.

# Rails 1.1.6 technique for setting session expiry:
ActionController::Base.session_options[:session_expires] = Time.mktime(2037)

In Rails 1.2, it appears that this sort of configuration can not appear within the Initializer block. After moving session configuration out of the Initializer block, things started running.

Deprecations & Warnings

I decided that every deprecation message would be squashed and every warning removed. I worked on most of the warnings before even trying to get all our tests to pass; they were just too noisy for my aesthetics.

DEPRECATION WARNING: The :dependent => true option is deprecated and
will be removed from Rails 2.0.  Please use :dependent => :destroy
instead.  See http://www.rubyonrails.org/deprecation for details.
See http://www.rubyonrails.org/deprecation for details. (called from
has_many at vendor/rails/activerecord/lib/active_record/associations.rb:558)

(Yes, that link is actually printed twice!)

These deprecation messages are great — they tell me what has changed and how to improve my code, but they reference source files and line numbers within the Rails core. This is not helpful. I was able to find some instances of deprecated usage by recursively grepping our source (from within Emacs, which makes results electric). Very quickly, I turned to a heavier hammer.

ActiveSupport::Deprecation.debug = true

Toggling this debug variable exposes the full callstack which caused the deprecation message.

Here are the changes I needed to make in order to eliminate all our deprecation messages:

1. Some model helpers have gone away or have changed in call syntax. Rewrite model methods in the following way:
   

   Rails 1.1.6	Rails 1.2.3
   Model.find_first	Model.find(:first)
   Model.find_all	Model.find(:all)
   Model.type	Model.class.to_s
   Model.count(conditions)	Model.count(:all, :conditions => conditions)

2. Has many associations no longer provide the has_ASSOCIATIONs? and ASSOCIATION_count methods. Rewrite:
   

   Rails 1.1.6	Rails 1.2.3
   Model.has_ASSOCIATIONs?	! Model.ASSOCIATIONs.empty?
   Model.ASSOCIATION_count	Model.ASSOCIATIONs.size

3. Rewrite controller/view instance variable references:
   @params, @session, @request, @response all become params, session, request, response.
4. Move to the new form generators.

   form_tag ... end_form_tag

   becomes:

   form_tag do ... end

   The same is true for form_remote_tag references. In addition, the form methods no longer need output-generating RHTML rules (remove the “=”).

5. The link_to helper prefers a different way for specifying that a link will generate a POST operation.

   link_to ... :post => true

   becomes

   link_to ... :method => :post

6. Association dependencies now support more granularity, which means that the old-style:

   has_many ... :dependent => true

   becomes:

   has_many ... :dependent => :destroy

7. Add parentheses and remove whitespace between parens and method names to prevent ambiguity. Not clear why these issues didn’t cause issues in Rails 1.1.6.

Fixing Broken Tests

With our application beaten into quiet submission, it was time to turn to our tests. A good majority of our tests still worked, but a few changes were needed to get everything passing.

1. Content-Type headers now include charset information; make test helpers resilient to the presence of charset.
2. assert_tag with a string value for the :content option now performs a strict match against the content for any sub-tag. As a result, some :content options were converted to regular expressions.

   Determining when a regular expression is necessary is not obvious, especially when the match text is embedded within another markup tag.

At this point, all of our tests passed.

Dependencies

Unfortunately, we weren’t done. The development server would unexpectedly give errors like:

User expected, got User

In addition, our User model would get missing_method errors for methods which clearly exist. The problem is an interaction between the Rails 1.2 dependency rewrite and plugins which have dependencies upon application models. Ramble On discusses a similar dependency issue.

We use a number of plugins in our application. One of these is a customized version of acts_as_commentable. This plugin defines the Comment model and expects a User model to be defined by your application.

In development mode, plugin models are not reloaded. The dependency infrastructure unloads the User model, but doesn’t realize that there are still live references from the plugin-defined Comment model. Subsequent references to a comment’s user information will generate an AssociationTypeMismatch exception giving the cryptic “expected User, got User” message. Because this is a reload problem, errors don’t happen on initial load. The errors start the second time a comment-displaying page is loaded.

Web searching didn’t reveal much at first. Some people recommended explicitly requiring the User model in application code. Others recommended replacing references to a User object with references to its id. Another solution called for explicit :class_name in the association declarations. Some of it worked some of the time; none of it worked consistently.

I started looking through the Rails dependency code and found that Nicholas Seckar (ulysses on IRC and in SVN) was responsible for much of the rewrite. After explaining my missing_method errors in an email, he replied:

See http://api.rubyonrails.org/files/vendor/rails/railties/CHANGELOG.html
Look under the changes for 1.2.0, 4th one down.

This references a late change to Rails 1.2 by Rick Olson:

Ensure plugins are in the Dependencies.load_once_paths collection by default. [Rick] If you really want your plugins to reload, add this to the very top of init.rb:

Dependencies.load_once_paths.delete(lib_path)

Ok. I need to reload my plugins. Bleh. Further, this has to happen in environment.rb, not in environments/development.rb. Bleh.

# Array of plugins with Application model dependencies.
reloadable_plugins = ["acts_as_commentable"]
# Force these plugins to reload, avoiding stale object references.
reloadable_plugins.each do |plugin_name|
  reloadable_path = RAILS_ROOT + "/vendor/plugins/#{plugin_name}/lib"
  Dependencies.load_once_paths.delete(reloadable_path)
end

None of this forced plugin reloading feels right. Rails should just figure it out, yes? More mail to Nicholas, leading to his response:

The problem is related to associations — the class object is cached and thus the class is never garbage collected, even though it is ‘removed.’ This causes all sorts of problems…

Since it only happens in development mode, it really doesn’t matter.

He’s partially right. It doesn’t affect production applications, but it really does affect peace-of-mind. I’m sure lots of people are freaked out when they bring up their upgraded app in development mode and find all sorts of things unexpectedly broken.

Update: I was wrong about the session configuration changes in Rails 1.2. I have updated this post accordingly.

We had a lot of fun presenting at the Boston Ruby Group last night.  We’d like to thank Tom Dyer and everyone else involved with organizing, setting up, providing space, and providing pizza.

Once again, the meeting was well attended: folks were standing along the walls and sitting on the floor.  This month’s program was more Rails-oriented than last month’s, but few people seemed to mind that.

Eric Mill from thoughtbot gave a talk on REST and ActiveResource.  The highlight was definitely Jester, a JavaScript REST implementation  developed at thoughtbot.  It’s modeled after ActiveResource and, boy howdy, does it look neat!

Jeremy Durham did a presentation on Memcache.  Jeremy has spoken at both of the meetings I’ve attended, and on both occasions he’s provided a lot of practical, meat-and-potatos info you can use.

You can find Aron’s presentation here, and mine here.

The group is always looking for speakers, so please volunteer!  If you’re in the Boston area and working with Ruby/Rails, it’s a great way to learn about the subject and to meet folks in the community. 

It is quite easy to extend your Ruby on Rails test infrastructure so every document is checked for validity.

1. Install the plugin assert_valid_markup.
2. Install my test_validation_helper.rb in your test directory.
3. Require the validation helper from your test/test_helper.rb:

   require File.expand_path(File.dirname(__FILE__) +
       "/test/test_validation_helper.rb")

After those three simple steps, your HTML and RSS documents are now being validated each time you run tests.

I am giving a brief presentation on this topic at the April 10, 2007 Boston Ruby Group meeting.

You can flip through a PDF of my slides if you would like more details.

Update: In addition to the PDF, my presentation is now hosted on slideshare and shown below. If you are reading this post in an aggregrator, you will need to click through for the show (or just use the PDF version).

Aron and I will be presenting at the upcoming meeting of the Boston Ruby Group on on Tuesday, April 10th, 7:00 PM at One Broadway, Cambridge, MA.

Aron will talk about the regression testing infrastructure we use to validate all of our markup.  I’ll briefly go over another component of our Rails testing environment that we call Isolated Fixtures.

More details can be found here.  Come on by and say hello (not to mention last time there was pizza!)

I’d like to describe and provide code for what I’m calling “isolated fixtures”. Isolated fixtures allow you to have an independent and separate set of fixtures for Test::Unit::TestCase in your Ruby on Rails project.

I’d already run into the situation a couple of times where adding info for my new test into existing fixtures broke existing tests, either due to assumptions of about the fixture data, or because multiple tests “borrowed” fixtures from previous test.. According to some other pieces I’ve read on the net, the situation would only get worse as our test suite grew (and ours does grow). So, instead of becoming adept at the sort of limbo-dancing required to finesse new tests into the testsuite, I decided to come up with a way to let each set of tests have their own fixtures.

Now, I don’t know if this is considered a hack, or another demonstration of the flexibility and power that is Ruby on Rails. I’m sure you’ll all let me know.

This works for me with Rails 1.1.6. I have no idea if it will work in 2.0. (2.0 might even have this built in for all I know!)

Usage

Let’s assume you have a test class called AccountControllerTest. In this test where you normally use fixture instead user isolated_fixture instead:

  isolated_fixture :customers, :accounts

Instead of putting the fixture data into test/fixtures it will create a subdirectory there based on the name of the test class. In this case the path of the directory and the fixture data will be:

  test/fixtures/account_controller/accounts.yml
  test/fixtures/account_controller/customers.yml

Code

Here is the code. Put it in your test/test_helper.rb, or do what I did and put it in test/test_helper/fixtures.rb and require it into test_helper.rb.

class Test::Unit::TestCase

  ## Each subclass needs its own copy 'fixture_path' so replace
  ## the existing single class variable.
  remove_class_variable :@@fixture_path
  class << self
    remove_method :fixture_path
    remove_method :fixture_path=
  end

  class_inheritable_accessor :fixture_path
  self.fixture_path = RAILS_ROOT + "/test/fixtures/"

  ## Add specialized fixture class methods and alias them.
  def self.standard_fixtures(*table_names)
    self.fixture_path = RAILS_ROOT + "/test/fixtures/"
    self.fixtures_without_fixture_path(table_names)
  end

  def self.isolated_fixtures(*table_names)
    fixture_subdir = self.to_s.ends_with?("Test") ? self.to_s[0..-5].underscore : self.to_s.underscore
    self.fixture_path = RAILS_ROOT + "/test/fixtures/#{fixture_subdir}/"
    self.fixtures_without_fixture_path(table_names)
  end

  class << self
    alias_method :fixtures_without_fixture_path, :fixtures
    alias_method :fixtures, :standard_fixtures
    alias_method :original_method_added, :method_added
  end

  def setup_with_fixtures_and_with_fixture_path
    setup_with_fixtures_and_without_fixture_path
  end

  ## Provide a default 'setup' method.
  alias_method :setup_with_fixtures_and_without_fixture_path, :setup_with_fixtures
  alias_method :setup, :setup_with_fixtures_and_with_fixture_path

  ## Handle the case where the user provides a 'setup' method for the tests.
  def self.method_added(method)
    case method.to_s
    when 'setup'
      unless method_defined?(:setup_without_fixtures_and_without_fixture_path)
        alias_method :setup_without_fixtures_and_without_fixture_path, :setup
        define_method(:setup) do
          setup_with_fixtures_and_with_fixture_path
          setup_without_fixtures_and_without_fixture_path
        end
      end
    else
      self.original_method_added(method)
    end
  end

end

Drop a line and let me know if you found this useful, or if you found a bug. I’ll post any updates.

Update

Here is a slide presentation covering the same material.

We are using JSON to return snippets of data from the server back to the requesting web browser. This post will explain the various steps needed to get this all working. Note: I am using Rails 1.1.6; some of this has improved in the beta releases for Rails 2.0.

First, we need an action. In my case, I’m doing a bit of data cleaning with a Ruby library, but you could use anything. The action sends a hash back to the client browser as a JSON object.

class WorkController < ApplicationController
  def do_work
    # Populate the hash however you choose, or use any
    # object that supports to_json
    result = {
      :data => @params[:data],
      :message => “the data is #{@params[:data]}”
    }
    render_json(result.to_json)
  end
end

The render_json call doesn’t exist in Rails 1.1.6. In Rails 2.0, there will be a way to directly render for JSON. Until then, add the following method to your application.rb.

class ApplicationController < ActionController::Base
  Mime::JSON = Mime::Type.new "application/json", :json, %w( text/x-json )
  Mime::LOOKUP["application/json"] = Mime::JSON
  Mime::LOOKUP["text/x-json"] = Mime::JSON
  def render_json(json, callback = nil, status = nil) #:nodoc:
    json = "#{callback}(#{json})" unless callback.blank?

    headers["Content-Type"] = Mime::JSON
    # Prototype.js will automatically evaluate X-JSON headers.
    headers["X-JSON"] = json
    #response.content_type = Mime::JSON
    render_text(json, status)
  end
end

Note that we are including the JSON twice in our rendered document. We include the JSON text in the document body but also in a X-JSON header. The X-JSON header is important, as it allows the Prototype.js JavaScript library to automatically build an object from the JSON text.

All the server-side work is now done. On the client, you will need to request and then process the result. We accomplish this with an asynchronous AJAX call:

/* Pass in some content id that will be updated with the result
    obtained from the server */
function setup_work(contentid) {
  /* Define a handler to process the result data.
      The Ajax.Request call passes the XmlHttpRequest object
      and an object created by evaluating the JSON data */
  function handler(request,object) {
    if (object) { // null if it couldn't be evaluated
      document.getElementById(contentid).innerHTML = object.message;
    }
  }

  new Ajax.Request("/work/do_work", {
    parameters: "data=rails",
    asynchronous:true,
    onComplete:handler // Call our handler when the request completes.
  });
}

For more help, you will want to reference the Prototype.js developer notes. The Ajax.Request method is well-documented along with its options and handlers.

We are using Josh Martin’s acts_as_favorite plugin (also here).

I have made a few acts_as_favorite bug fixes (contextual diff file). These properly set the user_id in your favorites table and update the favorites associations as you add/delete favorites.

Here is how we tell the user class that it will have favorites:

class User < ActiveRecord::Base
  acts_as_favorite_user
end

Here is how we tell a different model that it can be marked as a favorite:

class Thing < ActiveRecord::Base
  acts_as_favorite
end

In our controller, we allow a user to mark a favorite thing. There is code elsewhere which makes sure that the user is logged in.

class ThingController < ApplicationController
  def add_favorite
    @thing = Thing.find(params[:id])
    @user = User.find(session[:user_id])
    @user.has_favorite(@thing)
    redirect_to :action => ’show’, :id => @thing
  end

  def remove_favorite
    @thing = Thing.find(params[:id])
    @user = User.find(session[:user_id])
    @user.has_no_favorite(@thing)
    redirect_to :action => ’show’, :id => @thing
  end
end

That’s all there is to it. You now have users who can mark objects as favorites.

Update: It appears that the official repository for acts_as_favorite is not reliable. I have archived my snapshot of acts_as_favorite. This version includes the patch listed above.

I’m just starting to use Alex Wolfe’s paginating plugin. It works for my simple examples, but I’ve run into this case:

Image.find(:all,
    :joins=>"left join votes on
        votes.voteable_id = images.id and
        votes.voteable_type = 'Image'",
    :page=>{:first=>1, :size=>2, :current=>1},
    :select=>"images.*, ifnull(sum(case
            when votes.vote=TRUE then 1
            when votes.vote=FALSE then -1 end),0) as vote_tally",
    :group=>"images.id",
    :order=>"vote_tally DESC"})

I’m using acts_as_voteable to implement simple voting. This query is determining the score for each image (votes for - votes against). I’m not in love with needing to dynamically calculate this information, but that’s the default configuration for the plugin.

At any rate, the generated count SQL query is:

SELECT count(images.id, ifnull(sum(case when votes.vote=TRUE then 1 when votes.vote=FALSE then -1 end),0) as vote_tally) AS count_images_id_ifnull_sum_case_when_votes_vote_true_then_1_when_votes_vote_false_then_1_end_0_as_vote_tally FROM images left join votes on votes.voteable_id = images.id and votes.voteable_type = ‘Image’

Sorta ugly.

I’m working around this by altering the :page option, since I can easily determine the total number of images:

    :page => { ... :count => Image.count, ... }