Mattenoble.com, from the maker of mattenoble.com

Continuous Integration is pretty awesome and there’s a few options out there in the Ruby world. Most of them are more than what I want for my personal projects though, so I decided to hack together my own. I mainly use RSpec and all I wanted was something that would run my spec every time I save it or it’s corresponding class/module. Enter, Speccer.

Speccer really has two parts, a TextMate command that replaces the normal Apple + S shortcut and the command line script that “receives” these saves. Set up your TextMate command like the image above, making sure to duplicate the settings exactly. Then copy in the code below.

  1   #!/usr/bin/ruby
  2   require 'fileutils'
  3   require 'ftools'
  4   
  5   logpath = File.expand_path( "~/log" )
  6   
  7   unless File.directory?( logpath )
  8       FileUtils.mkdir( logpath )
  9   end
 10  
 11  log = File.open( File.join( logpath, "/spec.log" ), "w" )
 12  log.write "#{ Time.now }\n"
 13  log.write "#{ ENV['TM_FILEPATH'] }\n"
 14  log.write ENV['TM_PROJECT_DIRECTORY']
 15  log.close

Basically all this does is creates _/log if it doesn’t exist, opens /log/spec.log and writes the time, file saved and the project directory to it, every time you hit Apple + S. This acts as a sort of Queue, but in reality the file only ever has one record in it at a time. This is how we let the script in the next part know about when we saved the file.

Create /usr/local/bin/speccer (or in whichever bin you want) and add the following:

  1   #!/usr/bin/ruby
  2   require 'ftools'
  3   
  4   log = File.open( File.expand_path( "~/log/spec.log"), "r" )
  5   
  6   loop do
  7   
  8       trap("INT") { |this| puts ""; exit this }
  9   
 10      last_time = @time
 11  
 12      log.seek(0)	
 13      record = log.read.split("\n")
 14  
 15      @time         = record.first
 16      @saved_file  = record[1]
 17      @proj_path  = record.last
 18  
 19      begin
 20  	   
 21          unless @saved_file.include?( "_spec.rb" )
 22              saved_file_array = @saved_file.split( "/" )
 23              lib_position       = saved_file_array.index( "lib" ) 
 24              spec_filename   = File.basename( @saved_file, ".rb" ) << "_spec.rb"
 25              spec_dir            = saved_file_array[ lib_position+1..saved_file_array.length-2 ].join( "/" )     
 26              @spec               = "#{@proj_path}/spec/#{spec_dir}/#{spec_filename}"
 27          else
 28              @spec = @saved_file
 29          end
 30  
 31          if last_time != @time
 32              system("clear")
 33              puts @time
 34              system("spec #{@spec} -c -f specdoc")
 35          end
 36  	
 37      rescue
 38      end
 39  
 40      sleep 1
 41  	
 42  end

This actual speccer script first opens the log file then just continuously checks to see if the Date is the same as the previous run through. If it is, it won’t do anything, but if it’s new it’ll run the spec. The if-statement in the middle just figures out which spec is associated with the Class/Module file being saved by breaking apart the file’s path and reconstructing it within the spec/ folder. Speccer assumes you set up your spec/ directory in the same structure as your lib/ folder and name your specs classname_spec.rb. Save the script and chmod 755 speccer and you’re set.

Now just run speccer then go writing your tests/specs and the file needed to make them pass. Every time you save one of them it’ll automatically run. Quick and easy continuous integration for the solo developer.

I constantly have ideas floating around in my head for cool and useful web-services or products, the problem is that they’d never be useful to me. Most of them would be great for homeowners, teachers, etc. but I’d probably never use them after launch. This is why I think a lot of web-based business fail so often, they aren’t their own users.

Chris Wanstrath gave a great keynote at Startup Riot where he told his tale of FamSpam and GitHub, which illustrates my point perfectly. He talks about a ton of other great things as well; check it out. In the end GitHub was what he loved to develop and use, which made it better for everyone. For an idea to be successful (in any form of the word) it needs to be brilliant. I don’t mean brilliant like having the most ridiculous algorithm in the world (although sometimes that does work) or having all PhD’s working for you. I mean a product that is 100% useful to it’s audience and continues to be a great experience for them.

Your high school teacher who told you that if you do what you love, you’ll never work a day in your life, was so right. I forked that quote and changed it: if you create what you love, you’ll always have a user. What I mean by this is, create something that is useful to yourself and in the end it’ll be a better product because of it. GitHub is what it is today because Chris and PJ use it themselves, everyday. You need to take it a step past just believing in what you’re doing, you need to use what you are doing.

Being your own user gives you a front row seat to see what could make it a better product. You can be selfish; develop features and fix bugs that bother you, as a user. You’ll be making it better for yourself, but at the same time all your other users. From a user-experience point of view, it’s easier to put yourself in the shoes of your customers, if you are one (obviously). You don’t have to pretend to be part of them or try to speak their language, you naturally will and other users will appreciate that.

method_missing is a piece of Ruby’s meta-programming arsenal that’s pretty powerful in my opinion. It allows you to call methods that don’t actually exist, but still do something useful with it.

I’m trying to keep this short and sweet, so imagine you have a blog like this one. You have Article, Comment and Category models and you want a way to find all Articles/Comments/Categories by calling find_all_( articles | comments | categories). So instead of create a find_all_* method for each resource, we’ll let method_missing do the work for us:

  1   class ApplicationController < ActionController::Base
  2       
  3       private
  4       def method_missing(method, *args)
  5           resource = method.to_s.split("_").last
  6           if method.to_s.match /^find_all_/
  7               resource.singularize.camelize.constantize.find(:all)
  8           end
  9       end
 10  
 11  end

This is a very simple example of what method_missing can accomplish but it’s a relevant one at least. method_missing takes two main arguments, the name of the method the user tried to call and the arguments passed through to that non-existent method.

If you follow a find_all_( model ) naming convention the last word will always be the Model. In my example, first thing we do is set resource to that String representation of the Model (the last word of the find_all_* method). Next, we make sure the call is actually to a find_all_* method. The long “-ize” method chain makes the String singular ( “comments”.singularize #=> “comment” ), uppercases the first letter ( “comment”.camelize #=> “Comment” ), constantizes it ( “Comment”.constantize #=> Comment ) and finally calls a normal old find(:all) on it and returns the result. Now you can do this in your controllers:

  1   # CommentsController.rb
  2   @comments = find_all_comments
  3   
  4   # ArticlesController.rb
  5   @articles = find_all_articles
  6   
  7   # CategoriesController.rb
  8   @categories = find_all_categories

My example only shows one “magic method” and is only a bit more readable, but you could extend it to check for find_recent_ or maybe find_10_ pretty easily.

I’ve been thinking a lot lately about how I can utilize the Agile way doing things even though I’m typically the sole designer or developer for most of my projects. I think one of the reasons the Agile movement has become so popular is because it really forces a team to be disciplined about only spending time on things that really bring value to the project. So what do you do when you aren’t part of a team.

I think when you get down to it, being an Agile solo developer isn’t all that different than being on an Agile team. Obviously you don’t have the ability to pair-program or have stand-up meetings (which may belong more in the Scrum category anyway) but what really makes Agile so powerful is still in your reach. User stories, class cards, refactoring, constant communication with the client and TDD/BDD are all easily implemented by a solo developer.

I’m curious about how some of you manage to follow the Agile path while being on your own? I’m really interested in some real life example of how you would utilize these techniques by yourself to create better software.

Acts_as_solr, UltraSphinx, acts_as_ferret, acts_as_xapian; there definitely isn’t a shortage of full text search options when it comes to Rails projects. However, the number of solutions suitable for a shared hosting setup is somewhat limited. Anything that is server or daemon based is most likely out the window, so basically we are left with acts_as_xapian.

Xapian has been around for a while and has a pretty interesting history behind it. It’s fast, feature rich and apparently scales very well.

This guide is very heavy on the code and command examples and it’s pretty long so grab a fresh cup of coffee and dive in.

1. The Server Side

SSH into your server. I install custom libraries into ~/opt and the tarballs and source files into ~/opt/src. If your server doesn’t have wget installed (unlikely), use curl. The commands below are originally from Kevin Colyar.

  1   # Downloading and untaring the source files
  2   cd ~/opt/src
  3   
  4   wget http://oligarchy.co.uk/xapian/1.0.10/xapian-core-1.0.10.tar.gz
  5   wget http://oligarchy.co.uk/xapian/1.0.10/xapian-bindings-1.0.10.tar.gz
  6   
  7   tar zxvf xapian-core-1.0.10.tar.gz
  8   tar zxvf xapian-bindings-1.0.10.tar.gz
  9   
 10  # Installing the Xapian-Core library
 11  cd xapian-core-1.0.10
 12  ./configure --prefix=$HOME/opt
 13  make
 14  make install
 15  
 16  # This is where our Ruby bindings will live
 17  mkdir ~/opt/ruby_modules
 18  
 19  # Installing the bindings to the above directory
 20  cd ../xapian-bindings-1.0.10
 21  ./configure --prefix=$HOME/opt RUBY_LIB=$HOME/opt/ruby_modules RUBY_LIB_ARCH=$HOME/opt/ruby_modules
 22  make
 23  make install

2. Telling Rails About the Bindings

The bindings are now in ~/opt/ruby_modules, but Rails has no idea that directory exists. To tell Rails about it put this in your environment.rb file.

  1   if RAILS_ENV == 'production'
  2       config.load_paths << "#{ENV['HOME']}/opt/ruby_modules"
  3   end

3. acts_as_xapian Rails Plugin

Next thing is to install the acts_as_xapian plugin and generate the migration for it.

  1   script/plugin install git://github.com/frabcus/acts_as_xapian.git
  2   script/generate acts_as_xapian
  3   rake db:migrate

To test your setup from the command line (SSH’ed into your server), first rebuild the index and then perform a search:

  1   rake xapian:rebuild_index models="ModelName1 ModelName2"
  2   rake xapian:query models="ModelName" query="search string"

4. The Actual Implementation

In whatever models you want to be searchable, add the following. The title and body are just the attributes you want included in the search.

  1   class Article < ActiveRecord::Base
  2       acts_as_xapian :texts => [:title, :body]
  3   end

Now you need a way to get search results in your app. The method below can go in whatever Controller you want. It’s just like any other Rails action.

  1   def search
  2       if params[:search] && params[:search][:words] != ""
  3           @xap_search = ActsAsXapian::Search.new([Article], params[:search][:words].to_s, { :limit => 100 })
  4           @xap_articles = @xap_search.results.collect { |r| r[:model] }
  5       end
  6   end

The above code is checking to make sure the results page wasn’t just visited and a search form (params[:search]) was actually submitted. Also it makes sure the user actually typed something in the field. If they left the search field blank and you try to search for null it will spit out an error. The @xap_search variable is an ActsAsXapian::Search object that gets generated based on the query. [Article] is an array of the models to search in, the second parameter is the search phrase, and the third is a hash of options, in this case limiting it to 100 results. From there we can return each search result as an ActiveRecord model using the last line. Then just loop over @xap_articles and display them in your view like any other Model.

5. Automatic Updates With after_filter

Making your app update the index automatically every time you create, update or delete an Article (or whatever models you are making searchable) is actually super-easy. Just add the following into it’s corresponding Controller:

  1   after_filter :update_xapian_index, :only => [:create, :update, :destroy]
  2   
  3   private
  4   def update_xapian_index
  5       Thread.new do
  6           system("rake xapian:update_index")
  7       end
  8   end

I put it in the Controller instead of the Model because it doesn’t directly pertain to a single Model, it just interacts with them. It pops up a new process, runs the rake task and then dies. Easy enough, right?

Bonus Points: Capistrano

If you’re deploying your app with Capistrano you’ll need to rebuild the index every time you deploy. It gets pretty annoying doing it every time by hand, especially if you’re deploying a lot (like you should be). The solution is pretty straight forward, just add this task to your deploy.rb file and replace the models with the appropriate ones for your app. It will run right before the server gets restarted each time you deploy.

  1   desc "Rebuilds and updates the Xapian search index"
  2   task :before_restart, :role => :app do
  3       run "cd #{current_path};
  4            rake xapian:rebuild_index models='Article' RAILS_ENV=production;
  5            rake xapian:update_index models='Article' RAILS_ENV=production"
  6   end

So there it is, one huge guide to Rails full text search with Xapian on a shared host. Hopefully it helps someone who was in the same position as me. If you’ve done the same thing before and there’s an easier way to do part of it, by all means let me know.