Turning off email delivery for test users

For some reasons I happen to like a mail server configuration. Maybe it's just with Postfix, I didn't try anything else for ages. This is why the following task seemed interesting to me.

Let's say we have two sites: first a beta with latest features under testing and second is the production one. We don't want our e-mails to be sent on the beta except for some users who test the site. But we can't just remove other users because the good infrastructure is critical for the beta site (and it's fun to test with a recent copy of the production data). One good approach is to change the users' emails to point somewhere like username@example.com thus guaranteeing that those emails won't be delivered to the real person. But we can actually go further by telling Postfix to discard all messages for anyone@example.com without even trying to deliver them.

It's easy to do with header checks. So, here's a little snippet:

In /etc/postfix/main.cf:
header_checks = pcre:/etc/postfix/header_checks

In /etc/postfix/header_checks:
/^To:.*@example.com/ DISCARD

Avoiding simultaneous run for rake tasks

Sometimes we cannot guarantee that some periodic job will be finished before its next call. Especially if the environment is heavily loaded or the interval between job runs is small enough relative to a processed data set. This is why I decided to run some rake tasks wrapped in this helper:

def if_not_running(task, &block)
  lock_file = Rails.root + "/tmp/running-task-#{task.name}.lock"
  unless File.exists?(lock_file)
    FileUtils.touch(lock_file)
    begin
      yield
    ensure
      File.delete(lock_file)        
    end
  else
    puts "#{Time.now}: task #{task.name} is already running, skipping"
  end
end

I wonder if the helper covers all situations (and possible race conditions). Anyway it seems to work fine at the moment.

Usage:

namespace :test do
  task :test => :environment do |t|
    if_not_running(t) do
      puts Rails.env
      sleep 20
    end
  end
end

If your /etc/cron.d/* tasks don't seem to work

Then probably there is no newline at the end of file. I am working on a set of Capistrano tasks that set up Cron jobs, like this:

set :daily_tasks, %w(digest:daily)
set :weekly_tasks, %w(digest:weekly)
set :schedule_tasks, {"do_something" => "*/5 * * * *", "do_something_else" => "* */2 * * *"}

I generate shell scripts in /etc/cron.daily and /etc/cron.weekly for the daily and weekly tasks, and put cron records in /etc/cron.d/* files for the the custom schedules.

Here's the line responsible for a /etc/cron.d entry generation:

cron_record = "#{schedule} #{user} #{task_body(task)}"

Actually it didn't work. All files were in the place but they weren't picked up by the cron daemon because of no tailing newline in a cron_record. Not very obvious error for sure!

Here's the final version of the helper:

def add_cron_task(task, schedule, user, options = {})
  cron_record = "#{schedule} #{user} #{task_body(task)}\n"
  cron_file = "/etc/cron.d/#{task_name(task)}"
      
  process_and_push_config(cron_file, cron_record, options)
end

Pragmatic Thinking and Learning: a great book on learning and self-improving

Sometimes I think the software development is so nervous and stressful, with all the deadlines, loud co-workers and amount of information to process. So I ask myself: how all these guys out there handle this? Maybe it's just personal issues? What can we do to improve everyday work? How to acquire all the knowledge and stay sane?

This is why I really liked the new book from Pragmatic Programmer — "Pragmatic Thinking and Learning". Not only it gave me answers for those questions but uncovered such interesting topics as the human brain, the conscious and Dreyfus models of learning. Bravo, Andy Hunt!

It's still in Beta but looks pretty release-ready. My highest recommendation, especially for programmers, although it can be useful for anyone who wants to learn.

Handling deployment issues with Git and Capistrano

Well, that happens sometimes: a big bad piece of the early development changes suddenly leaks to the production. But if you have Capistrano and Git everything comes up pretty manageable.

On CookEatShare we use the next deployment scheme:

• the master branch of our git repository is the development mainline, it's accessible to all team members and used mostly to share intermediate work results; most of us do the development in the private branches, rebase those to the master branch, then merge into master and push/pull the changes;
• the staging branch used, well, for the staging. The master branch is being merged to the staging when there is something to demonstrate. Usually it's a working code with possible bugs here and there;
• the production branch is current cookeatshare.com state: staging is being merged there only if everything is fine and the current bugs were fixed. Capistrano deploys to the production from that branch.

We don't have a strict release management discipline. We're a startup and everyone is wearing multiple hats, so deployment can be done by anyone who has something to show. Sometimes it really screws up everything.

So, what to do if some very important patch came into the production and took some hairy guests with him. First, is, of course, cap deploy:rollback. After that we have our previous state restored but there are still 2 problems:

• Our important patch is somewhere deep inside of the current mainline and it's still not applied to the production;
• The production branch is no longer pristine, it has all the undesired commits.

So here's the thing:

cap deploy:rollback
git co production

Now take a look at git log and find the commit with patch (let's say it is P)

git diff P^1 P > our.patch # Keep patch at hand
git reset --hard PREV # PREV is pristine production state before unfortunate merging
git apply our.patch # We can do it in separate branch forked of current one if we need to rebase to other branches.

Actually this patch didn't apply at the first try, so I had to do some small changes, fortunately the patch was tiny, only 5 lines changed in 3 or 4 files.

git ci -a -m "Patch applied" # I use aliases for commit, checkout, status etc.
git push origin production --force # Note '--force' as it's not fast-forward push
cap deploy

Phew, thanks git for making all of this so easy (and there's maybe even easier way if not in the middle of the night with CEO on Skype).

Anyway, to correctly apply the patch to the production it should be forked from the production itself, merged back, and then rebased and propagated to other branches). But sometimes things are not that nice and maybe this will help you (if you use separate branches for the multistaging).

Finally, some sleep!