Capistrano 2.3.0

Posted by Jamis on May 02, 2008 @ 04:16 PM

On February 16 I set myself to travel a path that I hoped would eventually help me deal with the project overload I was feeling. Today I get to release Capistrano 2.3, which is very near to the end of that path!

gem install capistrano

Capistrano 2.3 is primarily significant in that it switches to the new Net::SSH v2 library, which is faster and slimmer than the older Net::SSH v1 library. It also pulls in Net::SFTP v2, and the new Net::SSH::Gateway and Net::SCP libraries.

In addition to that, cap 2.3 adds several exciting (if I do say so myself) new features, and a few bug fixes that had lingered for far too long.

FEATURE: The :copy strategy has been significantly improved. Instead of doing a blind checkout/export, archiving the result, and copying it over, you can specify that a local cached copy of your repository should be used. If the cached copy does not exist, it is created (via a checkout), otherwise it is simply updated (a much faster operation than a checkout, typically). This functions much like the :remote_cache strategy, but locally. To use this, simply set the :copy_cache variable to true:

1
2
set :deploy_via, :copy
set :copy_cache, true

By default, the cached copy will be in your machine’s temporary directory (/tmp, for example), but you can specify your own location by setting :copy_cache to the desired path:

1
2
set :deploy_via, :copy
set :copy_cache, "/u/caches/#{application}"

But wait! There’s more! Suppose you have certain files that you don’t want to deploy, like photoshop files or your .git directory. You can set the :copy_exclude variable to a file glob (or an array of globs):

1
2
3
set :deploy_via, :copy
set :copy_cache, true
set :copy_exclude, [".git", "materials"]

This should make your deploys faster than ever, especially using a tip I hope to share in the next day or two that uses this in conjunction with some vendor/rails symlinking.

FEATURE: Even though I strenuously believe it is a mistake to deploy anything that is not under source control, I’ve finally caved and added a dumb :none SCM module. It can be used to deploy a specific directory if used with the :copy strategy:

1
2
3
set :repository, "."
set :scm, :none
set :deploy_via, :copy

Again, there are very, very few cases when I think use of this technique is justified, but because I ran into one myself a month ago, I decided it was worth adding.

FEATURE: Support was added for “depend :remote, file” to test for the existence of a specific file:


depend :remote, "/etc/syslog.conf"

This is used in conjunction with the deploy:check task.

FEATURE: You can specify ssh_options per-server, now, simply by giving an :ssh_options key and corresponding hash with the server definition:


role "some.host", :ssh_options => { :keys => "/path/to/key" }

FEATURE: There are two new file transfer helpers, upload and download, which are much more powerful and resource-friendly than the old ‘put’ and ‘get’ helpers. You can use upload and download to transfer single files, or entire directory trees:

1
2
upload "/local/file", "/remote/file"
download "/remote/file", "/local/file"

This will transfer to or from all active servers, which is particular tricky when using the download helper, since it will download the file simultaneously from all active hosts. To make this work, you need to make sure each is downloaded to a different location:

1
2
Dir.mkdir("destination")
download "/remote/file", "destination/file-$CAPISTRANO:HOST$"

The above will download the file from each host to a file on the local host, where the local file includes the name of the source host. Tricky!

Also, you can now specify that you want to upload or download via SCP instead of SFTP:

1
2
upload "local", "remote", :via => :scp
download "remote", "local", :via => :scp

The default is :sftp.

The less exciting (but still mildly titillating) things are the bug fixes and pleasure-enhancing behavioral changes:

  • The default run options are now mixed into the command options when executing a command from the capistrano shell.
  • The git SCM now uses git-ls-remote instead of git-rev-parse to resolve the revision of the checkout.
  • A trivial export method has been added to the git SCM.
  • The git SCM will include tags when it fetches.`
  • The sudo() helper now works nicely with complex and chained commands.
  • The deploy:setup task will now use sudo if :use_sudo is true (the default)

So, have at it! Remember to report bugs to http://capistrano.lighthouseapp.com. And patches are always welcome via git—just fork the capistrano repository at git://github.com/jamis/capistrano.git (thanks GitHub!).

Cheers!

Posted in Announcements

Comments

Have something to add? Click here to leave a comment.

02 May 2008

1. Chris said...

Sweet!

Thanks so much for your hard work, Jamis.

It is very much appreciated!

2. Jim Morris said...

Thanks Jamis.

Does this fix the long standing hangs when using gateway?

Also I currently get …

ERROR: While executing gem … (Gem::RemoteFetcher::FetchError) Gem::RemoteFetcher::FetchError: bad response Not Found 404 reading http://gems.rubyforge.org/gems/capistrano-2.3.0.gem

I presume we need to wait a while to grab it?

Thanks

3. Jamis said...

@Jim, it should fix the hang problem, but then, upgrading to Net::SSH 1.1.4 should fix it, too, if you don’t particularly want to upgrade to Net::SSH 2 yet. Regarding the 404 error, yeah, the mirrors may not all have updated yet. It works for some and not for others—just retry a few times until you get a mirror that’s updated.

4. Jim Morris said...

It does indeed fix the gateway thanks (do I still need set :synchronous_connect, true?)

When I execute the following in my capfile…

sudo “svc -u /service/treasurehunt”

I get…
  • [err :: l_app2] command finished
  • executing “svc -u /service/treasurehunt” servers: “l_app2” executing command

It does seem to execute so what does the * [err: mean?

Thanks Jim

5. Jamis said...

@Jim, the :synchronous_connect variable is completely ignored in cap 2.3, so you can remove it. It has no effect on anything, anymore. As for the “err”, it just means that some text was received from the server on the STDERR stream. “err” is the tag for stderr, and “out” is the tag for stdout.

6. Jim Morris said...

oops it seems to have munged my output above

  • executing “svc -d /service/treasurehunt 2>&1” servers: “l_app2” executing command
    • [err :: l_app2] command finished

Trying again

7. anonymous said...

What about pluggable ssh? Why must we use net:ssh for cap when we use openssh for everything else?

03 May 2008

8. Jamis said...

@anonymous, pluggable anything makes things 10x more complicated and introduces a lot of edge cases. I’d rather not go there. That said, if you want to take a stab at a patch, I’d consider it if it doesn’t overcomplicate things. If you want to use openssh, though, there’s already vlad, which does.

9. gregf said...

Really like the new upload and download options. Going to help out with the site I’m working on right now. I was managing it before, but this is much cleaner. nice work.

10. andynu said...

I pity the fool who lives without revision control; however, I’m thrilled by `set :scm, :none` because I use Capistrano for deployment of java webapps that are built by our continuous integration server. Since all the code is in svn, and the builds are tied to specific revisions, there is no need for us to keep the binaries in the repository. This feature is going to drastically simplify my recipes. Thanks.

11. Matthijs Langenberg said...

Thanks for the improvements for the :copy strategy! Since git is nicely distributed, I like it to export from my local git repository. I’m just using: set :repository, ’.’

05 May 2008

12. Josh Knowles said...

The upload/download tasks are quite nice, thanks!

The gem didn’t however pick up on the dependencies on net-ssh-gateway and net-scp and I had to install these manually.

13. Jamis said...

@Josh, how did you install the gem? “gem install capistrano” should (and did, for me) pick up those dependencies. However, if you do “gem install ./capistrano-2.3.0.gem” (after downloading the gem separately), rubygems will not enforce dependencies.

14. Jeremy Lecour said...

Hi

I don’t know if something is wrong with my setup, but since I upgraded from 2.2 to 2.3; Cap is asking for my SSH password and doesn’t use the public keys anymore.

On a any tested “client” computer, cap is asking for the password, but in the meantime, a simple manual ssh connection is working with te pubkey.

I’ve looked on the server side (/var/log/auth.log) and it shows the cap password connection and the manual ssh pubkey connection.

I’ve tried this from 2 Macs (Mac OS X 10.5) and a PC (Debian 4).

15. Jamis said...

@Jeremy, yeah, it’s a known regression in Net::SSH 2.0 which has been fixed, but not yet released. I’ll release it this week. The problem is that you’re using keys, but not an ssh-agent, and Net::SSH 2 doesn’t know to load common key names by default. For now, you can fix it by doing this:

1
ssh_options[:keys] = %w(~/.ssh/id_dsa ~/.ssh/id_rsa)

That should “fix” it.

06 May 2008

16. Josh Knowles said...

@Jamis, I installed via “sudo gem update”, which apparently didn’t pick up the new dependencies. Is this a bug in gems?

17. Jamis said...

@Josh, ah, yes, I’ve heard of that. “gem update” doesn’t check dependencies. Not sure if that’s a bug or design decision—you should probably bring it up with the gems team.

07 May 2008

18. Rafael said...

Hi, first, ... I want to thank… I was so looking forward… but I have some strange errors… may you have a clue, what that could be?

/usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/git.rb:217:in `query_revision’: Unable to resolve revision for origin/master (RuntimeError) from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:35:in `send’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:35:in `method_missing’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:63:in `local’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:35:in `method_missing’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy.rb:37:in `load’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/configuration/variables.rb:87:in `call’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/configuration/variables.rb:87:in `fetch’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/configuration/variables.rb:110:in `protect’ ... 34 levels… from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/cli/execute.rb:14:in `execute’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/bin/cap:4 from /usr/local/bin/cap:19:in `load’ from /usr/local/bin/cap:19

19. Tim said...

i’m having the same problem as Rafael. 2.3 broke git checkouts for me.

20. Tony Perrie said...

I’m having the same problem as Rafael. Capistrano 2.3.0 broke git checkouts for me on Mac OS X Leopard.

command finished /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/git.rb:217:in `query_revision’: Unable to resolve revision for HEAD (RuntimeError)

21. Adam said...

Same for me.. 18, 19, 20.

08 May 2008

22. Michael Siebert said...

same for me.

  • executing “if [ -d /server/rails/nebraska/shared/cached-copy ]; then cd /server/rails/nebraska/shared/cached-copy && git fetch—tags origin && git reset—hard f9abb5c4263ba95dc6bae99b1fd559c89867c73a; else git clone git@svn.reporting-online.com:nebraska /server/rails/nebraska/shared/cached-copy && cd /server/rails/nebraska/shared/cached-copy && git checkout -b deploy f9abb5c4263ba95dc6bae99b1fd559c89867c73a; fi” servers: “dmt.reporting-online.com” executing command
    • [out] git@svn.reporting-online.com’s password:
    • [out]
    • [out] fatal: Could not parse object ‘f9abb5c4263ba95dc6bae99b1fd559c89867c73a’.

host is macosx, target is debian

23. Markús said...

Grat work Jamis!

But there’s some points with upload that should be considered (I didn’t test download yet).

I used a MacBook with OS 10.5 for local machine and a CentOS Linux distro as the remote machine.

1. Using sftp [ upload “path/to/local_file”, “path/to/remote_file” ]

With “path/to/local_file”

  • You cannot use “”, since it thows a wreid exception about “undefined method []=”
  • If you use “” in the path you must filter it with File.expand_path

With “path/to/remote_file”

  • You cannot use variables like “shared_path”, “current_release” etc if you specify “” in the “deploy_to” variable, since FTP does not understand “” (a little obious)
  • You MUST specify a remote_file name, whatever, if you don’t the upload will fail

2. Using scp [ upload “path/to/local_file”, “path/to/remote_file”, :via => :scp ]

With “path/to/local_file” happends the same that did with sftp method

With “path/to/remote_file”

  • You can specify ”~” in “deploy_to” variable, it understands it
  • You can omit the remote_file name, only the destination directory is needed

I’m considering that sftp and scp behave identically in all Unix based machines, so this should be the same for all Mac/Linux users, should not?

24. keeney said...

Rafael, remove the “origin/” part in “set: branch=”

09 May 2008

25. Tony Perrie said...

Ignore my previous comment, that was a problem with specifying the git repository.

26. globulous said...

With v2.3.0 now I receive: fatal: ’/usr/local/git/project.git’: unable to chdir or not a git archive

  • transaction: start * executing `deploy:update_code’

Not a problem if I revert back to v2.2

11 May 2008

27. zzeroo said...

Same problem like globulous.

?>> fatal: ’/home/masterson/blog.git’: unable to chdir or not a git archive

Look at http://pastie.caboo.se/pastes/193798. It’s represent the error well.

16 May 2008

28. Josh said...

On the deploy:cleanup i’d get the following error:

no passwd entry for app!

After a little digging I found where “app” was coming from in try_sudo as the default for :runner.

Not sure if it’s the best fix, but changing line 96 of cap’s deploy.rb file to read

as = fetch(:runner, fetch(:user, "app"))

did the trick.

17 May 2008

29. Jauder said...

I’m still building out my cap recipe but upgraded to Cap 2.3.0 along with associated friends.

However, doing a simple cap deploy:check results in the following.

trillian:v jauderho$ cap deploy:check * executing `deploy:check’ * executing “test -d /var/www-rails/v.com/releases” servers: [“dev.v.com”] connection failed for: dev.v.com (NoMethodError: undefined method `strip’ for nil:NilClass)

Any suggestions would be welcome. (this was previously passing in Cap 2.0.0)

The following is what’s installed for netssh net-scp (1.0.0) net-sftp (2.0.0, 1.1.0) net-ssh (2.0.1, 1.1.2) net-ssh-gateway (1.0.0) net-ssh-multi (1.0.0)

30. Jauder said...

I would add that the ssh connection done manually works just fine.

31. thomas Maurer said...

I guess, the problem with this git error on cap deploy…

fatal: Could not parse object ‘f9abb5c4263ba95dc6bae99b1fd559c89867c73a’.

is because of the ‘—tags’ option added to git fetch. without this, it works for me. when I look at the git-fetch manpage, it seems, that with ‘git fetch—tags’ it only fetches the tags and its objects. if you have no tags, it won’t work.. then you have to use ‘git fetch’. perhaps in capistrano it should just call both commands in series.

greetings.

21 May 2008

32. tompata said...

Hello,

the new ‘copy_cache’ directive is very useful, but hard to use on windows.

first of all, the default copy_compression is ‘tar.gz’, but it’s not the default on windows, another option is ‘zip’, but command line zip is also tricky…

the default transfer method is SFTP, but there’s no configuration option to overwrite to SCP – so i had to change in source: transfer.rb… :(

T

22 May 2008

33. Peter Bengtson said...

Same problem as Rafael. Cap 2.3 totally breaks deployment from git repos.

34. Jason Schock said...

@Jamis in regards to post #15: I have an identical setup to Josh’s and I’m in fact using Leopard’s built-in ssh-agent.

When I installed cap 2.3.0, I was also suddenly prompted for a password during deploy. Cap 2.2.0 was previously working fine.

The ssh_options[...] fix you suggested did not work for me, either (it goes in deploy.rb, correct?)

Thanks.

01 Jun 2008

35. Andrew Premdas said...

Ditto 31

36. Jamis said...

Sorry for the extended silence, all. I’ve released an update to Net::SSH, Net::SFTP, and Net::SCP which should fix some (but not all) reported issues (particularly those dealing with file uploads from windows being truncated). I’m still working on getting another capistrano release ready, which will fix git deploys (among other things).

02 Jun 2008

37. Chris Bailey said...

As someone eluded to above, the fix for the “Could not parse object” problem is to ensure you have at least one tag in your repository. What the tag is on doesn’t matter, and you just need one. I wrote this up here in case folks want a bit more step-by-step:

http://codeintensity.blogspot.com/2008/06/fixing-capistrano-230-and-git-deploy.html

03 Jun 2008

38. Thiyagarajan said...

Hi

I am thiyagarajan. I am using Capistrano for deploying rails application to Linux server, i am getting confusion do symlink for example /var/www/apps/#{application}/current/public/images/products to /var/www/apps/#{application}/shared/products.

39. Thiyagarajan said...

Hi

I am thiyagarajan. I am using Capistrano for deploying rails application to Linux server, i am getting confusion do symlink for example /var/www/apps/#{application}/current/public/images/products to /var/www/apps/#{application}/shared/products. Any one help me …

Regards,

Thiyagarajan Veluchamy

04 Jun 2008

40. KJW said...

The ‘(NoMethodError: undefined method `strip’ for nil:NilClass)’ error message is probably caused by a syntax error in your ’~/.ssh/config’ file.

My .ssh/config had the string ‘ForwardX11=yes’ which caused the error because the correct syntax is ‘ForwardX11 yes’ (without the ’=’).

I’ve submitted a patch for net-ssh to ignore the syntax error and produce a warning:

http://rubyforge.org/tracker/index.php?func=detail&aid=20510&group_id=274&atid=1125

05 Jun 2008

41. Chris Blackburn said...

Yay! Thanks so much Jamis. We’ve been looking forward to this release.

42. Brian Beers said...

@Jauder – I had the same exact problem. I ended update deleting the contents of ~/.ssh/config (I had some git configuration in there) and everything works now, capistrano and git.

Good luck

08 Jun 2008

43. Ray said...

I am using Git as the scm. cap deploy:cold works but subsequent cap deploy gives me an error when it executes the deploy:update_code task. The error is http://pastie.org/211195. I think it happens on the git reset—hard 6b677118fc7abe111098225eda9996275f5f2b30. I have these in my deploy.rb:

set :branch , “master” set :deploy_via, :remote_cache set :git_enable_submodules, 1

44. Jauder said...

KJW, Brian Beers,

Thanks for the hints. It turns out that ’=’ is not valid in .ssh/config when parsed by net-ssh (it works for normal ssh client stuff).

Removing that causes the error to go away. Thanks.

11 Jun 2008

45. Mark said...

Get the same error as a few others:

fatal: Could not parse object ‘cc6d19138062d69ca8ff3c7ba7dc5b04c160ec5f’.

When doing deploy (apart from the first one).

Hope somebody can find a fix for this soon?

Using:

set :branch, “master” set :deploy_via, :remote_cache set :keep_releases, 2 set :git_enable_submodules, 1

46. Mark said...

Just found the fix for this:

It is the ‘git fetch—tags …’ but as somebody pointed out earlier. A fix for this is explained by Jamis at http://groups.google.com/group/capistrano/browse_thread/thread/b14635454d40087b

27 Jun 2008

47. Ron said...

Not sure what I’m doing wrong, but Cap seems to want net-s(sh, ftp, cp) exactly at version 2.0.0. I have 2.0.3 (and the other newer versions) installed, but it refuses to install unless I drop in exactly v2.0.0 of the libs.

04 Jul 2008

48. Dave said...

So… how about that vendor/rails tip?