Capistrano 2.3.0
On February 16 I set myself to travel a path that I hoped would eventually help me deal with the project overload I was feeling. Today I get to release Capistrano 2.3, which is very near to the end of that path!
gem install capistranoCapistrano 2.3 is primarily significant in that it switches to the new Net::SSH v2 library, which is faster and slimmer than the older Net::SSH v1 library. It also pulls in Net::SFTP v2, and the new Net::SSH::Gateway and Net::SCP libraries.
In addition to that, cap 2.3 adds several exciting (if I do say so myself) new features, and a few bug fixes that had lingered for far too long.
FEATURE: The :copy strategy has been significantly improved. Instead of doing a blind checkout/export, archiving the result, and copying it over, you can specify that a local cached copy of your repository should be used. If the cached copy does not exist, it is created (via a checkout), otherwise it is simply updated (a much faster operation than a checkout, typically). This functions much like the :remote_cache strategy, but locally. To use this, simply set the :copy_cache variable to true:
1 2 |
set :deploy_via, :copy set :copy_cache, true |
By default, the cached copy will be in your machine’s temporary directory (/tmp, for example), but you can specify your own location by setting :copy_cache to the desired path:
1 2 |
set :deploy_via, :copy set :copy_cache, "/u/caches/#{application}" |
But wait! There’s more! Suppose you have certain files that you don’t want to deploy, like photoshop files or your .git directory. You can set the :copy_exclude variable to a file glob (or an array of globs):
1 2 3 |
set :deploy_via, :copy set :copy_cache, true set :copy_exclude, [".git", "materials"] |
This should make your deploys faster than ever, especially using a tip I hope to share in the next day or two that uses this in conjunction with some vendor/rails symlinking.
FEATURE: Even though I strenuously believe it is a mistake to deploy anything that is not under source control, I’ve finally caved and added a dumb :none SCM module. It can be used to deploy a specific directory if used with the :copy strategy:
1 2 3 |
set :repository, "." set :scm, :none set :deploy_via, :copy |
Again, there are very, very few cases when I think use of this technique is justified, but because I ran into one myself a month ago, I decided it was worth adding.
FEATURE: Support was added for “depend :remote, file” to test for the existence of a specific file:
depend :remote, "/etc/syslog.conf" |
This is used in conjunction with the deploy:check task.
FEATURE: You can specify ssh_options per-server, now, simply by giving an :ssh_options key and corresponding hash with the server definition:
role "some.host", :ssh_options => { :keys => "/path/to/key" } |
FEATURE: There are two new file transfer helpers, upload and download, which are much more powerful and resource-friendly than the old ‘put’ and ‘get’ helpers. You can use upload and download to transfer single files, or entire directory trees:
1 2 |
upload "/local/file", "/remote/file" download "/remote/file", "/local/file" |
This will transfer to or from all active servers, which is particular tricky when using the download helper, since it will download the file simultaneously from all active hosts. To make this work, you need to make sure each is downloaded to a different location:
1 2 |
Dir.mkdir("destination") download "/remote/file", "destination/file-$CAPISTRANO:HOST$" |
The above will download the file from each host to a file on the local host, where the local file includes the name of the source host. Tricky!
Also, you can now specify that you want to upload or download via SCP instead of SFTP:
1 2 |
upload "local", "remote", :via => :scp download "remote", "local", :via => :scp |
The default is :sftp.
The less exciting (but still mildly titillating) things are the bug fixes and pleasure-enhancing behavioral changes:
- The default run options are now mixed into the command options when executing a command from the capistrano shell.
- The git SCM now uses git-ls-remote instead of git-rev-parse to resolve the revision of the checkout.
- A trivial export method has been added to the git SCM.
- The git SCM will include tags when it fetches.`
- The sudo() helper now works nicely with complex and chained commands.
- The deploy:setup task will now use sudo if :use_sudo is true (the default)
So, have at it! Remember to report bugs to http://capistrano.lighthouseapp.com. And patches are always welcome via git—just fork the capistrano repository at git://github.com/jamis/capistrano.git (thanks GitHub!).
Cheers!
Reader Comments
Sweet!
Thanks so much for your hard work, Jamis.
It is very much appreciated!
2 May 2008
Thanks Jamis.
Does this fix the long standing hangs when using gateway?
Also I currently get …
ERROR: While executing gem … (Gem::RemoteFetcher::FetchError) Gem::RemoteFetcher::FetchError: bad response Not Found 404 reading http://gems.rubyforge.org/gems/capistrano-2.3.0.gem
I presume we need to wait a while to grab it?
Thanks
2 May 2008
@Jim, it should fix the hang problem, but then, upgrading to Net::SSH 1.1.4 should fix it, too, if you don’t particularly want to upgrade to Net::SSH 2 yet. Regarding the 404 error, yeah, the mirrors may not all have updated yet. It works for some and not for others—just retry a few times until you get a mirror that’s updated.
2 May 2008
It does indeed fix the gateway thanks (do I still need set :synchronous_connect, true?)
When I execute the following in my capfile…
sudo “svc -u /service/treasurehunt”
I get…It does seem to execute so what does the * [err: mean?
Thanks Jim
2 May 2008
@Jim, the :synchronous_connect variable is completely ignored in cap 2.3, so you can remove it. It has no effect on anything, anymore. As for the “err”, it just means that some text was received from the server on the STDERR stream. “err” is the tag for stderr, and “out” is the tag for stdout.
2 May 2008
oops it seems to have munged my output above
Trying again
2 May 2008
What about pluggable ssh? Why must we use net:ssh for cap when we use openssh for everything else?
2 May 2008
@anonymous, pluggable anything makes things 10x more complicated and introduces a lot of edge cases. I’d rather not go there. That said, if you want to take a stab at a patch, I’d consider it if it doesn’t overcomplicate things. If you want to use openssh, though, there’s already vlad, which does.
3 May 2008
Really like the new upload and download options. Going to help out with the site I’m working on right now. I was managing it before, but this is much cleaner. nice work.
3 May 2008
I pity the fool who lives without revision control; however, I’m thrilled by `set :scm, :none` because I use Capistrano for deployment of java webapps that are built by our continuous integration server. Since all the code is in svn, and the builds are tied to specific revisions, there is no need for us to keep the binaries in the repository. This feature is going to drastically simplify my recipes. Thanks.
3 May 2008
Thanks for the improvements for the :copy strategy! Since git is nicely distributed, I like it to export from my local git repository. I’m just using: set :repository, ’.’
3 May 2008
The upload/download tasks are quite nice, thanks!
The gem didn’t however pick up on the dependencies on net-ssh-gateway and net-scp and I had to install these manually.
5 May 2008
@Josh, how did you install the gem? “gem install capistrano” should (and did, for me) pick up those dependencies. However, if you do “gem install ./capistrano-2.3.0.gem” (after downloading the gem separately), rubygems will not enforce dependencies.
5 May 2008
Hi
I don’t know if something is wrong with my setup, but since I upgraded from 2.2 to 2.3; Cap is asking for my SSH password and doesn’t use the public keys anymore.
On a any tested “client” computer, cap is asking for the password, but in the meantime, a simple manual ssh connection is working with te pubkey.
I’ve looked on the server side (/var/log/auth.log) and it shows the cap password connection and the manual ssh pubkey connection.
I’ve tried this from 2 Macs (Mac OS X 10.5) and a PC (Debian 4).
5 May 2008
@Jeremy, yeah, it’s a known regression in Net::SSH 2.0 which has been fixed, but not yet released. I’ll release it this week. The problem is that you’re using keys, but not an ssh-agent, and Net::SSH 2 doesn’t know to load common key names by default. For now, you can fix it by doing this:
That should “fix” it.
5 May 2008
@Jamis, I installed via “sudo gem update”, which apparently didn’t pick up the new dependencies. Is this a bug in gems?
6 May 2008
@Josh, ah, yes, I’ve heard of that. “gem update” doesn’t check dependencies. Not sure if that’s a bug or design decision—you should probably bring it up with the gems team.
6 May 2008
Hi, first, ... I want to thank… I was so looking forward… but I have some strange errors… may you have a clue, what that could be?
/usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/git.rb:217:in `query_revision’: Unable to resolve revision for origin/master (RuntimeError) from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:35:in `send’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:35:in `method_missing’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:63:in `local’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/base.rb:35:in `method_missing’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy.rb:37:in `load’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/configuration/variables.rb:87:in `call’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/configuration/variables.rb:87:in `fetch’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/configuration/variables.rb:110:in `protect’ ... 34 levels… from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/cli/execute.rb:14:in `execute’ from /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/bin/cap:4 from /usr/local/bin/cap:19:in `load’ from /usr/local/bin/cap:19
7 May 2008
i’m having the same problem as Rafael. 2.3 broke git checkouts for me.
7 May 2008
I’m having the same problem as Rafael. Capistrano 2.3.0 broke git checkouts for me on Mac OS X Leopard.
command finished /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.3.0/lib/capistrano/recipes/deploy/scm/git.rb:217:in `query_revision’: Unable to resolve revision for HEAD (RuntimeError)
7 May 2008
Same for me.. 18, 19, 20.
7 May 2008
same for me.
host is macosx, target is debian
8 May 2008
Grat work Jamis!
But there’s some points with upload that should be considered (I didn’t test download yet).
I used a MacBook with OS 10.5 for local machine and a CentOS Linux distro as the remote machine.
1. Using sftp [ upload “path/to/local_file”, “path/to/remote_file” ]
With “path/to/local_file”
With “path/to/remote_file”
2. Using scp [ upload “path/to/local_file”, “path/to/remote_file”, :via => :scp ]
With “path/to/local_file” happends the same that did with sftp method
With “path/to/remote_file”
I’m considering that sftp and scp behave identically in all Unix based machines, so this should be the same for all Mac/Linux users, should not?
8 May 2008
Rafael, remove the “origin/” part in “set: branch=”
8 May 2008
Ignore my previous comment, that was a problem with specifying the git repository.
9 May 2008
With v2.3.0 now I receive: fatal: ’/usr/local/git/project.git’: unable to chdir or not a git archive
Not a problem if I revert back to v2.2
9 May 2008
Same problem like globulous.
?>> fatal: ’/home/masterson/blog.git’: unable to chdir or not a git archive
Look at http://pastie.caboo.se/pastes/193798. It’s represent the error well.
11 May 2008
On the deploy:cleanup i’d get the following error:
After a little digging I found where “app” was coming from in try_sudo as the default for :runner.
Not sure if it’s the best fix, but changing line 96 of cap’s deploy.rb file to read
did the trick.
16 May 2008
I’m still building out my cap recipe but upgraded to Cap 2.3.0 along with associated friends.
However, doing a simple cap deploy:check results in the following.
trillian:v jauderho$ cap deploy:check * executing `deploy:check’ * executing “test -d /var/www-rails/v.com/releases” servers: [“dev.v.com”] connection failed for: dev.v.com (NoMethodError: undefined method `strip’ for nil:NilClass)
Any suggestions would be welcome. (this was previously passing in Cap 2.0.0)
The following is what’s installed for netssh net-scp (1.0.0) net-sftp (2.0.0, 1.1.0) net-ssh (2.0.1, 1.1.2) net-ssh-gateway (1.0.0) net-ssh-multi (1.0.0)
17 May 2008
I would add that the ssh connection done manually works just fine.
17 May 2008
I guess, the problem with this git error on cap deploy…
fatal: Could not parse object ‘f9abb5c4263ba95dc6bae99b1fd559c89867c73a’.
is because of the ‘—tags’ option added to git fetch. without this, it works for me. when I look at the git-fetch manpage, it seems, that with ‘git fetch—tags’ it only fetches the tags and its objects. if you have no tags, it won’t work.. then you have to use ‘git fetch’. perhaps in capistrano it should just call both commands in series.
greetings.
17 May 2008
Hello,
the new ‘copy_cache’ directive is very useful, but hard to use on windows.
first of all, the default copy_compression is ‘tar.gz’, but it’s not the default on windows, another option is ‘zip’, but command line zip is also tricky…
the default transfer method is SFTP, but there’s no configuration option to overwrite to SCP – so i had to change in source: transfer.rb… :(
T
21 May 2008
Same problem as Rafael. Cap 2.3 totally breaks deployment from git repos.
22 May 2008
@Jamis in regards to post #15: I have an identical setup to Josh’s and I’m in fact using Leopard’s built-in ssh-agent.
When I installed cap 2.3.0, I was also suddenly prompted for a password during deploy. Cap 2.2.0 was previously working fine.
The ssh_options[...] fix you suggested did not work for me, either (it goes in deploy.rb, correct?)
Thanks.
22 May 2008
Ditto 31
1 Jun 2008
Sorry for the extended silence, all. I’ve released an update to Net::SSH, Net::SFTP, and Net::SCP which should fix some (but not all) reported issues (particularly those dealing with file uploads from windows being truncated). I’m still working on getting another capistrano release ready, which will fix git deploys (among other things).
1 Jun 2008
As someone eluded to above, the fix for the “Could not parse object” problem is to ensure you have at least one tag in your repository. What the tag is on doesn’t matter, and you just need one. I wrote this up here in case folks want a bit more step-by-step:
http://codeintensity.blogspot.com/2008/06/fixing-capistrano-230-and-git-deploy.html
2 Jun 2008
Hi
I am thiyagarajan. I am using Capistrano for deploying rails application to Linux server, i am getting confusion do symlink for example /var/www/apps/#{application}/current/public/images/products to /var/www/apps/#{application}/shared/products.
3 Jun 2008
Hi
I am thiyagarajan. I am using Capistrano for deploying rails application to Linux server, i am getting confusion do symlink for example /var/www/apps/#{application}/current/public/images/products to /var/www/apps/#{application}/shared/products. Any one help me …
Regards,
Thiyagarajan Veluchamy
3 Jun 2008
The ‘(NoMethodError: undefined method `strip’ for nil:NilClass)’ error message is probably caused by a syntax error in your ’~/.ssh/config’ file.
My .ssh/config had the string ‘ForwardX11=yes’ which caused the error because the correct syntax is ‘ForwardX11 yes’ (without the ’=’).
I’ve submitted a patch for net-ssh to ignore the syntax error and produce a warning:
http://rubyforge.org/tracker/index.php?func=detail&aid=20510&group_id=274&atid=1125
4 Jun 2008
Yay! Thanks so much Jamis. We’ve been looking forward to this release.
5 Jun 2008
@Jauder – I had the same exact problem. I ended update deleting the contents of ~/.ssh/config (I had some git configuration in there) and everything works now, capistrano and git.
Good luck
5 Jun 2008
I am using Git as the scm. cap deploy:cold works but subsequent cap deploy gives me an error when it executes the deploy:update_code task. The error is http://pastie.org/211195. I think it happens on the git reset—hard 6b677118fc7abe111098225eda9996275f5f2b30. I have these in my deploy.rb:
set :branch , “master” set :deploy_via, :remote_cache set :git_enable_submodules, 1
8 Jun 2008
KJW, Brian Beers,
Thanks for the hints. It turns out that ’=’ is not valid in .ssh/config when parsed by net-ssh (it works for normal ssh client stuff).
Removing that causes the error to go away. Thanks.
8 Jun 2008
Get the same error as a few others:
fatal: Could not parse object ‘cc6d19138062d69ca8ff3c7ba7dc5b04c160ec5f’.
When doing deploy (apart from the first one).
Hope somebody can find a fix for this soon?
Using:
set :branch, “master” set :deploy_via, :remote_cache set :keep_releases, 2 set :git_enable_submodules, 1
11 Jun 2008
Just found the fix for this:
It is the ‘git fetch—tags …’ but as somebody pointed out earlier. A fix for this is explained by Jamis at http://groups.google.com/group/capistrano/browse_thread/thread/b14635454d40087b
11 Jun 2008
Not sure what I’m doing wrong, but Cap seems to want net-s(sh, ftp, cp) exactly at version 2.0.0. I have 2.0.3 (and the other newer versions) installed, but it refuses to install unless I drop in exactly v2.0.0 of the libs.
27 Jun 2008
So… how about that vendor/rails tip?
4 Jul 2008