On February 16 I set myself to travel a path that I hoped would eventually help me deal with the project overload I was feeling. Today I get to release Capistrano 2.3, which is very near to the end of that path!

gem install capistrano

Capistrano 2.3 is primarily significant in that it switches to the new Net::SSH v2 library, which is faster and slimmer than the older Net::SSH v1 library. It also pulls in Net::SFTP v2, and the new Net::SSH::Gateway and Net::SCP libraries.

In addition to that, cap 2.3 adds several exciting (if I do say so myself) new features, and a few bug fixes that had lingered for far too long.

FEATURE: The :copy strategy has been significantly improved. Instead of doing a blind checkout/export, archiving the result, and copying it over, you can specify that a local cached copy of your repository should be used. If the cached copy does not exist, it is created (via a checkout), otherwise it is simply updated (a much faster operation than a checkout, typically). This functions much like the :remote_cache strategy, but locally. To use this, simply set the :copy_cache variable to true:

1
2

set :deploy_via, :copy
set :copy_cache, true

By default, the cached copy will be in your machine’s temporary directory (/tmp, for example), but you can specify your own location by setting :copy_cache to the desired path:

1
2

set :deploy_via, :copy
set :copy_cache, "/u/caches/#{application}"

But wait! There’s more! Suppose you have certain files that you don’t want to deploy, like photoshop files or your .git directory. You can set the :copy_exclude variable to a file glob (or an array of globs):

1
2
3

set :deploy_via, :copy
set :copy_cache, true
set :copy_exclude, [".git", "materials"]

This should make your deploys faster than ever, especially using a tip I hope to share in the next day or two that uses this in conjunction with some vendor/rails symlinking.

FEATURE: Even though I strenuously believe it is a mistake to deploy anything that is not under source control, I’ve finally caved and added a dumb :none SCM module. It can be used to deploy a specific directory if used with the :copy strategy:

1
2
3

set :repository, "."
set :scm, :none
set :deploy_via, :copy

Again, there are very, very few cases when I think use of this technique is justified, but because I ran into one myself a month ago, I decided it was worth adding.

FEATURE: Support was added for “depend :remote, file” to test for the existence of a specific file:

depend :remote, "/etc/syslog.conf"

This is used in conjunction with the deploy:check task.

FEATURE: You can specify ssh_options per-server, now, simply by giving an :ssh_options key and corresponding hash with the server definition:

role "some.host", :ssh_options => { :keys => "/path/to/key" }

FEATURE: There are two new file transfer helpers, upload and download, which are much more powerful and resource-friendly than the old ‘put’ and ‘get’ helpers. You can use upload and download to transfer single files, or entire directory trees:

1
2

upload "/local/file", "/remote/file"
download "/remote/file", "/local/file"

This will transfer to or from all active servers, which is particular tricky when using the download helper, since it will download the file simultaneously from all active hosts. To make this work, you need to make sure each is downloaded to a different location:

1
2

Dir.mkdir("destination")
download "/remote/file", "destination/file-$CAPISTRANO:HOST$"

The above will download the file from each host to a file on the local host, where the local file includes the name of the source host. Tricky!

Also, you can now specify that you want to upload or download via SCP instead of SFTP:

1
2

upload "local", "remote", :via => :scp
download "remote", "local", :via => :scp

The default is :sftp.

The less exciting (but still mildly titillating) things are the bug fixes and pleasure-enhancing behavioral changes:

• The default run options are now mixed into the command options when executing a command from the capistrano shell.
• The git SCM now uses git-ls-remote instead of git-rev-parse to resolve the revision of the checkout.
• A trivial export method has been added to the git SCM.
• The git SCM will include tags when it fetches.`
• The sudo() helper now works nicely with complex and chained commands.
• The deploy:setup task will now use sudo if :use_sudo is true (the default)

So, have at it! Remember to report bugs to http://capistrano.lighthouseapp.com. And patches are always welcome via git—just fork the capistrano repository at git://github.com/jamis/capistrano.git (thanks GitHub!).

Cheers!

At last! Net::SSH 2.0 is available! Also available are Net::SFTP 2.0, Net::SCP 1.0, Net::SSH::Gateway 1.0, and Net::SSH::Multi 1.0.

1
2
3
4
5

$ gem install net-ssh \
  net-sftp \
  net-scp \
  net-ssh-gateway \
  net-ssh-multi

All of these Ruby libraries are for communicating with remote servers via the SSH protocol in different ways.

Net::SSH and Net::SFTP are both significant upgrades from their previous incarnations; if you have used either library in the past, you’ll want to read the documentation (Net::SSH, Net::SFTP). The Net::SSH API is still fairly similar to the way it was before, but the Net::SFTP API is entirely different.

All have pretty complete RDoc documentation, so you should be able to employ “ri” to good effect to find your way around the libraries. (Try “ri Net::SSH”, for example, to get started.)

Per-library synopses follow.

Net::SSH 2.0

This is a significant upgrade from Net::SSH 1.x. Changes from 1.x include (but are not limited to):

• Net::SSH 2.0 no longer requires the Needle dependency-injection library. This has made the library faster, lighter, and easier to understand.
• The “sync” and “sync-shell” services have been removed and will not return. (If you need the functionality of those services, I strongly encourage you to port them to Net::SSH 2 and release them as separate extensions.)
• Options in ~/.ssh/config files are (partially) supported, and are loaded by default now.
• The Net::SSH::start method now requires both a host and a username as the first two parameters, rather than inferring the username.
• There is now a Session#exec method that makes it easier than ever to just execute and interact with a command.
• Channel now sports an #on_process method that you can send a block to, which will be executed on every pass of the event loop.
• Channel#on_request now accepts a string to indicate which request you want to respond to.
• Channel open failures may be captured via Channel#on_open_failure.
• Lots, lots, LOTS, more—refer to the rdocs for all the gory details.

1
2
3
4
5
6

require 'net/ssh'

Net::SSH.start('localhost', 'jamis') do |ssh|
  ssh.exec('hostname') # prints the results to $stdout
  ssh.loop
end

Net::SFTP 2.0

This is a complete rewrite of the original Net::SFTP 1.x code, and shares very, very little in common with it. The new version has a much cleaner implementation and API, and provides some really handy methods for transferring files and directories.

1
2
3
4
5
6
7
8
9
10

require 'net/sftp'

Net::SFTP.start('localhost', 'jamis') do |sftp|
  sftp.upload! "/local/file", "/remote/file"
  sftp.download! "/remote/file", "/local/file"

  sftp.file.open("/remote/file", "w") do |file|
    file.puts "here is some data"
  end
end

Net::SCP 1.0

This provides a way to transfer files and directories via the SCP protocol, over Net::SSH.

1
2
3
4
5
6

require 'net/scp'

Net::SCP.start('localhost', 'jamis') do |scp|
  scp.upload! "/local/file", "/remote/file"
  scp.download! "/remote/file", "/local/file"
end

Net::SSH::Gateway 1.0

This library makes it easy to tunnel connections though firewalls. You simply connect to the gateway machine, and then specify which ports you want forwarded.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

require 'net/ssh/gateway'

gateway = Net::SSH::Gateway.new('host', 'user')

gateway.ssh("host.private", "user") do |ssh|
  puts ssh.exec!("hostname")
end

gateway.open("host.private", 80) do |port|
  require 'net/http'
  Net::HTTP.get_print("127.0.0.1", "/path", port)
end

gateway.shutdown!

Net::SSH::Multi 1.0

This library makes it simple to open multiple Net::SSH connections and tie them all together, running commands in parallel (much like Capistrano does).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

require 'net/ssh/multi'

Net::SSH::Multi.start do |session|
  # access servers via a gateway
  session.via 'gateway', 'gateway-user'

  # define the servers we want to use
  session.use 'user1@host1'
  session.use 'user2@host2'

  # define servers in groups for more granular access
  session.group :app do
    session.use 'user@app1'
    session.use 'user@app2'
  end

  # execute commands on all servers
  session.exec "uptime"

  # execute commands on a subset of servers
  session.with(:app).exec "hostname"

  # run the aggregated event loop
  session.loop
end

Enjoy!

With the impending releases of the improved Net::SSH family of libraries, and the next Capistrano release that will depend on them, it was time to go back and make sure that the oldies-but-goodies will still work along side these new-comers.

To that end, I present:

Net::SSH 1.1.3

This maintenance release will hopefully fix the hangs that people have been reporting when dealing with long-running requests in Net::SSH, Net::SFTP, and Capistrano.

gem install net-ssh

It is strongly recommended that all Net::SSH users upgrade to this version.

Capistrano 1.4.2

This release merely adds an explicit dependency on Net::SSH and Net::SFTP versions less than 1.99.0, allowing you to use both cap1 and cap2 side-by-side after Net::SSH v2 (and friends) are released.

gem install --version 1.4.2 capistrano

If you are needing to run cap1 and cap2 side-by-side (for instance, if you haven’t yet upgraded some recipes, but want to enjoy the cap2 goodies on newer projects), you should upgrade your cap1 installation to 1.4.2. If you are completely upgraded to cap2, you can safely ignore this release.

I’m almost ready to do a preview release of the next Capistrano version, which is built on top of the new Net::SSH library. To do that, though, I need to prime the pump by pushing out a few small fixes to Net::SSH. So, behold!

Net::SSH 2.0 Preview Release #4 (1.99.3):

gem install --source http://gems.jamisbuck.org net-ssh

This release has two minor new features and two fixed bugs:

• Make sure HOME is set to something sane, even on OS’s that don’t set it by default. This makes things work a little better on Windows than before. (Ahem…any windows users out there that want to start playing with Net::SSH…I’d love to hear from you how it does.)

• Add a :passphrase option to specify the passphrase to use with private keys. This lets you use encrypted keys without interaction (though since it requires you to hard-code the passphrase in your script…it kind of makes it pointless to use an encrypted key. Still, I’ve had lots of requests for this, and Francis Sullivan finally provided a patch).

• BUG FIX: a new auth-agent connection will now be opened for every auth-agent channel request, rather than reusing a single auth-agent connection. This fixes some rather bizarre errors on some hosts.

• BUG FIX: explicitly provided keys (those given via the :keys option) are now loaded correctly, thanks to some timely File.expand_path’ing provided in a patch from Thomas Hudson.

Net::SSH::Multi 1.0 Preview Release #2 (0.99.1)

Unrelated to Capistrano (since the next release of cap won’t be using it), I’m also pushing out a second preview release of Net::SSH::Multi, a library for managing multiple Net::SSH connections in parallel.

gem install --source http://gems.jamisbuck.org net-ssh-multi

This release has a single bug fix:

• BUG FIX: don’t try to do IO.select on closed streams. This fixes a problem related to auth-agent channels getting closed and causing the parallel session processing to blow up.

Capistrano 2.3.0 preview release?

I’ll hopefully have a preview release for the next Capistrano version next week sometime. It’s close! I’m using it myself, locally, but I want to try it on a few more deploys to make sure it really works as advertised. I haven’t actually run any timings on it, but Capistrano with Net::SSH v2 feels significantly faster than it was on Net::SSH 1.x. Stay tuned!

I’ve got lots of code to share with you all today:

Net::SSH 2.0 Preview Release #3

Not very many external changes, but channels now have an on_open_failure callback that you can employ to be told when a channel could not be opened.

1
2
3
4
5
6
7

channel = ssh.open_channel do |ch|
  # ...
end

channel.on_open_failure do |ch, code, reason|
  puts "could not open channel because #{reason.inspect} #{code}"
end

This change was necessary to make port forwarding a bit more reliable for Net::SSH::Gateway (see below).

Also, individual Net::SSH sessions may contain their own custom properties, much as Net::SSH channel objects do, which can be quite handy when you need to keep some state associated with the session.

ssh[:key] = "value"

This change (and several other internal refactorings) was necessary for the correct working of the Net::SSH::Multi library (see below).

To get the preview release #3 (version 1.99.2):

gem install --source http://gems.jamisbuck.org net-ssh

You can follow development via GitHub:

http://github.com/jamis/net-ssh/tree/master

Net::SFTP 2.0 Preview Release #2

All that was added in this preview release was custom properties on Upload instances:

1
2
3

uploader = sftp.upload("local.file", "remote.file")
uploader[:failed] = false
# ...

This was necessary to get Capistrano file transfers to work with Net::SFTP 2.0.

To get the preview release #2 (version 1.99.1):

gem install --source http://gems.jamisbuck.org net-sftp

You can follow development via GitHub:

http://github.com/jamis/net-sftp/tree/master

Net::SSH::Gateway 1.0 Preview Release #1

Net::SSH::Gateway is (in essence) the extraction of the gateway code from Capistrano into its own library. It lets you tunnel SSH connections through some “gateway” server to servers that would be otherwise inaccessible.

1
2
3
4
5
6

require 'net/ssh/gateway'

gateway = Net::SSH::Gateway.new('gateway.host', 'username')
gateway.ssh('remote.host', 'user') do |ssh|
  # ...
end

You can also use it as a general facilitator for forwarding connections over a local port, when you don’t care what port is to be used—you just want to connect to a remote server.

1
2
3

gateway.open('remote.host', 80) do |port|
  Net::HTTP.get_print '127.0.0.1', '/path', port
end

The current thinking is that the next release of Capistrano will ditch its own gateway implementation in favor of Net::SSH::Gateway. To get this first preview release (version 0.99.0):

gem install --source http://gems.jamisbuck.org net-ssh-gateway

You can follow development via GitHub:

http://github.com/jamis/net-ssh-gateway/tree/master

Net::SSH::Multi 1.0 Preview Release #1

Net::SSH::Multi is the guts of Capistrano, extracted into a library of their own. It allows you to define and categorize servers, and then execute commands in parallel on them, or on subsets of them, using an interface similar, if not identical, to that of Net::SSH::Connection::Session and Channel.

Eventually, Capistrano will be refactored to take advantage of Net::SSH::Multi, but doing so will require some significant changes to Capistrano’s innards, and would almost certainly break many third-party libraries. Thus, you won’t see Capistrano on Net::SSH::Multi until Capistrano 3.0 or so (which will not be the next release of Capistrano). However, you can use Net::SSH::Multi to implement most of Capistrano’s functionality in whatever form you like. Want Capistrano-in-a-rakefile? Have at it!

1
2
3
4
5
6
7
8
9

require 'net/ssh/multi'

Net::SSH::Multi.start do |session|
  session.use 'jamis@somewhere.com'
  session.use 'jamis@elsewhere.com'

  session.exec 'hostname'
  session.loop
end

Grab the first preview release (version 0.99.0):

gem install --source http://gems.jamisbuck.org net-ssh-multi

You can follow development via GitHub:

http://github.com/jamis/net-ssh-multi/tree/master

For all of these libraries I’ve tried to make the rdoc and ri documentation as informative as possible, so please refer to those to get started. If those aren’t helpful enough, let me know what could make them more useful!

After far, far too long, I’m extremely pleased to announce the next preview release of Net::SSH v2 (preview release #2, actual version 1.99.1), as well as the first ever preview releases of Net::SFTP v2 (preview release #1, actual version 1.99.0) and Net::SCP v1 (preview release #1, actual version 0.99.0). You can grab them from my gem server:

1
2
3

$ sudo gem install --source http://gems.jamisbuck.org net-ssh
$ sudo gem install --source http://gems.jamisbuck.org net-sftp
$ sudo gem install --source http://gems.jamisbuck.org net-scp

I’ve taken Eric Hodel’s advice this time around and ditched the pretty documentation I had started for Net::SSH; I’m going to see how far I can get with ri and rdoc alone, this time. Give it a try after installing the above: “ri Net::SSH”, for instance.

There are several caveats about this release:

1. The new Net::SSH and Net::SFTP are not backwards compatible with the previous 1.x releases, so make sure any existing scripts are explicitly depending on versions less than 1.99.0 of these libraries. (The latest Capistrano already has that explicit dependency in it, so you should be safe installing these preview releases even if you are using Capistrano).

2. The “shell”, “open3” and “sync-shell” services are gone from Net::SSH. (There may be others, too, which aren’t coming to mind at the moment.) I do not intend to bring them back. If you used those services, I strongly encourage you to reimplement them for the new Net::SSH, and release them as third-party extensions.

3. These libraries are reentrant (or ought to be), but are not thread-safe. That is to say, if you are accessing the same Net::SSH connection from multiple threads, you’re definitely going to want to do something to protect that resource (mutexes, etc.).

That said, let me know what you think, what issues you have, what suggestions you have, and so forth. I’m so happy to have this release done, though. I can see the light at the end of the tunnel finally!

Oh, and lastly, if you happen to be drinking the git kool-aid, you can follow along via github:

• Net::SSH: http://github.com/jamis/net-ssh/tree/master
• Net::SFTP: http://github.com/jamis/net-sftp/tree/master
• Net::SCP: http://github.com/jamis/net-scp/tree/master

My son, Britton, woke up this morning a little before 4am. As I was getting back into bed after feeding him, I saw that the clock read “4:04”, and (being the geek I am) I immediately thought “heh, 404 Sleep Not Found”.

He woke up a few minutes later, needing a bit of comforting (he’s definitely not usually this needy!), and when I got back into bed, the clock read “4:22”, obviously telling me that sleep was an “Unprocessable Entity”.

And thus the Magic 8ttp Ball was born.

Go on! Ask it anything! It’ll give you a suitably ambiguous HTTP response code in return. I’ve even been told it works great on the iPhone for when you have those burning questions on-the-go.

Capistrano 2.2.0 is now available (well, it’s released, anyway, you might need to wait for the file to propagate to the gem mirrors).

gem install capistrano

Version 2.2.0 sports the following changes:

FEATURE: Dynamic role definition. The role() method now accepts a block, which should return either a host name, a Capistrano::ServerDefinition object, an array of host names, or an array of Capistrano::ServerDefinition objects. This can be used to describe the servers in a role at runtime.

1
2
3
4

role :app do
  hosts = some_method_that_looks_up_the_current_hosts
  hosts[0,3]
end

FEATURE: Alternative server-centric role definitions, using the server() method:

1
2
3
4
5

role :app, "server"
role :web, "server"

# the above is the same as this:
server "server", :app, :web

FEATURE: Support for a :max_hosts option in tasks, that restricts the task so that it is only executed in hosts at a time, in chunks. This helps people who use Capistrano with very large numbers of servers, and prevents them running into connection caps and from running out of memory.

1
2
3
4
5
6
7
8
9
10
11
12
13

task :ping, :max_hosts => 100 do
  # anything here will only run against 100 hosts at a time
end

# alternatively, you can pass :max_hosts to the run command itself for
# finer granularity
task :pong do
  # this will run on ALL hosts at once
  run "something"

  # this will run on no more than 100 hosts at a time
  run "something-else", :max_hosts => 100
end

ENHANCEMENT: Improved Git support! Not being a git user myself (yet), I can’t really speak intelligently about what the changes are, but if you use git chances are you aren’t afraid to dig into the code and read it. For those of you that use git, I’d encourage you to blog about the changes, to assuage some of my embarrassment about releasing code that I can’t describe intelligently!

ENHANCEMENT: Password prompt support in the Mercurial SCM.

ENHANCEMENT: Implement Bzr#next_revision so that pending changes can be reported correctly, and use checkout—lightweight instead of branch.

ENHANCEMENT: Bring back the :p4sync_flags and :p4client_root variables for perforce SCM.

Additionally, there are several minor bugs and typos that have been fixed. You can see the CHANGELOG for all the gory details.

As ever, please report bugs via the Rails trac, at http://dev.rubyonrails.org. And if you aren’t yet subscribed to the Capistrano mailing list, it’s where all the cool cappists hang out.

Net::SFTP is a Ruby library for transferring files using the SFTP (Secure FTP) protocol over SSH. This is a maintenance release consisting of only two changes. First, you no longer need to explicitly call the #connect method. It is called automatically as needed. Second, this release has an explicit (RubyGems) dependency on Net::SSH versions less than 1.99.0. This should pave the way for making smoother the eventual release of Net::SSH v2 (which is not backwards compatible and would otherwise break existing installations of Net::SFTP 1.x).

All users of Net::SFTP are encouraged to upgrade. Using Rubygems, installation is simple:

gem install net-sftp

Alternate source packages (bz2 and zip) may be downloaded from the Net::SSH project page:

http://rubyforge.org/frs/?group_id=274&release_id=19594

Alright, I’m almost ready to unleash this monster on the world, but I’d like a few brave souls to kick the tires. Especially, if you use git or bzr with Capistrano, you might want to give this new version a spin, since it includes a few changes to those SCM modules.

Grab the latest beta gem thus:

gem install --source http://gems.rubyonrails.org capistrano

The significant changes in this release:

FEATURE: Thanks to James Duncan Davidson, Cap 2.2.0 will sport an additional way to define roles. If you have servers that overlap roles, this can DRY things up:

server "host.name", :web, :app

FEATURE: Rob Holland submitted a patch that makes cap play more nicely with very large numbers of servers. If you had too many, cap would bog down, and you may even get errors from having too many simultaneous connections. You can now specify a :max_hosts option, and if a task needs to operate on more than that number of servers at once, the server list will be chopped up and fed through the task in pieces.

FEATURE: David Masover gave us a way to define roles dynamically, by passing a block to the role macro:

1
2
3

role :app do
  ["host1", "host2"]
end

This means that if you have a recipe that knows the roles, but won’t know the hosts until runtime (think EC2 and such), you can more easily code for it.

GIT: The git SCM module has the following changes:

• Distributed git support for better operability with remote_cache strategy
• Use checkout instead of merge for git SCM [nuttycom]
• Add support for :git_enable_submodules variable, to enable submodules with the git SCM

And, of course, there are quite a few other smaller changes, too. Check out the CHANGELOG for the full list.

So, kick the tires and report back. If all looks good by the middle of next week or so, we’ll see an official release.

I’m very pleased to announce that Tyler Bird has been selected to coordinate the documentation for Capistrano. I appreciate (very much!) all of you that volunteered for this.

Tyler has got his work cut out for him, here, but he’s eager to tackle the job, and I look forward to seeing what comes out of it. I’ll post more details as I get them; I just wanted to let everyone know that a coordinator has been selected, at last.

Capistrano’s documentation is, to put it mildly, lacking.

I fully acknowledge that. And I assume all the blame for it, too. I want to do something about it, but I’ve been far, far (far!) too swamped for the last six months to do more than acknowledge the need.

So, I’m pleading for your help. Here’s what I’d like to happen:

1. I’d like one (maybe two) people to step forward, to volunteer to coordinate the documentation effort. This person would ideally be responsible for identifying what needs to be documented, standardizing on a format for documentation, etc. They would work with me to make sure it fits with my vision for Capistrano, but mostly they would have a free reign to do what needs to be done.
2. I’d like as many people as possible to volunteer to submit documentation. This would happen after the coordinator identified what needs to be documented. People would then sign up to take on one of the areas pending documentation. If no action occurs on it within a month or so, that task would automatically become available again for someone else to sign up for.

The documentation thus generated will be hosted at capify.org.

Thoughts? Volunteers? If you’d like to coordinate the documentation effort, please email me directly (jamis at jamisbuck dot org). We’ll see how many volunteer, and I’ll pick the “winner”. :)

If worse comes to worst, though, and no one volunteers, I’m still determined to work out some skeletal documentation as soon as I finish updating Net::SSH and friends. I have no ETA on that at this point, though. :(

After a much larger delay than I would have liked, Capistrano 2.1 is now available! There is a lot going on in this release, including some pretty exciting changes. As ever, install it via RubyGems with:

gem install capistrano

Here’s what’s new, roughly in order of magnitude:

No default PTY. Prior to 2.1, Capistrano would request a pseudo-tty for each command that it executed. This had the side-effect of causing the profile scripts for the user to not be loaded. Well, no more! As of 2.1, Capistrano no longer requests a pty on each command, which means your .profile (or .bashrc, or whatever) will be properly loaded on each command! Note, however, that some have reported on some systems, when a pty is not allocated, some commands will go into non-interactive mode automatically. If you’re not seeing commands prompt like they used to, like svn or passwd, you can return to the previous behavior by adding the following line to your capfile:

default_run_options[:pty] = true

Disable sh wrapping. Some shared hosts do not allow the POSIX shell to be used to execute arbitrary commands, which is what Capistrano has done since 2.0. If you’re on such a host, you can add the following line to your capfile:

default_run_options[:shell] = false

Capistrano will then run the command directly, rather than wrapping it in an “sh -c” command. Note, though, that this means that your own user shell on the remote hosts must be POSIX compatible, or you’ll get cryptic errors.

Git SCM support. Many thanks to Garry Dolley, Geoffrey Grosenbach, and Scott Chacon for their work on the new Git SCM module for Capistrano. If you’re a user of Git, you can now do:

set :scm, :git

Accurev SCM support. Thanks to Doug Barth, all you Accurev users can now enjoy Capistrano, too. Just do:

set :scm, :accurev

Rails’ Plugin Support. Capfile’s generated via the “capify” utility will now include a line that will autoload all recipes from vendor/plugins/*/recipes/*.rb. If you want this feature and you’ve already got a Capfile (and you don’t mind losing any changes you might have made to your Capfile), you can delete the Capfile and re-run “capify .”. Or, you can just add the following line to your Capfile, before the line that loads ‘config/deploy’:

Dir['vendor/plugins/*/recipes/*.rb'].each { |plugin| load(plugin) }

Windows-safe reads. Any time Capistrano needs to read a file’s contents, it will now use the “b” flag, so that binary reads on Windows do not corrupt the file.

Cap shell and sudo. The Capistrano shell now properly recognizes sudo commands and prompts for the password correctly.

Use `match’ to check dependencies. There is a new remote dependency method for deploy:check: “match”. You can now look for arbitrary regular expressions in the output of various commands to see if things are set up correctly:

depend :remote, :match, "rake -V", /version 0\.7/

Namespaces#top. Sometimes you’ll find yourself wanting to execute a task from within another task, but the parent namespace of the target task is conflicting with a similarly-named namespace, and things are breaking. You can now use the “top” method to jump to the top of the namespace hierarchy:

1
2
3
4
5
6
7
8

namespace :apache do
  namespace :deploy do
    task :restart do
      run "restart apache"
      top.deploy.restart
    end
  end
end

Other changes. There are lots of other, smaller bug fixes and changes, too:

• Default to 0664 instead of 0660 on upload.
• Fix deploy:pending to query SCM for the subsequent revision so that it does not include the last deployed change.
• Prefer ‘Last Changed Rev’ over ‘Revision’ when querying latest revision via Subversion.
• Explicitly require ‘stringio’ in copy_test.
• When Subversion#query_revision fails, give a more sane error.
• Don’t run the upgrade:revisions task on non-release servers.
• Use the—password switch for subversion by default, but add :scm_prefer_prompt variable for those who’d rather not send the password on the command-line.
• Use sudo -p switch to set sudo password prompt to something predictable.
• Allow independent configurations to require the same recipe file within the same Ruby process.
• Allow auth-caching of subversion credentials to be enabled via :scm_auth_cache.
• Don’t let a task trigger itself when used as the source for an “on” hook.
• Add version_dir, current_dir, and shared_dir variables for naming the directories used in deployment.
• Use the :runner variable to determine who to sudo as for deploy:restart.
• Change the “-h” output so that it does not say that “-q” is the default.

Enjoy! And please report any bugs on the Rails trac, with the component set to “Capistrano”.

I had intended to wait until Net::SSH v2 (and friends) were done before making another Capistrano release, but the list of pending tickets was growing and there is still a good-sized pile of work to be done on Net::SFTP v2 before it can be released. So, I sat down this weekend and worked through the tickets, and I’m happy to present Capistrano 2.1 Preview Release #1.

Grab it via RubyGems from the Rails beta gem repository:

gem install -s http://gems.rubyonrails.com capistrano

It will install as Capistrano 2.0.100.

Why a preview release? Well, there are a few changes in this release that might have far-reaching consequences, and I wanted to give people a chance to kick the tires and explore what the ramifications are without giving the impression that this is a generally-recommended production release. (I’m using it, but then, I might be a tad mentally unstable. Who knows?)

So, what are these new features and changes that have me so uncertain?

Feature: By default, Capistrano no longer requests that a pty be allocated for each command. This means that your default profile settings will be loaded automatically for every command. Yes! Yes! Your PATH tweaks and other environment variables will all be set as expected! However, because I did not ever document why I was requesting a pty in the first place, I do not know if there was a compelling reason why I was doing so. Thus, there may be unexpected side effects of this change. (I’ve not seen any such side-effects myself, but that doesn’t mean there won’t be.) If, for whatever reason, you do want a pty to be allocated for a particular command, just pass :pty => true to the run or sudo command. You can also use the default_run_options hash as described below to set it globally.

Feature: The subversion SCM module no longer uses the --password option to authenticate. Instead, it waits until the server prompts for the subversion password and send it that way. This is more secure (“ps” won’t show the password anymore), but I’m not entirely sure what the side-effects of this might be, so please report any subversion authentication anomalies.

Feature: If you are embedding Capistrano (in a Rails app or whatever), and you need to instantiate a Capistrano configuration more than once and use “require” to load a third-party recipe file, it should work now for every time you require the file. Please report any wonkiness with loading recipes via require.

Feature: The sudo password prompt is now detected much more reliably, thanks to the “-p” switch. My only concern is that somewhere out there (cough Solaris cough) there might be a sudo implementation that does not implement the -p switch for sudo. In that case, sudo via Capistrano will be utterly broken. Please let me know if you encounter a sudo that is broken this way.

Aside from the above, the following changes have also been made in this release:

Feature: If you use the “capify” script to generate your Capfile, it will now include a line that autoloads any recipe in vendor/plugins/*/recipes/*.rb. If you want this functionality, just delete your Capfile and rerun capify on your project directory.

Feature: If you’re on a shared host that does not let you spawn new shells, you can tell Capistrano to execute all commands directly, rather than via a spawned shell. Just pass :shell => false to the run command, or use the new default_run_options hash to set it globally:

1

default_run_options[:shell] = false

Note that if you use that setting, your default shell must be POSIX-compatible (and csh and tcsh, among others, are not POSIX-compatible).

Feature: There is a new “match” remote dependency method:

1

depend :remote, :match, "rake -V", /version 0\.7/

The given command will be executed on all hosts, and if the output does not match the given regular expression, the dependency will fail.

Feature: If you are using Subversion and you want your authorization credentials to be cached on the remote host, set :scm_auth_cache to true. (By default, authorization credentials are not cached on the remote hosts.)

Feature: There are now :version_dir, :current_dir, and :shared_dir variables in the deployment system that let you customize exactly what those deployment directories are called.

Feature: Reads now all open files in windows-safe (binary “rb”) mode.

Feature: Support for the Accurev SCM is now bundled with Capistrano.

Feature: The deploy:restart task honors the :runner variable when it uses sudo.

Feature: There is now a Namespace#top method for accessing the top-level namespace. This is useful when you want to execute a task in another namespace but you have an namespace between you and the top level that is interfering:

1
2
3
4
5
6
7
8
9
10
11

load 'deploy'

namespace :custom do
  namespace :deploy do
    task :something do
      # Have to do "top." because otherwise the bare reference to "deploy"
      # would find the current namespace.
      top.deploy.restart
    end
  end
end

I’m really pleased with how Net::SSH v2 is turning out. I’ve actually got it to a point where it’s about ready for a first preview release! If you’re an adventurous soul, you can grab the preview release gem (version 1.99.0) here:

• Net::SSH 2.0 preview #1 (1.99.0) gem

Note that Net::SSH v2 is not going to be 100% backwards compatible with 1.x. Many things will work mostly the same, but here are a few of the most significant gotchas:

• Net::SSH.start now uses a fixed parameter list, instead of the polymorphic monster it was. This makes it much easier to document, and use. The first parameter is always the host you are connecting to, and the second is always the username you’re connecting as. Options always come last.
• The shell, sync-shell, and open3 services have been dropped. You won’t find them in Net::SSH 2.0. If you want them, I strongly encourage you to write them and release them as third-party extensions to Net::SSH.

There are also a couple of (much needed!) short cuts for executing commands:

1
2
3
4
5
6

Net::SSH.start("localhost", "jamis") do |ssh|
  puts ssh.exec!("echo hello world!")

  ssh.exec("hello world")
  ssh.loop
end

The first, #exec!, will block until the command finishes, and will return the output. The second will return immediately, before executing the command, and will run when the event loop does, writing output to the terminal.

The documentation is pretty much done, too. To see it, just install the gem, change to the net-ssh-1.99.0 gem directory, and run “rake manual”. Then open “doc/out/index.html” in a browser. It includes a brief tutorial, a quick-reference, and an FAQ-style “how do I do X” interface.

So! Give it a try. Remember that the current release of Net::SFTP won’t work with this new version (at all), and neither will Capistrano. But the lovely thing about RubyGems is that you can easily install and uninstall, on demand.

Oh, and before I forget, a couple of little teasers about what’s coming up:

1
2
3
4
5
6
7
8
9
10

require 'net/scp'
Net::SCP.download("localhost", "jamis", "/Users/jamis/.bashrc", "./my-bash-rc") do |name, sent, total|
  if sent == 0
     print "downloading #{name} (#{total}b): "
  elsif sent == total
     puts
  else
     print "."
  end
end

and…

1
2
3
4

require 'uri/open-scp'
open("scp://jamis@localhost/Users/jamis/.bashrc") do |io| 
  puts io.read 
end