Buckblog: Preview Releases: Net::SSH and friends

Preview Releases: Net::SSH and friends

22 March 2008 — 2-minute read

After far, far too long, I’m extremely pleased to announce the next preview release of Net::SSH v2 (preview release #2, actual version 1.99.1), as well as the first ever preview releases of Net::SFTP v2 (preview release #1, actual version 1.99.0) and Net::SCP v1 (preview release #1, actual version 0.99.0). You can grab them from my gem server:

123	$ sudo gem install --source http://gems.jamisbuck.org net-ssh$ sudo gem install --source http://gems.jamisbuck.org net-sftp$ sudo gem install --source http://gems.jamisbuck.org net-scp

I’ve taken Eric Hodel’s advice this time around and ditched the pretty documentation I had started for Net::SSH; I’m going to see how far I can get with ri and rdoc alone, this time. Give it a try after installing the above: “ri Net::SSH”, for instance.

There are several caveats about this release:

1. The new Net::SSH and Net::SFTP are not backwards compatible with the previous 1.x releases, so make sure any existing scripts are explicitly depending on versions less than 1.99.0 of these libraries. (The latest Capistrano already has that explicit dependency in it, so you should be safe installing these preview releases even if you are using Capistrano).

2. The “shell”, “open3” and “sync-shell” services are gone from Net::SSH. (There may be others, too, which aren’t coming to mind at the moment.) I do not intend to bring them back. If you used those services, I strongly encourage you to reimplement them for the new Net::SSH, and release them as third-party extensions.

3. These libraries are reentrant (or ought to be), but are not thread-safe. That is to say, if you are accessing the same Net::SSH connection from multiple threads, you’re definitely going to want to do something to protect that resource (mutexes, etc.).

That said, let me know what you think, what issues you have, what suggestions you have, and so forth. I’m so happy to have this release done, though. I can see the light at the end of the tunnel finally!

Oh, and lastly, if you happen to be drinking the git kool-aid, you can follow along via github:

Net::SSH: http://github.com/jamis/net-ssh/tree/master
Net::SFTP: http://github.com/jamis/net-sftp/tree/master
Net::SCP: http://github.com/jamis/net-scp/tree/master

Reader Comments

Nice work Jamis, I’ll be playing with the new net/ssh this week.

Ezra
23 Mar 2008

I thought by definition (at least according to http://en.wikipedia.org/wiki/Reentrant) reentrant code was also thread-safe.

Brian
24 Mar 2008

@Brian, you’re right. I’m not an expert on concurrency, and misused the term. What I meant was, you can call SSH methods from themselves (e.g., from within callbacks and such), with no negative consequences. But you cannot safely call methods on the same connection from multiple threads of execution without using mutexes or similar.

Jamis
24 Mar 2008

I’ve found (and fixed) a bug in Net::SSH v2 preview #2 that caused forwarding a local port to fail with an exception. You can get the update via github, if you don’t want to wait for preview #3. I’m working on plugging Net::SSH v2 and friends into capistrano now, so we’ll see what else I bump against this week. I’ll then bundle all the fixes into another set of preview releases at the end of the week.

Jamis
24 Mar 2008

Amazing stuff, Jamis! Really beautiful looking code as well. Two quick questions: 1) Given your comments about threadsafe-ness, how should a separate thread push data into the channel? I’ve been using the ‘channel.on_process’ callback as a way to periodically inject data into my outbound stream, but I’ve got a select() in there, which feels a little dirty; 2) On Windows XP the require will bomb unless the HOME environment variable is set, could you allow it to failover to HOMEPATH (which apparently is set by default)? Thanks again!

Steve Shreeve
25 Mar 2008

@Steve, for #1, I did the following (more or less) with Capistrano’s gateway class, which runs in a thread but must allow other threads to begin connections through the gateway:

require 'thread'

@mutex = Mutex.new
Thread.new do
  loop do
    @mutex.synchronize { @gateway.process(0.1) }
  end
end

@mutex.synchronize do
  @gateway.forward.local(1234, "remote-host", 22)
end

c = Net::SSH.start("localhost", "user", :port => 1234)

In other words, run the event loop manually (by looping and calling Net::SSH::Connection::Session#process manually), and wrap the #process call in a mutex. Then, any time you need to access the session outside of a thread, employ the mutex again.

As for #2, which “require” is failing?

Jamis
25 Mar 2008

Thank you for the very helpful tips!

Regarding #1, I’m now using something similar to what you suggested.

Regarding #2, I drilled down a bit and it’s actually the reference to “_{/.ssh/config” on line 35 of net/ssh/config.rb. This causes Ruby on Windows to complain when running Net::SSH.start() because the “}” expects an environment variable called “HOME”, which is not defined by default (at least on Windows XP). Assuming this isn’t too terribly dirty, might this be inserted immediately after line 35 in config.rb?

ENV[‘HOME’] ||= ENV[‘USERPROFILE’] if RUBY_PLATFORM.include?(‘win32’)

Steve Shreeve
25 Mar 2008

Jamis,

Thanks for your hard work on this and Capistrano, it is much appreciated.

Would you have any idea when we could hope to see a Cap preview release with these new clients built in? The main reason I am interested is because of the sftp / scp issue that leaves 0 length files on the target host, but doesn’t actually copy when connected through a gateway. From what I could gleam on the mailinglist this was something that these new libraries would resolve. (I could of course be very wrong :))

Regardless, thanks again for all the effort you put into this.

Thomas
8 Apr 2008

Jamis, I’m seeing something strange. I have passwordless ssh set up between several machines. If the username on the remote machine is the same as my local one, it works. However, if the remote username is different, I get a auth failure.

I verified that this works in the older packages:

$ whoami thudson $ irb -rubygems irb(main):001:0> gem ‘net-ssh’, ’=1.1.2’ =>true irb(main):002:0> require ‘net/ssh’ => true irb(main):003:0> Net::SSH.start(‘test-app04’,’wlnext’)

This succeeds, but using 1.99.2: $ whoami thudson $ irb -rubygems irb(main):001:0> require ‘net/ssh’ =>true irb(main):002:0> => true irb(main):003:0> Net::SSH.start(‘test-app04’,’wlnext’) Net::SSH::AuthenticationFailed: wlnext from /usr/local/lib/ruby/gems/1.8/gems/net-ssh-1.99.2/lib/net/ssh.rb:178:in `start’ from (irb):2

Any ideas?

Tomy
10 Apr 2008

The Buckblog

assorted ramblings by Jamis Buck

Preview Releases: Net::SSH and friends

Reader Comments