Net::SSH 2.0 preview #1
I’m really pleased with how Net::SSH v2 is turning out. I’ve actually got it to a point where it’s about ready for a first preview release! If you’re an adventurous soul, you can grab the preview release gem (version 1.99.0) here:
Note that Net::SSH v2 is not going to be 100% backwards compatible with 1.x. Many things will work mostly the same, but here are a few of the most significant gotchas:
-
Net::SSH.startnow uses a fixed parameter list, instead of the polymorphic monster it was. This makes it much easier to document, and use. The first parameter is always the host you are connecting to, and the second is always the username you’re connecting as. Options always come last. - The shell, sync-shell, and open3 services have been dropped. You won’t find them in Net::SSH 2.0. If you want them, I strongly encourage you to write them and release them as third-party extensions to Net::SSH.
There are also a couple of (much needed!) short cuts for executing commands:
1 2 3 4 5 6 |
Net::SSH.start("localhost", "jamis") do |ssh| puts ssh.exec!("echo hello world!") ssh.exec("hello world") ssh.loop end |
The first, #exec!, will block until the command finishes, and will return the output. The second will return immediately, before executing the command, and will run when the event loop does, writing output to the terminal.
The documentation is pretty much done, too. To see it, just install the gem, change to the net-ssh-1.99.0 gem directory, and run “rake manual”. Then open “doc/out/index.html” in a browser. It includes a brief tutorial, a quick-reference, and an FAQ-style “how do I do X” interface.
So! Give it a try. Remember that the current release of Net::SFTP won’t work with this new version (at all), and neither will Capistrano. But the lovely thing about RubyGems is that you can easily install and uninstall, on demand.
Oh, and before I forget, a couple of little teasers about what’s coming up:
1 2 3 4 5 6 7 8 9 10 |
require 'net/scp' Net::SCP.download("localhost", "jamis", "/Users/jamis/.bashrc", "./my-bash-rc") do |name, sent, total| if sent == 0 print "downloading #{name} (#{total}b): " elsif sent == total puts else print "." end end |
and…
1 2 3 4 |
require 'uri/open-scp' open("scp://jamis@localhost/Users/jamis/.bashrc") do |io| puts io.read end |
Reader Comments
I recently started using Capistrano, which of course requires Net::SSH. I was surprised when it did not honor the settings in my .ssh/config file. It took me a while to realize that that was the problem I was having. While I was able to add :port and :user options in the deploy.rb file, it seemed like it should have been DRYer, pulling that info in from my existing SSH config file.
Other than that minor quibble, great work. I love your code, as well as your blog.
21 Aug 2007
I suppose I’m not opposed to having .ssh/config support, but I’m not the one to write it, since I’ve never needed it. If someone else were to write a parser for the ssh config file, and then patch that into Net::SSH, I’d consider applying it.
21 Aug 2007
There is no rakefile.
Why isn’t the documentation just RDoc? This way you can launch gem server and see everything with all the rest of your gem documentation, and you get online help with ri.
Also, you don’t have to make people figure out where their gem dir is then run rake as sudo inside it.
21 Aug 2007
Bummer, sorry about that Eric. Apparently I hadn’t configured the gem build correctly. You can grab the Rakefile right here.
As for RDoc, if you can show me an easy way to insert syntax highlighted code snippets in an RDoc file, I’ll consider it.
Lastly, if you don’t know where your gem dir is, you aren’t ready to be running preview-release software.
Frankly, I know this is a rough release. It’s not “done” yet. Expect warts. Expect bugs. Expect holes in the docs. If that’s not what you’re up for, then this isn’t for you.
21 Aug 2007
Also, I should state, there is rdoc documentation with net-ssh. Feel free to “rake api” to your heart’s content. I intend to flesh out a real README file before release as well. HOWEVER. I have yet to see real rdoc documentation that “works” as a gentle introduction to a project. RDoc is great for building API documentation. Not so great for building tutorials and “how-to” style manuals. Prove me wrong, please.
21 Aug 2007
I noticed that your API documentation seems duplicated between the RDoc and the doc/ dir, but is subtly different between the two, which one is right?
It is not currently possible to do syntax highlighted code in RDoc as it is too smart/dumb to let you.
I have been adding a quick-start guide to my top-level classes. For example `ri Sphincter` will tell you everything you need to know to get started with pointers to documentation elsewhere for the fancy bits.
I’ve been doing the same for RubyGems, adding RDoc to appropriate classes as I need to use them. Not having to go to the web is immensely helpful.
23 Aug 2007
Eric, in what way does the documentation differ between the pretty docs and the rdoc api docs? If you can give me some specific examples, I’ll try and make sure the two don’t disagree.
23 Aug 2007
Everything in the quickref is different from the RDoc in the implementation. Some methods seem to not exist, others have different documentation.
Also:
class Net::SSH::Connection::Session
is much preferred to all those semicolons.
Also:
No comments feed?
23 Aug 2007
Eric, right at the top of the quickref is the following paragraph:
All that documentation (the FAQ’s, the quickref, etc.) is only intended to help people get up and running with Net::SSH. It is not intended as an exhaustive reference of all possible methods and options—that’s what the rdoc is for. Obviously, this documentation will not be helpful for you, but from support emails I’ve received and issues I’ve dealt with in the past, the simpler and more direct you can make documentation, the more it aids people who are new to your library.
Also:
Only works if Net::SSH::Connection has been declared previously as a module.
Also:
Right, no comments feed.
23 Aug 2007
Sorry, I got confused between the erb and RDoc and thought I saw methods that didn’t exist.
I don’t see why the quickref and RDoc are different.
I don’t see why ri Net::SSH is empty. A link to the quickref, at least.
24 Aug 2007
Eric, if you honestly don’t understand why the quickref is a subset of the rdoc, I don’t think I’m going to have much success explaining it, but I’ll try.
Some programmers are exceptionally comfortable with learning a new library from a handful of examples and the API documentation. You are one such programmer. Drinking from the proverbial fire-hose doesn’t faze you particularly. Many other programmers (in my experience) prefer to take things in diluted form, at least initially. For example: a tutorial that walks you through just a few simple operations, and hints at more complex possibilities. Or a short list of “frequently asked questions” which in no way includes every possible question a user of the library might ever ask. Or a quickref page that includes just a few of the most common API’s. The API docs themselves are right there too, available for them when they encounter a problem that the “gentler” docs don’t provide any help for.
As for “I don’t see why ri Net::SSH is empty”, it’s because (and I’ll say it just one more time) “I’m not done yet.” Give me some time, please, Eric. This is just preview release #1, not the final package.
24 Aug 2007
I don’t understand why the words per method are different between quickref and RDoc.
For Net::SSH::Connection::Session#exec you have in the quickref:
Queues the command for execution and returns immediately. Requires an event loop (see #loop or #process) in order for the command to actually be executed. If a block is given, output is yielded to the block (three parameters, channel, type, and data). Otherwise, output is written to the terminal.
In the RDoc:
A convenience method for executing a command and interacting with it. If no block is given, all output printed via $stdout and $stderr. Otherwise, the block is called for each data and extended data packet, with three arguments: the channel object, a symbol indicating the data type (:stdout or :stderr), and the data (as a string).
Note that this method returns immediately, and requires an event loop (see Session#loop) in order for the command to actually execute.
24 Aug 2007
Hello Jamis, I am trying out the new stuff. When trying to transfer multiple files with a wildcard to Net::SCP, the process hangs for me. For a single (no wildcard) transfer, it works fine.
e.g. Net::SCP.download(TARGET_IP, “admin”, ”/data/db/new_items.csv”, ”./fromRemote”, :ssh => sshParms = {:password => ‘xxxxxxx’}) works fine, but Net::SCP.download(TARGET_IP, “admin”, ”/data/db/new_*.csv”, ”./fromRemote”, :ssh => sshParms = {:password => ‘xxxxxxx’}) does not. It transfers one file then hangs.
Should this work with a wildcard in the value for ‘remote’ ?
Thanks, —Mike Berrow
3 Sep 2007
Interesting Mike, I’d never tried that case. I’ll make a note to investigate it before release.
3 Sep 2007
Hi Jamis,
This is much cleaner – nice job.
We have some older Fedora Core 4 boxes here that the examples above won’t connect with, but it works with Macs, etc. Assuming that you cared about being compatible with older OSes, I’d be glad to get you and debugging if it would be useful to you. Otherwise, if you are still working on this part, just disregard.
Keep up the good work!
11 Sep 2007
Hi Jamis
I’m just trying out the new gem. It looks pretty good so far, aside from an apparent problem connecting to SunOS boxes (though that might be a problem with the sun boxes rather than net-ssh—I’m investigating.)
My question is a little simpler. When I open up a login shell via send_channel_request, how do I ‘return’ control to the loop? It seems to hang after displaying the command output.
24 Sep 2007
Hey Jamis,
I’m loving the new cleaner SSH, but what’s the status on SCP now, is there a preview release or a repository I can check out?
9 Oct 2007
@namelessjon, I’m not sure I follow? Can you pastie the code?
@Jamie, SCP is close, I’m just finding it hard to make time to work on it all. You can find the repo here: http://svn.jamisbuck.org/projects/net-scp.
10 Oct 2007
I’m unable to do an SVN checkout of either the net-scp or net-ssh repository. I just see a 400 BAD REQUEST message:
svn: REPORT request failed on ’/projects/!svn/vcc/default’ svn: REPORT of ’/projects/!svn/vcc/default’: 400 Bad Request (http://svn.jamisbuck.org)
If I try to use the svn:// scheme, it just hangs.
12 Oct 2007
@Jason, very odd, it works here. The svn:// scheme is not supported with my svn configuration, only http:. I wonder if you just hit it at a bad time or something?
15 Oct 2007
The code from the FAQ ‘How do I request that a login shell be started?’ shows problem I’m having (or perhaps just a misunderstanding.)
After the output from the ls command (or whichever) is displayed, the script will then just wait, with the only way to exit being to press ^C. (It needs to be a login shell to set the enviroment right for certain commands I would like to run)
What I would like to be able to do is send some command, act on the results and then send another etc and then have it exit cleanly at the end …
16 Oct 2007
@namelessjon, which FAQ is that? Can you show me the URL where that FAQ exists?
Note that when you are interacting with a login shell, you are no longer able to use the “exec” method to invoke commands, since there is already a command running: the login shell. Instead, you need to implement an “expect”-like event loop, that watches the output from the shell (via the on_data and on_extended_data channel callbacks) and responds by sending data to the channel (via Channel#send_data). Working programmatically with a login shell is non-trivial, for the general case. :(
16 Oct 2007
I think it’s the version of SVN I have running on this PowerBook. I tried it again, and I still receive the same message. But it works from a Gentoo Linux machine with an older version. Go figure…
16 Oct 2007
Hi Jamis,
I’m running into the following error:
NoMethodError: undefined method ‘join’ for 10.10.10.10
method process_cache_miss in strict.rb at line 43 method verify in strict.rb at line 37 method verify in lenient.rb at line 15 method verify_server_key in diffie_hellman_group1_sha1.rb at line 165 method exchange_keys in diffie_hellman_group1_sha1.rb at line 68 method exchange_keys in algorithms.rb at line 331 method proceed! in algorithms.rb at line 172 method send_kexinit in algorithms.rb at line 163 method accept_kexinit in algorithms.rb at line 118 method poll_message in session.rb at line 168 method poll_message in session.rb at line 146 method wait in session.rb at line 183 method wait in session.rb at line 181 method initialize in session.rb at line 74 method start in ssh.rb at line 55 at top level in untitled document at line 3
I think, host is always a String in process_cache_miss, so the join can be removed from:
If I remove it, I get the expected error.
Thanks!
16 Oct 2007
Good catch, thanks Mike!
16 Oct 2007
Nothing online. The FAQ I was refering to is the one that is built in the doc directory for the 1.99.0 gem. (on my system: ${GEMS_DIR}/net-ssh-1.99.0/doc/out/faq/login_shell.html ) Generally I’ve found it pretty useful.
Anyway, thank you for the pointers about the ‘expect’ loop. Now to implement it … :)
17 Oct 2007
Hi again Jamis,
I am apparently working with a SSH server that is a little buggy. In tearing down the ssh session it sends a channel request for exit-status, then a channel end of file and then a channel close. The problem is that immediately after that, in some cases, it sends another channel request for exit status on the channel that is already closed. This causes your new version of net-ssh to throw an exception (the old version seemed to ignore this):
The script I’m working with is pretty simple:
I get the following debugs during session teardown:
I see this same behavior when using an OpenSSH client as well (however, it simply ignores the extra channel request) so I’m pretty sure it’s the ssh server that is at fault here. I’m going to work at getting the ssh server fixed because I don’t think it should be sending anything on the channel after both sides have declared the channel closed. However, it might be worthwhile for Net::SSH v2 to simply ignore packets for channels that don’t exist.
Thanks,
Mike
P.S. If there is a more appropriate place to report this stuff please let me know.
24 Oct 2007
Hi Jamis,
Reposting since I didn’t see my last attempt get posted.
I am apparently working with a buggy SSH server which on occasion sends a channel_request for exit-status after both sides have declared the channel closed. This causes Net::SSH v2 to throw an exception for:
The old version of Net::SSH doesn’t seem to get tripped up like this. The script I’m working with is pretty simple:
Here are debugs showing the errant behavior of the server:
I’m working to get the bug corrected in the server, but it might be worthwhile to simply ignore packets for channels that don’t exist.
Thanks!
Mike
24 Oct 2007
Hi again Jamis,
I am working with a SSH server that is buggy. The server sends a channel end of file then a channel close. Net::SSH version 2 sends it’s own channel close, but then the server incorrectly sends a channel request for exit-status after the channel is already considered closed by Net::SSH version 2. This results in an exception in Net::SSH version 2 when trying to dispatch the packet from a channel that no longer exists.
Obviously I need to get the server fixed, but other clients (including Net::SSH version 1) just ignore this extra packet and some print out a warning or debug message about it.
Thanks!
25 Oct 2007
Ack. Sorry for the duplicate comments. Seems the comment system had a huge (three day) lag for posting the original comment(s).
26 Oct 2007