A few days back we needed to copy some files around in the production environment and the task was given to me. It was one of those rather simple tasks as in the email was some examples from a colleague how I should use scp to get those files copied across.
I have to admit the syntax was a bit obscure but in general it seemed harmless enough.
sftp -o IdentityFile=id_rsa_somekey someuser@somemachine.ourdomain.com
Indeed if I ran that command from my terminal it worked like a champ, well it did after they copied the id_rsa_somekey to our ~user/.ssh directory.
This was a great way to connect to the other machine to manually put a file but not all that great for batch copying. I discussed this with my colleague who agreed and suggested that we simply use scp instead – after all it is the same general protocol family.
So a script was born. It was an awesome script. It was a powerful script.
OPTIONS="-o IdentityFile=~batchuser/.ssh/id_rsa_somekey"
EXTSYS_USER=bob
EXTSYS_HOST=somemachine.ourdomain.com
EXTSYS_PUTDIR=/to_dropoff
scp $OPTIONS $1 $EXTSYS_USER@$EXTSYS_HOST:/$EXSYS_PUTDIR
Well, it at least did get the job done.
Well, this script actually didn’t get very far when Dave from accounting (wish it was accounting and not IT) decided rather than to fool around with using an internal service that works and has been paid for, we should simply connect directly to the vendor and put the file there instead.
It wasn’t the technical portion that was the issue for me but rather that John from support was asked to put this together instead of me. I watch John struggle with this for quite a few hours until he realized that it is just not possible to use a key in this manner in a batch script when it is protected by a passphrase.
Things got a lot easier for John once he decided to remove the passphrase.
echo put $1 > batch_file
sftp -b batch_file -o IdentityFile=~batchuser/.ssh/id_rsa_key -o Port=20022 -o PreferredAuthentications=publickey externaluser@machine.someotherdomain.com
I guess I am a bit of a snob. I think that the program directory should contain programs or at least static configuration files. John decided that he would simply create the batch_file in the current working directory.
I may be a snob, but other than that, Johns script was pretty good. I just kept wondering why he would do it this way instead of using secure copy. I tried it and despite working on our internal machines it failed when I did it to our vendor’s machine.
exec request failed on channel 0
exec request failed on channel 0
I did some research and it seems that the problem is more one of setup. The destination machine was probably setup to accept sftp but not ssh.
Well, it turns out that the scp command line tool in OpenSSH is implemented using the secure copy protocol which is implemented more like running secure shell commands. It is (apparently) possible to have sftp but not scp configured on your server.
I did learn a few things about sftp by watching John’s progress, but I did learn with certainty it is impossible to use a private key with a passphrase without entering it interactively. There are other programs such as ssh-agent which can help out but that is a topic for another day.
