Scientific Computing Division

File Transfers from/to

SCD Supercomputers

Siddhartha S Ghosh

Consulting Services Group

SCD/NCAR

Scientific Computing Division

Current Security PolicyExposed Network

UCARExposed

Hosts

gatekeeper

ServersOffices

Servers

MSS

Protected Network

RAS

VPNssh, scp, sftp

ftp, telnet, ssh, rcp/rlogin

ssh

Semi Exposed Network token + ssh

Supers

Data Analysis

Super Net

MSS

roy

token + ssh

sshscpsftp

Scientific Computing Division

Options

1. Invoke scp/sftp from Supers to remote Computers

2. Invoke ftp-proxy from Supers to remote Computers

3. Invoke scp from remote Computers to roy.ucar.edu

4. Invoke scp from Supers to salo-s.ucar.edu and a following scp from remote Computers to salo and vise-versa.

5. Stage into MSS and download using ftp/TLS to peewink.ucar.edu

Scientific Computing Division

Invoke scp/sftp from Supers to remote Computers• Most convenient, may be automated through scripts by

Installation of ssh-keys.• Secure

Ref: http://www.scd.ucar.edu/docs/ibm/scp.html

Caveats• Remote site must allow incoming ssh• Little computational overhead for encryption/decryption

Scientific Computing Division

Invoke ftp-proxy from Supers to remote Computers

• Issue ftp roy-s from Supers• At the proxy prompt give username@remote-host.remote-domain• Follow usual ftp syntax• Convenient for anonymous remote ftp access

• Caveats - Remote site should allow incoming active mode ftp connection - Password for remote account is transmitted in clear - Few reported cases of failures particularly when remote ftp server is behind a firewall

Scientific Computing Division

Invoke scp from remote Computers to roy.ucar.edu

• Scp files from remote computers tousername@roy.ucar.edu:supercomputer-name• Periodically roy transfers those files tousername@supercomputer:/ptmp/username/file• Some ssh key and other setup is required, Ref:https://www.scd.ucar.edu/docs/access/internal/inbound.html

• Caveats: - Only works for inbound transfers

Scientific Computing Division

Invoke scp from Supers to salo-s.ucar.edu following scp from remote Computers to salo and vice versa.

• Works when remote site too is hardened

• Transfer to salo (3rd machine)

username@salo.ucar.edu:ptmp

• Pick it up from salo within 2 hrs.

(Refer to the last section of previous reference)

• Requires account in salo, please email consult1@ucar.edu.

Scientific Computing Division

Stage into MSS and download using ftp/TLS to peewink.ucar.edu

• Convenient, if it needs to be archived in MSS anyway• May be used for transferring files in both ways.

Ref: http://www.scd.ucar.edu/docs/mss/ftp.html

Caveats

- Installation of ftp/TLS client is required, (usually

straightforward in Linux)

- MSS holdings and transfers incur charges.

Scientific Computing Division

Installation of ssh-keys in roy• This is described in the web page referred to earlier.• Create key in your remote computer: ssh-keygen –t rsa press <return> at passphrase prompts• Grab the .pub key using mouse• type “key roy” when prompted• type “key supercomputer-name” e.g. “key bluesky”

and type passwd for bluesky when prompted.• Once completed you can do: automated file-transfers (method-1) You only need to give One Time Crypto passwd

while logging into the supercomputers.

Scientific Computing Division

Conclusion

• All the options have little caveats attached

• Things will change for the better for the users in near future, we have been exploring different options

Questions ?