Transferring files

From ACCRE Wiki

Transferring files from your local computer to a cluster typically involves using file transfer protocols or tools. The exact method can depend on the file size and your preferences. Here are a few common methods:

Secure Copy (SCP)

scp command can be used to securely copy files from your local machine to the cluster or vice versa.

scp source vunetid@login.accre.vanderbilt.edu:destination

For more details on using SCP, see advanced setup section in here

Secure File Transfer Protocol (SFTP)

Similar to SCP, SFTP provides a secure way to transfer files between cluster and your local system. Use an SFTP client like WinSCP or FileZilla, or use the sftp command. We recommend you to use FileZilla if you prefer graphical interface. Here is an example of how the configuration on the FileZilla should look like :

rsync

rsync is a powerful command line tool for syncing files between your local machine and the cluster.

rsync -vz source vunetid@login.accre.vanderbilt.edu:destination

You can perform a dry run to see what rsync would do without actually making any changes. This is achieved using the --dry-run option.

Globus Connect Personal

The following steps walk you through the installation of Globus Connect Personal in your home folder, create a collection to ACCRE, and start the connection to a /data directory.

1. Login to the cluster ssh vunetid@login.accre.vu and install Globus Connect Personal in your home folder using the wget command below:

[bob@gw02 ~]$ wget https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz

--2025-12-10 10:04:09--  https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz
Resolving downloads.globus.org (downloads.globus.org)... 18.155.1.122, 18.155.1.53, 18.155.1.58, ...
Connecting to downloads.globus.org (downloads.globus.org)|18.155.1.122|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 137055277 (131M) [application/x-tar]
Saving to: ‘globusconnectpersonal-latest.tgz’

globusconnectpersonal-latest.tgz                100%[====================================================================================================>] 130.71M   175MB/s    in 0.7s

2025-12-10 10:04:10 (175 MB/s) - ‘globusconnectpersonal-latest.tgz’ saved [137055277/137055277]

2. Extract the files from the downloaded tarball which will create a versioned globusconnectpersonal:

[bob@gw02 ~]$ tar xzf globusconnectpersonal-latest.tgz

3. Go to the newly created directory, using the listed version number. You can 'ls' your home folder to see the version installed. In this example, golbusconnectpersonal-3.2.8 is installed, so we will change directories to this version:

[bob@gw02 ~]$ cd globusconnectpersonal-3.2.8
[bob@gw02 globusconnectpersonal-3.2.8]$

4. Next, we will run the setup command, copy the login link provided to your browser, name the Collection, and enter the auth code provided:

  • Run the setup command
[bob@gw02 globusconnectpersonal-3.2.8]$ ./globusconnectpersonal -setup
Starting CLI interactive setup

For advanced configuration or High Assurance options, cancel and review
  globusconnectpersonal -setup --help

Globus Connect Personal needs you to log in to continue the setup process.

We will display a login URL. Copy it into any browser and log in to get a
single-use code. Return to this command with the code to continue setup.

Login here: **Copy the link you see in your terminal like this example below**
-----
https://auth.globus.org/v2/oauth2/authorize?client_id=4d6448ae-8ca0-40e4-aaa9-8ec8e8320621&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&scope=openid+profile+urn%3Aglobus%3Aauth%3Ascope%3Aauth.globus.org%3Aview_identity_set+urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Agcp_install&state=_default&response_type=code&code_challenge=mx92F3_Ebs1PNQgN9E1unqqDeBIfEjqviPUox3nbpkI&code_challenge_method=S256&access_type=online&prefill_named_grant=gw02.vm.accre.vu
-----
Enter the auth code:
  • Enter link in your browser and select Vanderbilt University for organizational login.

  • Allow Globus Connect Personal Setup:

  • Copy the Authorization Code and enter the code back in your terminal session where it asks for the auth code:

Enter the auth code: NDF9oxFIvgelc0yUOrPWTh4hT6pSCK

== starting endpoint setup


Input a value for the Endpoint Name: ACCRE
registered new endpoint, id: 16522dcc-d5e4-11f0-8759-0eb0b913a0ab
setup completed successfully

5. Now that you have successfully created your endpoint, you will run the start command to the path you want to use for your ACCRE collection. In this example, I am connecting to /data/mcallister_group so you want to make sure you are using the full path as shown below. To find the full path, you can run ls -l in the terminal shown below and use that for your start up.

[bob@gw02 globusconnectpersonal-3.2.8]$ ls -l /data/mcallister_group
lrwxrwxrwx 1 root root 45 Oct 30  2024 /data/mcallister_group -> /panfs/accrepfs.vampire/data/mcallister_group
[bob@gw02 globusconnectpersonal-3.2.8]$ ./globusconnectpersonal -start -restrict-paths rw/panfs/accrepfs.vampire/data/mcallister_group

6. Login to https://app.globus.org/file-manager to view the status of your “ACCRE” collection. Click on File Manager, then click on 'search' next to collections and find your ACCRE collection. It should be showing as “green” to indicate the collection is connected. Select the collection from the file manager and verify access to the files. You are now ready to start transferring files to or from your ACCRE collection.