Difference: CommandLineBox (1 vs. 13)

Revision 132024-05-18 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

If you use SSO (through UT Box), you need to first set up an external password.

Changed:
<
<
You will need LFTP installed on your system. It is already installed on TACC
>
>
You will need LFTP installed on your system. It is already installed on TACC If you are on a different system, you can install it using Conda.
  Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user YYY@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories TO BOX, you will need this command:

Changed:
<
<
mirror -R {local dir} {remote dir}
>
>
mirror -R {local path} {remote path}
 

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)
Changed:
<
<
Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.
>
>
Note: lftp may linked/compiled on some TACC servers such that you need an extra module loaded (i.e., Maverick2). Try this first if the above does not work.
 
module load gcc 
Changed:
<
<
For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:
>
>
To recursively copy directories FROM BOX, you will need this command:
Added:
>
>
 
Changed:
<
<
glob pget /.gz
>
>
mirror {remote_path} [local_path]
 
Deleted:
<
<
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.
 
Changed:
<
<
To recursively copy directories FROM BOX, you will need this command:
>
>
This website contains useful examples for both mirroring to and from a remote location
 
Added:
>
>
For copying all files in subdirectories (but not the directories themselves) from Box to TACC the following command will work:
 
Changed:
<
<
mirror
>
>
glob pget /.gz
 
Changed:
<
<
This website contains useful examples for both mirroring to and from remote location
>
>
Various changed to the glob pattern can be used to retrieve other files.
 

Creating a Bookmark File

This only works if you are UT-affiliated.

First, follow these instructions to set up an external password.

Next, create a bookmark called utbox.

Security Note: this saves your external password in clear text in your home directory.

lftp ftps://ftp.box.com:990
set bmk:save-passwords yes
user [your_EID]@eid.utexas.edu
[Enter Password]
bookmark add utbox
quit

Then, you can log in using just one command!

lftp utbox

Be sure to test your setup after logging in by running a command like `ls` because it will not give you an error if login failed!

Changed:
<
<
It should list all of your Box file.
>
>
It should list all of your main Box files.
  This is also the standard way that we make it possible to mass download data as part of brefito workflows.

Revision 122024-04-18 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

If you use SSO (through UT Box), you need to first set up an external password.

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user YYY@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories TO BOX, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.

To recursively copy directories FROM BOX, you will need this command:

mirror 
This website contains useful examples for both mirroring to and from remote location

Creating a Bookmark File

Changed:
<
<
This only works if you are UT-affiliated. You need to first set up an external password.
>
>
This only works if you are UT-affiliated.
 
Added:
>
>
First, follow these instructions to set up an external password.
 Next, create a bookmark called utbox.

Security Note: this saves your external password in clear text in your home directory.

lftp ftps://ftp.box.com:990
set bmk:save-passwords yes
Changed:
<
<
user YYY@XXX.utexas.edu
>
>
user [your_EID]@eid.utexas.edu
 [Enter Password] bookmark add utbox
Added:
>
>
quit
 

Then, you can log in using just one command!

lftp utbox
Added:
>
>
Be sure to test your setup after logging in by running a command like `ls` because it will not give you an error if login failed!

It should list all of your Box file.

  This is also the standard way that we make it possible to mass download data as part of brefito workflows.

Revision 112024-04-13 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Changed:
<
<
If you use SSO (through UT Box), you need to first [[https://support.box.com/hc/en-us/articles/360043694574-Box-SSO-Working-with-External-Passwords][set up an external password].
>
>
If you use SSO (through UT Box), you need to first set up an external password.
 
Deleted:
<
<
If you log into Box another way, you should be able to use your normal Box password.
 You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user YYY@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories TO BOX, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.

To recursively copy directories FROM BOX, you will need this command:

mirror 
This website contains useful examples for both mirroring to and from remote location

Creating a Bookmark File

Changed:
<
<
Create the bookmark. Note: this saves your password in clear text in your home directory.
>
>
This only works if you are UT-affiliated. You need to first set up an external password.
 
Added:
>
>
Next, create a bookmark called utbox.

Security Note: this saves your external password in clear text in your home directory.

 
lftp ftps://ftp.box.com:990
set bmk:save-passwords yes
user YYY@XXX.utexas.edu
[Enter Password]
bookmark add utbox
Changed:
<
<
Then, you can login using just one command:
>
>
Then, you can log in using just one command!
 
lftp utbox
Added:
>
>
This is also the standard way that we make it possible to mass download data as part of brefito workflows.
 

Revision 102024-01-11 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Changed:
<
<
Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux
>
>
If you use SSO (through UT Box), you need to first [[https://support.box.com/hc/en-us/articles/360043694574-Box-SSO-Working-with-External-Passwords][set up an external password].
Added:
>
>
If you log into Box another way, you should be able to use your normal Box password.
  You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user YYY@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories TO BOX, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.

To recursively copy directories FROM BOX, you will need this command:

mirror 
This website contains useful examples for both mirroring to and from remote location

Creating a Bookmark File

Create the bookmark. Note: this saves your password in clear text in your home directory.

lftp ftps://ftp.box.com:990
set bmk:save-passwords yes
user YYY@XXX.utexas.edu
[Enter Password]
bookmark add utbox

Then, you can login using just one command:

lftp utbox

Revision 92023-12-25 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
Changed:
<
<
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu
>
>
lftp ftp.box.com:~> user YYY@XXX.utexas.edu
 

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories TO BOX, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.

To recursively copy directories FROM BOX, you will need this command:

mirror 
This website contains useful examples for both mirroring to and from remote location
Added:
>
>

Creating a Bookmark File

Create the bookmark. Note: this saves your password in clear text in your home directory.

lftp ftps://ftp.box.com:990
set bmk:save-passwords yes
user YYY@XXX.utexas.edu
[Enter Password]
bookmark add utbox

Then, you can login using just one command:

lftp utbox
 

Revision 82023-03-08 - DanielDeatherage

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.
Changed:
<
<
To recursively copy directories TO TACC, you will need this command:
>
>
To recursively copy directories TO BOX, you will need this command:
 
mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.
Changed:
<
<
To recursively copy directories FROM TACC, you will need this command:
>
>
To recursively copy directories FROM BOX, you will need this command:
 
mirror 
This website contains useful examples for both mirroring to and from remote location

Revision 72023-03-07 - DanielDeatherage

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.
Changed:
<
<
To recursively copy directories, you will need this command:
>
>
To recursively copy directories TO TACC, you will need this command:
 
mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.
Added:
>
>
To recursively copy directories FROM TACC, you will need this command:

mirror 
This website contains useful examples for both mirroring to and from remote location
 

Revision 62020-01-24 - MattMcGuffie

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)
Added:
>
>

Note: Some TACC servers may not be compiled correctly and might need an extra module loaded (ie Maverick2). Try this in if the above does not work.

module load gcc 
 

For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:

glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.

Revision 52020-01-06 - DanielDeatherage

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true
Changed:
<
<
Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.)
>
>
Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.) The user name is the one listed in the account tab of the account settings page on UTBOX.
 
lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Here are some useful commands for working with your local directory from within the LFTP shell:

  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.

To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}

This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.

Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!

mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)
Added:
>
>
For copying all files in subdirectories (but not the directories themselves) from Box to tacc the following may work well:
glob pget */*.gz
Note: the above will result in a fairly quick determination and changing message of what is to be transferred followed by a lengthy file transfer which will result in messages such as pget-chunk: B-A1/B-A1_R1.fastq.gz: seek failed for each file once each file transferred.
 

Revision 42019-08-17 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.)

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

Changed:
<
<
One useful thing is that you can run a command on your local machine by adding an exclamation point before the command. For example !ls lets you list the contents of your local directory.
>
>
Here are some useful commands for working with your local directory from within the LFTP shell:
Added:
>
>
  • lcd lets you change local directories.
  • !ls lets you list the contents of your local directory.
  To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}
Changed:
<
<
This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync.
>
>
This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync, but be very careful when doing so! You can use the --dry-run option to see what would have been deleted before running it.
Added:
>
>
Note: Sometimes you may get an error looking like this during a mirror -R call. The file has been successfully transferred despite this message!
mirror: Access failed: 550 Box: Item with the same name already exists (code=item_name_in_use, request_id=XXXXXXXXXXXXXXX). (FILENAME)
 

Revision 32019-08-15 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting:
https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.)

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

One useful thing is that you can run a command on your local machine by adding an exclamation point before the command. For example !ls lets you list the contents of your local directory.

To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}
Added:
>
>
This does not delete a file from {remote dir} if it has been deleted in {local dir} since the last backup. You would need to add the --delete flag for that type of sync.
 

Revision 22019-08-14 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Changed:
<
<
Follow the directions here to set up an additional Box password that you will use for connecting:
>
>
Follow the directions here to set up an additional Box password that you will use for connecting:
 https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Changed:
<
<
Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. This step is, strictly speaking, optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)
>
>
Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. Strictly speaking, this step is optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)
  Add these lines to the ~/.lftp/rc file.
set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.)

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu
Changed:
<
<
LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.
>
>
LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.
  One useful thing is that you can run a command on your local machine by adding an exclamation point before the command. For example !ls lets you list the contents of your local directory.

To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}

Revision 12019-08-14 - JeffreyBarrick

 
META TOPICPARENT name="ComputationList"

Copying files to/from UT Box at the command line

Follow the directions here to set up an additional Box password that you will use for connecting: https://uisapp2.iu.edu/confluence-prd/display/SOICKB/Using+Box+under+Linux

You will need LFTP installed on your system. It is already installed on TACC

Open the file ~/.lftp/rc in a text editor. (You may need to first create the ~/.lftp folder if it does not exist. This step is, strictly speaking, optional. You could run these commands every time you enter the LFTP shell to temporarily set the options.)

Add these lines to the ~/.lftp/rc file.

set ftps:initial-prot ""
set ftp:ssl-force true
set ftp:ssl-protect-data true

Now you can use these commands to connect to Box (The second one is run inside the LFTP shell and will prompt you for the password you created.)

lftp ftps://ftp.box.com:990
lftp ftp.box.com:~> user XXXX@XXX.utexas.edu

LFTP commands are a dialect of FTP commands. If you are not familiar with these, here is a reference.

One useful thing is that you can run a command on your local machine by adding an exclamation point before the command. For example !ls lets you list the contents of your local directory.

To recursively copy directories, you will need this command:

mirror -R {local dir} {remote dir}
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright ©2025 Barrick Lab contributing authors. Ideas, requests, problems? Send feedback