College of Science Z:drive to OneDrive using Rclone

Tags Storage

Use in rclone on the CoS Shell server to sync data from your Z:drive to your MS OneDrive. This will require you to have rclone installed on your local system with a web browser to finish the rclone config process on the shell server. The rclone binary can be downloaded here

The current instructions that are provided at rclone site for installing rlcone on a remote host do not work. The issue is the rclone command that you need to run on your local system to obtain the token key doesn't work. Instead, we will install rclone on your local system with a web browser. That will still generate the token key that you can then use for the rclone config on the remote host.
Steps to configure rclone on local system for OneDrive. NOTE: you must run the rclone config command in a terminal window like PowerShell in Windows or a Terminal window in Mac/Linux.

1. run 'rclone config' command in a terminal window.

2. Select 'n' for new remote

3. Enter name for the remote site, ex. onedrive

4. Select the corresponding number for 'Microsoft OneDrive' from the list.

5. Leave 'client_id' blank

6. Leave 'client_secret' blank

7. For Option region, select '1 Microsoft Cloud Global'

8. Answer 'No' for Edit advanced config?

9. Answer 'Yes' for use web browser to automatically authenticate rclone with remote? This will launch your local web browser to log in to your OneDrive account. If successful, this will display a simple web page that says "Success".

10. Back at the rclone config prompt it should now be asking for "Type of connection". Choose "OneDrive Personal or Business"

11. Next it will show the one drive found, select it.

12. It should find the drive "root" of type "business" and it will show the URL. Select Yes:

 URL: https://oregonstateuniversity-my.sharepoint.com/personal/ONID_oregonstate_edu/Documents

13. It will now display the Token key. You will need to copy this with your mouse. Select Yes.

 {"access_token":"{LONG_TOKEN_KEY_HERE","expiry":"2024-05-31T09:18:52.00658-07:00"}

14. It will now list the new share and type. Type 'q' to exit out of the rclone config command.

Now login to the shell.science.o.e server. Depending on the amount of data in your Z:drive the process to sync the data up to OneDrive could take several hours if looking at 10's of Gigs of data or more. We recommend using the UNIX command 'screen' to start a separate process to run the rclone process. This will allow you to start the rclone process, detach from the screen session, and log out. When you return you can easily reconnect to your screen session to check the process of the rclone sync. But first, we now need to configure rclone on the shell science server.

Steps to configure rclone for OneDrive on shell.science.o.e server.

1. log into the shell server (shell.science.oregonstate.edu).

2. run 'rclone config'

3. Select 'n' for new remote

4. Enter name for the remote site, ex. onedrive

5. Select the corresponding number for 'Microsoft OneDrive' from the list.

6. Leave 'client_id' blank

7. Leave 'client_secret' blank

8. For Option region, select '1 Microsoft Cloud Global'

9. Answer 'No' for Edit advanced config?

10. Answer 'No' for use web browser to automatically authenticate rclone with remote?

   This will give you the command to run on your local system with a web browser to run with rclone
   Ex: rclone authorize "onedrive" "eyJkZXNjcmlwdGlsdfadsf6IiJ9"

Ignore that command to run as you already have the token key copied with your mouse above. Paste in the token key here and hit return.

11. It should now ask for "Type of connection". Select the "OneDrive Personal or Business".

12. It should show you one drive, select it.

13. It should find "root" or type "business". Select "Yes" default.

14. It will now ask if you want to keep this remote. Answer "Yes"

15. It will now list the new share. Type 'q' to exit out of the rclone config command.


You can now test to see if rclone works. Type in the following command:

 rclone ls onedrive:/

This should start to show the contents in your OneDrive. You are now ready to sync your Z:drive data to your OneDrive.
Here depending on how much data you have you might want to run the rclone sync inside the screen session mentioned above. To start a screen session, simply type "screen -S rclone" command at the prompt. It will start a new named session 'rclone' in your terminal and clear the screen. Now you can run the following command to start copying your Z:drive up to your OneDrive.

 rclone sync ./ onedrive:/zdrive --exclude ".*" --progess

This will start to copy your Z:drive to your OneDrive under the folder "zdrive" and display the progress stats. At this point if you did start a screen session you can type the keys 'ctrl-a d' to detach from the session. It will keep running. You can list the screen sessions by typing 'screen -ls' This will list the running screen sessions you have. Type 'screen -r rclone' to reattach to your session. When the rclone sync finishes you can then type the keys 'ctrl-d' to quit and exit out of the screen session.

Using SSH Tunnel to configure Rclone

Found another method that doesn't require you to install rclone on your local system.  You can run this on Linux, Mac or Windows stations. 

1. On your local system start up a terminal window (Powershell in Windows)

2. Start a SSH tunnel to redirect the shell server port 53682 to your local machine by using the following command:

  ssh -L localhost:53682:localhost:53682 ONID@shell.science.oregonstate.edu

3. Then follow the first set of the rclone config above. At step 9, copy and paste the auth url http://127.0.0.1:53682/auth?state=xxxxxxxxxxxx to the browser on your local machine, complete the auth, and proceed to follow the remaining steps. 

Print Article