Automated SCP Copy Script (with Wake-On-LAN)

As some of you may know currently I have a file server and a download server. The file server has a script that checks if anyone is connected if not , it goes to sleep. It can be waken via Wake-on-Lan. This is to reduce power consumption , as due to electricity tariffs which are going up in 2014 (that is in Malaysia). My File Server consumes about 60-70W of energy when it is idling. When it is asleep it only consumes about 2W 

server-to-server

My Torrent Server on the other hand consumes about 20W (It’s a little machine that runs on Intel Atom 330) and has a 80GB of Storage. Basically I use it to download things from Deluge.

The problem is that once the file has been downloaded , I would have to move the files manually from my torrent server to my file server . It’s painful – I’ve to ssh over , scp and rm the files once it is done. It is becoming more and more troublesome when you have a large files to copy over – especially downloading movies

ssh [email protected]
cd /downloads/
scp -r file [email protected]:/largeHD/media/
rm file

This is what i generally do . What’s worst is that if my file server is sleeping , I would have to issue a wake on lan command to activate it. Also I’ll have to change the destination every time based on the type of file , as i’ve my file server organised !

Requirements

I was looking for a script that does the following :-

  • Automatically determine the kind of file (if it is a folder or a file)
  • Copy the file/folder from my Torrent/Download Server to my File Server and Organise based on Type (i.e Movies/Music/Softwares) via SCP
  • Wake my File Server if it is sleeping
  • Notify me if the transfer was successful or it was a failure. If it was a failure , do not delete the file , if it was successful delete the file from my torrent server since I have a copy in my file server
  • To automatically detect the kind of file and move it to the right location in destination server (i.e if I’ve downloaded a movie , automatically movie it to the right folder in destination server/file server)

I tried scouting over at Google to check if such script exists . But sadly nope , there is no such thing as one click automated file copier that does it. There are commercial softwares out there but I’m not interested in it .

Final Product (mv2svr) & The Workings

Therefore , I said to myself , “Pradeesh , it is time for you to build your own script“. That’s what I did , embarked on a quest to learn bash syntax and programming to build a script that exactly does that. I’ve learned a lot from Google and not to mention StackOverFlow!

I’ve written a shell script which automatically does that (based on my requirements) and I’ve decided to call mv2svr (short for Move-To-Server). Basically all I have to do is invoke it by calling.

My objective is to help people to learn bash language , so therefore i’ve included a lot of comments explaining functions

mv2svr $file $type

where $file refers to the file or folder and $type refers to the type (is it a movie ? music ? games ?). $type is actually an optional parameter. If nothing is specified , the script tries to determine the type (Currently it only works with Movies) and moves it to the right place

It checks if the destination server is asleep or a wake , if it is a sleep it will send a magic packet and wake the system up. Otherwise it continues by uploading the file to the destination server by using scp *using arcfour encryption*. The reason why I picked arcfour is that basically it gives me the highest transfer speed . I got the information from this site which offers benchmark of all sorts of different cipher. Since I’m not worried about security (as it is in the same network/LAN)

Once it is done uploading , it would then remove the file from the source server in this case my torrent server.

Download & Installation

You may download the script from Pastebin by clicking HERE

To Install the script

1. Simply ssh to your server

2. Execute the following command (If you are an expert probably you could wget or copy/paste it)

cd /usr/bin/
sudo touch mv2svr
sudo nano mv2svr

3. Copy/Paste the script into nano. Be sure to modify the paths such as where do you want to copy in destination server and be sure to change the IP Addresses , MAC Addresses .Do read the comments as well !

4. Make it executable !

chmod +x mv2svr

5. To use the script it is very simply just invoke by calling mv2svr , followed by the folder/file you wish to transfer and it’s kind , you may leave it blank for the script to automatically detect

mv2svr "Sivaji The Boss"

Alternatively

mv2svr "Sivaji The Boss" movie

or

mv2svr "SomeRandomGame.exe" games

Example of how it works :-

Screen Shot 2013-12-28 at 9.37.34 PM

*For Wake On Lan to work or setting it up , you would have to install wakeonlan. To do so simply run (on Debian/Ubuntu) systems

sudo apt-get install wakeonlan

For more information setting up Wake On Lan , kindly refer to this guide

Known Issues/Things need to be done

1. Automated Detection only works with Movies – you can easily change it for other types. in the future , there will be a function which allows users to customise it (i.e detects if its movies/music by file extension and sorts them automatically)

2. Removing folder does not work. The detection is there (refer to the source code) , all you have to do is just to incorporate it. I’ll be including in the next version. Just think of it as an exercise for you to learn bash programming or to improve my script

3. Remove all other hardcoding and make it more flexible !

What’s in Store In Next Version/Future Plans

No doubt is for me to improve the script. But I’ll have to learn more on bash programming first in order to improvise a lot of things. I am also working on a PHP front-end which uses this script , thus making it easier for me to move files without the need of logging into the server and doing it

It will automatically list all the files/directory and I’ll just have to select the type and poof it will automatically invoke mv2svr with the right parameters and move the file !

Screen Shot 2013-12-28 at 10.43.29 PM

Feel free to comment or to improve my script. If you have done any sort of customisation , once again do not forget to share

 

Leave a Reply

Your email address will not be published. Required fields are marked *

CommentLuv badge