General :: Automate Backups Over SSH On A Desktop?
Jun 7, 2011
I currently have a setup which allows me to connect to all computers on my home network via SSH and RSA keys. I'm very security-conscious, so all of my keys are passphrase protected. I'd like to essentially set something up where I'm running Unison on a cron job to back up to a file server on my network, which we'll call timmy. I've noticed that the first time I try to use a key on my Ubuntu laptop teeks, I get a dialog which pops up asking me to type in my key passphrase. I've heard that for servers needing to make automated backups like this that one should use ssh-agent to ask for the key passphrase on login/server start. How can I set this up on teeks?
I'd essentially like to have the following happen:When I boot and come into the OS, prompt visually for the passphrase as is done when I first use a key.If I SSH into this computer (as it's internet-facing) and I haven't provided the SSH passphrase yet, then prompt for it. (Sometimes, I might need to remotely reboot the machine over SSH, so I'll be SSH'ing into it after it reboots and I'd like to be able to authenticate the key without having to VNC in and do it manually.)
Making a script. This is for my linux class. Basically when you run it, whatever directory it is in, zips everything, backs it up to the folder its in, and also makes a copy of it in the root directory. Here is what I have, but its not running right.
Code: #!/bin/bash echo echo echo "Automate Backups" echo -n "Would you like to backup? Y or N:" read answer1 if [$answer1 = "y"] then if [$USER = "root"] then echo tar -cPvzf "$USER"_backup.tar.gz.`date +%F` /"$USER"/* cp "$USER"_backup.tar.gz.`date +%F` /backups/"$USER"_backup.tar.gz.`date +%F` echo "Successful Backup" fi echo tar -cPvzf "$USER"_backup.tar.gz.`date +%F` ~/"$USER"/* cp "root"_backup.tar.gz.`date +%F` /backups/"$root"_backup.tar.gz.`date +%F` echo "Successful Backup"
How do you get Rsync to do incremental backups rather than full backups? At the moment I have a script that will create a backup folder (if it doesnt already exist) then copy the source files into the backup directory with the command
Target is where the files will be backed up to Sources is the dir(s) to be backed up Exclude files is the list of files not to backup log file is where the output will be saved to. At the moment it only does full backups, but I would only like to do incremental, how would this be achieved? Am I missing out an option in the Rsync that is required.
I've been a DOS/Windows guy for 20 years, and recently became a SW test lab helper. My company uses CentOS for a lot, so I've become familiar with it, but obviously not as comfortable as I am with Windows.
Here's what I have planned:
machine: Core 2 Duo E8400, 8GB DDR2, 60GB SSD OS drive, ATI 4650 video card, other storage is flexible (I have 3 1TB drives and 4 750GB drives around that can be used in this machine.)
uses: HTPC, Network Storage, VMWare server host: SMTP, FTP server, and Web server virtual machines
I've figured out how to do much of this, but I haven't figured out how to do backups in Linux. I've been spoiled with Windows, with the built in backup system so simple to use. I find myself overwhelmed with the array of backup software, and unable to determine which to use. none of them seem to do everything I need them to do, but some come close, I think. I'm hoping someone here can help me out in figuring out which program to use and how to use it.
Here is what I need the backup software to do: 1. scheduled unattended backups, with alerts if the backups fail 2. a weekly full backup with incremental every 12 hours 3. removing the old backups when the new full backup runs, I would prefer to keep 2 weeks of backups, but that's not necessary 4. a GUI would be preferable, since my arthritic fingers don't always do as I want them to do. I typo things a lot, and the label worn off my backspace can attest to that.
I'd like to get the information about automatization of the tasks in Linux, more specifically in Fedora with minimal installation (without graphical interface and so on).The application (developed in C) needs to be started automatically when the computer is initialized. I've read about cron but I guess it's not a solution in this case because whenever computer starts the application must run once.I also have read about the/etc/ rc.local file but I made some tests changing it and they didn't work, the computer starts asking me the login and password but nothing run after that.The inittab was the same. In this way, someone know how to initialize an application developed in C when the computer starts?
I have RHEL Machine in production server where we gets usual alerts that logs filling up spaces. We archive those logs but in case it grows maximum we delete the old ones. Any idea how can we autiomate the process.
I need to tar this logs, but i dont how to make it simplier to me. Everyday there are created this five logs. I need to make five tar files from every day from this files at the end of the month
For example
Till now i have tar it manualy (copied every file)
I need to source the my /home/me/.bashrc file every time I "su -" to root. Is there any way to automate this? I cannot edit any thing in the root's environment as it is shared by people.
I need to ftp some files nightly from my linux box to an arbituary ftp server not controlled by me.
The ftp server admin has granted me an account for the purpose, but do not wish me to store the plain username or password in any script files for security reasons. How can I do that?
the wrong way would be:
Code: $ cat my_script open server_address user plane_ftp_username plane_ftp_password put a.txt
I have a heavily used file server that I want to restart, then if it requires e2fsck's on any volume to run them after it restarts. The only problem is that the server is rarely rebooted, and they said it might kernel panic because its been so long. I've heard there's a way to have it go past the kernel panic if it does happen, but I'm not sure how to do that or the other stuff.If it was a Windows server, I would schedule a shutdown with the force switch, and have the chkdsk's already scheduled for each volume on reboot. But for RHEL, I really don't know.I'm hoping this can be done, so that way I can have it kick off at say 7am, then when I get in at 8am it will probably be near the end of the e2fsck's so I can see what's going on.
I've been using Windows Home Server as a Backup Appliance, Media Server and Share Server at home for some time. I decided it was costing me allot of juice so very early on added the "Lights Out" add-on to ensure it was only running as and when needed.
I'm now looking to switch to a Linux based server and I'm looking for a similar tool/set of tools for advanced power management.
Now the question;
Anyone got any all-in-one suggestions (i.e with client parts for both Windows and Linux and a server part for the Linux server), or can anyone simply verify that I'll need to set-up all the individual bits for this myself separately?
Description: I am a newly appointed system engineer taking care of linux servers. We have a new set of data coming in which need below configuration: How to do a script with function?:
for files with ".txt" in sm copy each of the files to folder : sm1 and sm2 (log every copy) if succesful: remove original log into the log file if not successful: (not successful copying 1 particular file to all the folders) retain and retry log into the log file mail out the admin with that particular file name
I have already do try a bit: cd /export/home/ for dir in sm1 sm2; do cp -p sm/*.txt $dir/ done Is my starting right? How to do the rest parts?
I've inherited the following Virtual Machine scenario and am new to Linux Administration and Patch Management. The Host Operating System is Windows 2003 Enterprise, which has VMware Server 2.0.2 installed. Under the VMware Server 2.0.2 I have a Ubuntu 32-bit OS web server running Apache2 Web Services. When I log onto the Ubuntu server (9.10 32-bit) I see the following two lines just above the new mail/last logon lines.
85 packages can be updated 55 updates are security updates
I would like to see at least a summary of each update and its urgency so I can notify the various developers/server owners to get their input regarding whether we should or should not apply that particular update to the server. We apply the patches in our test/dev environment first then once vetted there we roll them out to our production servers. What I am looking for is a way to automate the gathering of the information and once approval has been received automating the actual patching process so that I do not have to manually perform the apt-get process for each separate package needed/approved.
Ideally I would like a recommendation for a GUI based package to manage this process and that is capable of generating the appropriate reports for the 'powers that be' regarding the current security/patch management environment. For proof of concept I would like a free version that is not hamstrung in functionality but is not too costly to procure the production version with no limitations.
I'm not that familiar with sed and awk in order to be able to solve this problem on my own, so I'm calling on you for a bit of assistance. I'm writing a Nagios plugin to check our Oracle tablespaces and the output is given in one line like this: 1.04007771 TEMP 0 UNDOTBS1 .005340579 USERS 0 7 rows selected. I've been playing around with sed like below to delete the obsolete info and change every second space into a newline:
[code]...
how many tablespaces there are so I'd have to check all databases and 'hardcode' the tablespaces in my script. Is there any way to 'automate' this knowing that 'rows selected' preceded by a number is always the last line and using a sort of counter to auto-adjust the number to put in the -e 's/ / /2' part?
This is my sample code in /etc/httpd/conf.d/applications.conf file currently we are creating subdomain mannually for every new subdomain. I want to automate this process througs bash script , how its possible.
In this code which i marked BOLD that content only we'l change for every subdomain. while manually doing this most of the times errors are coming to avoid that i need to automate this process.
I am using rsync to backup dirs on my ubuntu server onto a NAS (which is mounted onto the filesystem), but the problem is that it is constantly doing full backups rather than doing incrementals and I am not really sure why. After doing a bit of expermienting with the script I noticed that if I just backed up a home dir (/home/user) the incremental backups work fine. If however I was to back up a dir like (/home/domain/user) it always does full backups.I have tried various different scripts but still the same end result. The latest script is a variation on the a script found on the samba rsync examples webpage, see below...
#!/bin/bash # rsyncbu.sh -- backup to nas using rsync # This script backups files listed in BDIR to the BSERVER. The verbose output along with the date is listed in the LOG_FILE specified # verbose output
I have a small IT consulting business and I am finding many of my clients couldn't afford Microsoft solutions and thus, going without... So a revelation hit me that I could offer Linux solutions, its just that I'm not a Linux guru... So after much research and installations of nearly every latest Linux distro, I decided on ClearOS as a good option for my clients that just require a good File/Print server, firewall and VPN solution... Everything was going fine until I got to the point where I was trying to get it to do backups to a USB drive... ClearOS does not come with the ability to do such, so from further research I found that I could install Webmin to handle that task and that it would not break ClearOS... Great, its just that its not working... It says its working, but its not.
[Code]...
Next I hit the "Save and Backup Now" button to test it out... all says it goes good... But when I check the device, there's nothing saved to it...
So I create another "Filesystem backup", I select the FlexShare directory that I want backed up and this time I select a local directory for the backup to go to... I hit the "Save and Backup Now" button to test it out. all says it goes good... But once again when I check the directory, there's nothing saved to it.
I currently use cp to backup data. I prefer it over rsync. I use the -b switch to make a backup of data and recently found you can use --backup=t to create numbered backups.Using --backup=t means that I could end up having 100 versions of a file if I change it 100 times. With the -b switch I will only ever have 2 versions. Is it possible to limit the numbered backups to 5 for example? So I would only ever have 5 versions?
I have just purchased a 1 TB external hard disk to be used for backups. The backups will be performed with rsync and since I don't really care about accessing the data from other operating systems, I think I'll use ext3 on the partition. I'll just be backing up my home directory and probably /etc as well. In my home directory, I have a small number of files that are several GB, but most are tens of MB in size or less.
I'm just wondering if there are any special options I should pass when I create the filesystem with mkfs.ext3.
I have a small IT consulting business and I am finding many of my clients couldn't afford Microsoft solutions and thus, going without... So a revelation hit me that I could offer Linux solutions, its just that I'm not a Linux guru... So after much research and installations of nearly every latest Linux distro, I decided on ClearOS as a good option for my clients that just require a good File/Print server, firewall and VPN solution... Everything was going fine until I got to the point where I was trying to get it to do backups to a USB drive... ClearOS does not come with the ability to do such, so from further research I found that I could install Webmin to handle that task and that it would not break ClearOS... Great, its just that its not working... It says its working, but its not...
So here's what I got, ClearOS 5.1 with Webmin 1.5
Here's what I have done to try to make it work after a good installation...
Under "Hardware" and "Partitions on Local Disks", it shows the USB drive as Device B... So I create a partion called "/dev/sdb1" Great, I'm thinking...
So I go to "System" and under "Filesystem backup", I select the FlexShare directory that I want backed up and then I select "/dev/sdb1" as the backup to device...
Next I hit the "Save and Backup Now" button to test it out... all says it goes good... But when I check the device, there's nothing saved to it...
So I create another "Filesystem backup", I select the FlexShare directory that I want backed up and this time I select a local directory for the backup to go to... I hit the "Save and Backup Now" button to test it out... all says it goes good... But once again when I check the directory, there's nothing saved to it...
I am a Mac user since 1988. I recently discovered Linux Ubuntu and love it. So much that I use it about 95% of the time. On the Mac there is an application I use called Superduper which makes a bootable backup to an external USB drive.
Can I do the same kind of thing using the dd command? I use the excellent Cron GUI Scheduled Tasks. I was hoping that maybe I can use that to schedule nightly bootable backups. Is dd the right tool to use? Once the initial backup is done (which I understand can take a long time), does dd do incremental backups after that.
Looking forward to how I can set this up so that I can just set and forget reassured that bootable backups are occurring overnight.
I have inherited a network server running Redhat Enterprise (4?) which uses an external USB drive for backups. These have been scheduled to run at midnight each night.I want to use 2 external drives to hold the backups (exchanging the usb drive each day). My question is: Is there anything I need to do to a new USB drive before exchanging the drives (eg. formatting etc) or can I simply just plug the new one in and let it run?I apologise for the very basic nature of this question but I have no clue about Linux.
Just trying to set up a new backup using tar, but there are a few things I dont want to include. Using --exclude I can exclude sub-directories, but how do I exclude specific files in a subdirectory that are (for instance) executables or have a specific extension?
What do I need to add to my .emacs to get it to save all of my autosaves and backups into one directory? I don't do a lot of .emacs configuration, and I just can't get the variables out of emacsWiki to play right. Anybody mind sharing how they do it? I would prefer to have the saves placed in /tmp/emacs/{username}/{autosaves | backups}
I am getting the databases from mysql and my database name is username_something. I am getting the username and then puting the respective backups in corresponding folders like
tar bala bla /backups/sql/username/username_something.tar.sql.gz
The problem is system worrks if i have the folder username already there but for new databases if get the error like unknown file path.
How can i do that if username folder is not there it should be created
I have a script that moves files from one directory to another, based on the numeric date of the file name (i.e., 20091212 would go to the December directory). Now, since this script will be ran at the beginning of the following month (December's files to be tarred and gzipped will be performed the first day of January, and January's in February, etc.), it appears to me that the script will be tricky when it comes time to do the December files in January.
Here's part of the script: Code: # Define working directory and target directory. DIR=/var/log target=$DIR/month hostname='uname -n' .....
I can't seem to figure out a way to carry the output of the date command to the next command, and the year for the December files will always be wrong.
