Server :: Use FileSystem For Backups?
Jul 8, 2010With so many filesystems available which one should I use to make backups? All I care about is reliability and stability. I don't care at all about portability.
View 8 RepliesWith so many filesystems available which one should I use to make backups? All I care about is reliability and stability. I don't care at all about portability.
View 8 RepliesI have just purchased a 1 TB external hard disk to be used for backups. The backups will be performed with rsync and since I don't really care about accessing the data from other operating systems, I think I'll use ext3 on the partition. I'll just be backing up my home directory and probably /etc as well. In my home directory, I have a small number of files that are several GB, but most are tens of MB in size or less.
I'm just wondering if there are any special options I should pass when I create the filesystem with mkfs.ext3.
I've been a DOS/Windows guy for 20 years, and recently became a SW test lab helper. My company uses CentOS for a lot, so I've become familiar with it, but obviously not as comfortable as I am with Windows.
Here's what I have planned:
machine: Core 2 Duo E8400, 8GB DDR2, 60GB SSD OS drive, ATI 4650 video card, other storage is flexible (I have 3 1TB drives and 4 750GB drives around that can be used in this machine.)
uses: HTPC, Network Storage, VMWare server host: SMTP, FTP server, and Web server virtual machines
I've figured out how to do much of this, but I haven't figured out how to do backups in Linux. I've been spoiled with Windows, with the built in backup system so simple to use. I find myself overwhelmed with the array of backup software, and unable to determine which to use. none of them seem to do everything I need them to do, but some come close, I think. I'm hoping someone here can help me out in figuring out which program to use and how to use it.
Here is what I need the backup software to do:
1. scheduled unattended backups, with alerts if the backups fail
2. a weekly full backup with incremental every 12 hours
3. removing the old backups when the new full backup runs, I would prefer to keep 2 weeks of backups, but that's not necessary
4. a GUI would be preferable, since my arthritic fingers don't always do as I want them to do. I typo things a lot, and the label worn off my backspace can attest to that.
How do you get Rsync to do incremental backups rather than full backups? At the moment I have a script that will create a backup folder (if it doesnt already exist) then copy the source files into the backup directory with the command
rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE --exclude '/Ls-wtgl1c8/**' -rt --delete $source/ $TARGET/$source/ >> $LOG_FILE
Target is where the files will be backed up to Sources is the dir(s) to be backed up Exclude files is the list of files not to backup
log file is where the output will be saved to. At the moment it only does full backups, but I would only like to do incremental, how would this be achieved? Am I missing out an option in the Rsync that is required.
I have a crontab related question which I am hoping someone can answer. I recently took over a Redhat Enterprise 5 Server, and I was told by the previous Server Admin that there is a cron job that does the backups. I ran the following command to get a list of all users:
Code:
cat /etc/passwd | grep "/home" |cut -d: -f1
I then ran the following command for each of those users to see if they have any crontabs associated with them:
Code:
crontab -u USER -l
It doesn't show any crontab entries for any users (including root). But I am positive that there is a scheduled job somewhere because the backups are still running every night.
Can I use rsync for incremental backups of the running linux server?
View 5 Replies View RelatedI've had several HDD crashes on my personal server over the years and it's just gotten to be a real pain in the rear. Crashed again this morning. Currently, I make monthly tarball backups of the entire filesystem using my script:
Code:
#!/bin/sh
# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz
[code].....
I have an ubuntu (8.04.3) server where I use bacula to make backups of the files stored on the server. Ive been trying to find a solution (with no luck) trying to succesfully implement the following:-
A Backup tape for each day of the week besides Thurs which is resused on a weekly basis. For the thursday tapes we have a backup tape corresponding to the week number that the thursday falls so for the first thursday of the month it would be ThursOne For example. These tapes are resued on a monthly basis. We then have a monthly tape that is used on the last thursday of the month. These tapes will be resused on a yearly basis.
Another requirement is just in case a tape is accidently not changed a backup should still occur regardless of what tape is in the drive (so if its tuesday and mondays tape is still in the tape drive it should rewrite that tape).
I did have this successfully set up where the tape was appended after each use rather than being recycled after the nightly backup. But then after a few weeks I would have to manually purge tapes when they became full (which isnt ideal - as Im not always in the office so in my absence it may be that a backup may not take place), so have been playing around and have now got the tapes to be marked as used after a max of 2 jobs (so the backup of the files and the catalog of the night). I also added this line 'Recycle Current Volume = yes' so that it would hopefully recycle the volume in the drive.
However what I am finding is that the tape that should be recycled is not, but in yesterday case the Mondays tape was recycled rather than the Tuesday although Mondays was the last written so Im not even sure why it choose to recycle this tape.
**Edit: path for mount was incorrect Distro Server: CentOS 5.5
Clients:
Fedora(latest)
OSX(latest)
Backround I am attempting to setup a server in my house mostly(for the first time) for backups and file sharing. It is important to me that file permissions are preserved. So its my understanding that I must use idmapd in order for this to work. As of now I'm only working with the linux distros while osx will be dealt with once these two work together. portmapper is up and running, along with lockd on both machines. Firewalls are also down on both machines for now. The server side was all setup using the GUI interface with no extra options selected. Problem When attempting to "mount -t nfs4 10.0.0.2/$sharedfolder /mnt" I get an error operation not permitted with no error printing in /var/log/message. If I use "mount -t -o nolock nfs4 10.0.0.2/$sharedfolder /mnt" it mounts just fine. Ive checked both machines multiple times to make sure that lockd is up and running. In the idmapd.conf file I the domain as "localdomain" for both machines but I doubt that is right; like I stated above this is my first attempt at a server. I'm assuming the problem is a whole missing step that involves some kind of id mapping server I need to setup.
I'm trying to find a secure way to backup files on my Prod Server to Backup Server. It must be automated, so I will need to run a command with cron which will login to Prod Server from Backup Server and backup data. 1. Do you think it would be secure enough to do this by creating an passwordless RSA private key on Backup Server and adding it's public key to authorized_hosts file on Prod Server? I can't think of a way to Automate this without having to enter any passwords without passwordless RSA key. Is there another. more secure way? 2. Should I create a special user for backup, which will only have read access to all files in the directory that I am backing up? If so, How can I run a check that this new backup user indeed has read access to ALL files in the folder that I intent to back up? How can I ensure the backup process will not skip files due to some permission problem? 3. I'm thinking of using rsnapshot tool, which uses rsync.
View 10 Replies View RelatedI am very new to linux, and I have a question regarding the filesystem check (fsck). The power recently went out and when I tried to restart linux the following error appears:
*/dev/sda1 contains file system w/errors, check forced it then goes on to say..
*An error occured during the file system check. Dropping you to a shell; the system will reboot when you leave the shell. Give root password for maintenance (or type Control-D to continue) I wasn't sure what to do, but checked some other online forums and they suggested running fsck manually - so I typed in the root password - and used the command, "fsck -A -V ; echo == $? ==" it then gave the following message
*WARNING!!! Running e2fsck on a mounted filesystem may cause SEVERE filesystem damage
*Would you like to continue (y/n)
Again, I wasn't sure what to do so i just checked no. I then manually turned off the computer and was prompted at the beginning to press Alt-3. I was brought to another screen and it informed me one of the drives was degraded and suggested rebuilding the array. I tried doing this, but it still brings me back to the original error of, "/dev/sda1 contains file system w/errors, check forced," and the process continues.
Also, when I tried to rebuild the array, I didn't backup any of the data on our home directory before doing this (which was probably a big mistake). After being prompted to type the root password, I was able to give the ls command and look at all the directories...the home directory where our data was stored was empty and I am afraid I may have lost some information. Is there a possibility that data was lost when I was trying to rebuild using the old drives?
I am working on a NFS server embedded on a PowerPC plateform (4650EX, 512Ram, 1 GB Ethernet) but i can't mount my exports folders from my client. Here are messages :
[Code]...
I have an XFS filesystem (not my root filesystem) that is not mounting.
We had a power failure over the weekend. All but one of our XFS mounts is comming up. Its a large mount (2TB) and I am not seeing errors per se...
But after attempting to mount it, it has been 4 hours and all we see in either dmesg output or in the messages file from the kernel is:
Our mount process is dead and I don't see any disk i/o going on. Am I missing something here on this?
I rebooted the box once earlier and was able mount read-only with norecovery...
I tried xfs_check and it also produced a dead "xfs_df -i -p xfs_check -c check /dev/sdd1"
When I try to boot to OpenSUSE I get the following error during boot-up: unknown filesystem type 'reiserfs' could not mount root filesystem - exiting to /bin/sh$
This only started happening quite recently - before this I could boot to Linux quite happily.
I am trying to mount a file image, like this
mount -o loop /tmp/apps.img /media/apps
But I get the following:
mount: you must specify the filesystem type
I try ext3:
mount -o loop /tmp/apps.img /media/apps -t ext3
dmesg says:
error: can't find ext3 filesystem on dev loop6.
I've also tried ext2, vfat etc. How can I detect the filesystem type of apps.img?
I have a following problem: Recently my drive with Ubuntu 9.4 has mysteriously stopped working, i.e. when I switch the computer on it informs me that GRUB didn't find the filesystem. Well, I suppose it happens.
First, I though it was due to the drive dying, but I popped it in an external enclosure and HDTune told me the drive was fine. Wanting to recover the files on the drive before reinstalling I first tried to mount it in said external enclosure under Windows (I have Win Ext2 driver installed which used to work just fine). This time, however, drive gets assigned a letter but upon opening it Windows popped up an error saying that the drive was not formatted and whether I would like to format it then.
Unfazed by this streak of failures I tried to mount it under Linux but, alas, to no avail. I might have tried every single -t operator under mount command but it still won't budge and let me mount.
I have an HP proliant server n i am new to linux the problem i am facing is that when i start the system it turns ON normally but after the linux starting up options like when we select whether to start it normally or in failsafe mode after selecting one option out of the two it comes up with following messages on the screen and get stuck over there.
[messages]
I would have attached the snapshot i have taken with a camera but there is no option of attaching files overhere.
I've had a look at some similar threads but as I'm very new to linux they're already a bit technical for me. Sorry, this calls for someone with patience. I gather from other threads that disconnecting an external drive without unmounting is a no-no, and this seems to be the likely cause. Now the disk is read only and I'm unable to change any settings through the usual control panel on ubuntu. I'm just not familiar with the terminal instructions. I tried to cut and past a few command lines from other threads but I got some warnings that proceding could damage data. Like this one: WARNING! Running e2fsck on a mounted filesystem may cause SEVERE filesystem damage.
View 5 Replies View RelatedAh, the most dreaded SVN error of them all. It's not even showing up in my Apache logs, either, just on trying to commit.
When I try to add a file to the repository via the following command, I get the error:
Code:
add -N F:eetlemed-1.7workspace
ootClientsrccomeetlemedwebappdataContactInfo.as
A F:/beetlemed-1.7/workspace/rootClient/src/com/beetlemed/webapp/data/ContactInfo.as
commit -m "no more broken commits?" (24 paths specified)
code....
It seems that this error only shows up when I try to add a file to the repo, not when I try to update. I've already run chmod +rwx -R repository, so it should have all of the permissions it could possibly need. The only thing I don't know about is adding it to a group. I don't think my server has an apache group, so I wouldn't know where to start and what to do with that.
This problem happens from time to time, but I can't seem to shake it this time. I'm running SVN over HTTPS on Apache 2.x.
Are there any tools out there to scan, say our fileserver(20TB), to check for things that shouldn't be there like mp3s, videos, software crackers, etc?
View 2 Replies View Relatedwe have a data transfer network drive, shared via nfs and samba.But now I got the special demand to make any of the files read and wirteable, regardsless of the permissions they had before.With acl I get the right permissions (via default values) but the standard unix permissions overwrite this. e.g. when I have 644, it does not care that the group has write permissions)Does someone have an idea (except chmod via cronjob )
View 1 Replies View RelatedI wanted to know if it's posible to import filesystem quotas. And if so, how can I do it? I recently migrated a server, and presently the users don't have any quota limitations.
View 3 Replies View RelatedI'm running CentOS 4.3 on a VM which is an application server for Quinstreet. trouble is when i keep coming in during the mornings it seems to keep making this root filesystem read only. There is no pattern for this and neither is it clear in the messages log why this keeps happening.
View 5 Replies View RelatedWhat is the recommended filesystem to use when creating a home server/nas?
I'd be sharing files using SAMBA, DLNA Server or some sort of streaming. I'll have two win7 laptops, 2 ubuntu desktops and ps3 accessing the files. Most of the time the server will just 75% read from 25% writing.
Would ext4 be an ideal?
I changed the first line of /etc/fstab in an attempt to get rid of an annoying message "mount: unknown filesystem type 'static'". All I did was put a "#" on the first line .... changing the original from:
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
to:
#
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
I have tried desperately to get to the file system so that I can edit that first line in fstab out but nothing works and the file system comes up as read only ...
I have a very specific requirement outside the typical use case of "enterprise server". If possible, I need to run a FUSE filesystem on a headless server, and share it via Samba.Physical security is a non-issue.I see two challenges with this, that Google and I have been unable to figure out: I'm assuming that in order to mount a FUSE system, a user must be logged in. (Hence the "U" in "FUSE".) So the first challenge is how to get Ubuntu to automatically login, on the physical console, to a user terminal session. (I know this can be trivially done in GNOME, but this won't have X-Windows or GNOME installed.) This has to happen automagically after boot-up; manually logging in locally or via SSH isn't an optin.Second challenge (?), how to share a FUSE file system via Samba?
View 2 Replies View RelatedI have configured the RHEL cluster suite in RHEL 5.3 64 bit, I have formatted a storage 100 GB LVM with GFS2 filesystem with lock_dlm protocol default journel I have used 8 I have also added this filesystem in cluster resource in GFS resource option, all work fine cluster is able to remount this GFS2 LVM on another node in time of failover. But when I was checking cluster configuration file (/etc/cluster/cluster.conf) in resource syntex it was showing fs_type=gfs so my query is if it uses fs_type=gfs then is it also mount my formatted gfs2 lvm as gfs filesystem in my cluster node rather then gfs2 filesystem. Also how do i check that mounted filesystem is gfs or gfs2, Please clear my doubt.
View 4 Replies View RelatedHow can I give 1 user access to mount 1 particular filesystem? This is for Debian 6 64bit.
View 4 Replies View RelatedIn CGI scripts, there are certain files that are getting "permission denied" when it seems they should be accessible by the apache user. I am running the default package install of apache under fedora. Here is an example:The following is /var/www/cgi-bin/test.pl
Code:
#!/usr/bin/perl
use strict;
[code]...
I'm trying to config Apache and Subversion to work together, I did:
1. yum install mod_dav_svn subversion then I had Subversion and httpd installed!
2. create: /usr/local/subversion/repos
create: /usr/local/subversion/repos/test (by svnadmin create)
create: /usr/local/subversion/access/svnauth.conf (by htpasswd)
added to file : /etc/httpd/conf.d/subversion.conf:
<Location /repos>
DAV svn
SVNListParentPath on
[Code]...