Fedora :: Use Rsync To Backup My Music Collection - Add Files To The Destination Drive?

Mar 6, 2010

I'm currently learning to use rsync to backup my music collection. I have a Firefox tab open to the rsync manual page(s) and have been reading man rsync and running experimental rsync operations.I've been doing this for the last 3-4 hours. I've used rsync for this purpose in the past with disastrous results. What was and is once again (due to a month and a half of file pruning) a 9000 file music collection had mysteriously grown to over 25,000 music files and 80GB of data! This was likely due to the fact that I didn't really know what I was doing with rsync and had never spent too much time learning about all the parameters, what their functions are and how they may relate to my goal.Here are the particulars:

* Source drive is a 500GB disk, /media/sata500/music/.

* Destination drive is a 250GB USB disk, /media/FreeAgent/music, connected to the same computer that houses the 500GB disk.

* I want to copy or backup files from /media/sata500/music to /media/FreeAgent/music.

* I do not want to create ANY duplicates of files that exist.

* I only want to add files to the destination drive if they are new on the source drive, like if I rip a CD and add the contents to the source. I want them copied over next time I run rsync.

Here's the rsync command in it's most recently used form, and probably very immature at this point.

Code:

rsync -t -r -vv --stats -i --log-file=/home/glenn/rsync.log /media/sata500/music/* /media/FreeAgent/music/

This appears to have copied all files and folders and I'm satisfied that my goal has been met with some success. To convince myself of this I ran the command and then once it was complete I added 2 new songs putting them in their respective folders on the source drive and ran the same command again. The resulting output was

[code]....

Two files transferred. Exactly what I want.Both folders now house 20,931 files and use 40.6GB. Identical as far as I can tell.What I'm concerned about are time stamps and play count data, etc. Anything that changes the original file. I don't want this data to cause a file to be transferred as I'm afraid that the new file will be created along side the old file of the same name thereby starting this whole music collection expansion thing all over again. I've invested a lot of time and effort to get it pruned down to where there are virtually no duplicates and albums are correct in that they contain the proper songs in the proper order.

View 14 Replies


ADVERTISEMENT

General :: Files Seem To Take Up More Space In Destination After Rsync Copy?

Feb 9, 2010

I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?

Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".

These are the mount points :

/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1

I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.

rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/

Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :

rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/

I tried an fsck on the home partition :

fsck -f /dev/sdb6

This is the output from the last rsync :

rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]

[code]....

Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.

Here is some additional info :

$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp

[code]....

View 2 Replies View Related

General :: Rsync Deleting Files From Destination Directory?

Jul 14, 2011

We have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.

Here's our current rsync .sh:

/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1

[code]....

So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.

A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)

I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.

View 2 Replies View Related

General :: Amarok - Collection - Scanning Is Not Finding Any Music Files?

Mar 13, 2011

I have been using Linux close to 2 years now. One thing that always bewildered me is audio support in Linux. These days I login to windows only for listening to music. After reading various blogs, i decided to give it a try in Linux with Amarok. There again I am facing a problem.

while i try to scan for music files it is not finding any files.

what I did is as follows:

Quote:

Setting --> configure Amarok --> collection --> scan

I have tried with mp3 .wma format files. While I try to add these files individually, Amarok is able to play those.

I am using Amarok 2.3.2 with KDE 4.4.5. Fedora - 12 is my flavor.

Given below is the log obtained with amarok -debug option

Quote:

TagLib: MPEG::Header:arse() -- First byte did not match MPEG synch.
TagLib: MPEG::Header:arse() -- Invalid sample rate.
TagLib: A frame of unsupported or unknown type 'TSC' has been discarded
TagLib: A frame of unsupported or unknown type 'TSC' has been discarded
TagLib: A frame of unsupported or unknown type 'TSC' has been discarded

[Code].....

View 3 Replies View Related

Ubuntu :: Using Rsync To Sync Music Folder To Another Folder Called Music On An External USB Drive?

Nov 25, 2010

I am thinking of using rsync to sync my Music folder to another folder called Music on an external USB drive. I will be using the Scheduled Tasks front end to schedule the syncs. What should the syntax look like when I put it in Scheduled tasks. I want this to be as simple as possible.

View 8 Replies View Related

Ubuntu :: Backup Files To Ntfs With Rsync?

May 31, 2010

well, i know ther are issues when using rsync to copy files to ntfs partition like file permission blah blah. the thing is, i need to backup my music files periodically onto a ntfs partition from ext4. i really dont care about file permissions or any other stuff. when i use rsync, it should update the mp3 files on my ntfs (external) disc with the new ones.can i give a go with this operation? i have lot more important files on the external disc and i dont want this rsync corrupt or delete those files coz they are highly important files.

View 2 Replies View Related

General :: Setup Rsync To Backup A Remote Directory To Local Drive?

May 24, 2010

I'm trying to setup rsync to backup a remote directory to my local drive.

I cd to the directory that I want to pull the files to, then I enter:

rsync -vrtW account@remote.com:~/public_html

I enter the password then it starts running. I get all the files listed, but none of them actually transfer. What am I missing?

View 1 Replies View Related

OpenSUSE :: Rsync Backup *cache* Directories And Files?

Mar 4, 2010

I'm creating a backup scheme with rsync.

It is pretty clear not to include these:
/var/lib/named/proc
/var/lib/ntp/proc

[code]...

View 3 Replies View Related

Ubuntu Servers :: Rsync Backup - Transfer Files With Appropriate Permissions

Dec 4, 2010

I wrote a script to wake up my windows machine and do an rsync backup of some of my files. I wanted to make this command a accessible through local bin so I made it executable. However the problem is that when I copies files is copies them with root permissions and i can edit or delete them. How can I set the files so they transfer with the proper permissions for my Ubuntu user?

Code:
#!/bin/bash
# Description: This script first wakes up the client machine and syncs the appropriate folders.
# Finally the script shuts down the client if it was off to begin with.
if [ "$(whoami)" != "root" ]; then
echo "Permission Denied"
exit 1
fi .....

View 8 Replies View Related

OpenSUSE :: Backup Important Files - Automate The Process Using Rsync And Cron

Aug 14, 2011

I would like to backup important files (totaling about 400GB) on my ext 4 RAID 5 array to an ext4 external hard drive over USB (external drive is mounted to /mnt. In the future I'd like to automate the process using rsync and cron so for now I'm using rsync to transfer the files. My problem is that using the rsync command like this: # rsync -Pr "/dir1" "/dir2" "/dir3" "/dir4" /mnt

rsync shows me the checks and transfers for awhile and then throws up an i/o error (wish I had a screenshot to show but I don't). When I ls /mnt I get a similar i/o error. I then check /dev for the drive and find that it no longer shows up. Originally the partition was /dev/sdc1. I tried unplugging the USB at this point, plugging it back in and mounting the drive back to /mnt, however it has now assigned it to (you guessed it) /dev/sdd1. I get the drive mounted and try the original rsync command again, hoping the first error was a fluke or some kind of one-time drive fart. This time it makes it quite a bit further and then throws up the exact same problem. Am I doing something terribly wrong here? As I said, I'm very new to bash so I'm not making some absolutely moronic, newbie mistake.

View 9 Replies View Related

General :: Mount Point For Backup Drive - Partition To Backup The Important Files On My Main HDD

May 21, 2011

I'm just setting up a partition on a seperate HDD in my system. I plan to use the partition to backup the important files on my main HDD (to guard against HD crash).

The question I have is about where would be the typical location to auto mount this partition? Which would it be normal to go for:

1. /backup/
2. /media/backup/
3. /mnt/backup/
4. /home/chris/backup/

View 7 Replies View Related

General :: RSYNC - Ignore A Directory On Destination Box?

Jul 28, 2011

I am performing a dry run using Rsync on 2 different boxes.While i'm doing that, Under destination directory, I want a specific directory x to be ignored for sync.Please let me know the exact pattern to ignore the directory.The current command I'm using is:rsync -avnc --delete $LOCAL_DIR $USERNAME@$DESTINATION_IP:$REMOTE_DIRunder DESTINATION_IP, I would want to ignore a particular directory under REMOTE_DIR.

View 1 Replies View Related

Server :: RSync Without Delete Any File On The Destination?

Feb 16, 2010

I'm using rsync for a backup-sript at the moment and want to keep all files. The files are always unique, so I want to rsync without delete any file on the destination.

I've tried with --no-delete and --max-delete=0 but nothing seens to work. Is there even a possibility to do so?

View 2 Replies View Related

Ubuntu :: Database For Music CD Collection?

Feb 20, 2010

I'm looking for a recommendation for an app that will keep details of all my audio CDs. Ideally I'd like it to be able to produce html or xml so that I can check it from anywhere ("Have I got that track?").

View 4 Replies View Related

Ubuntu One :: Does 'entire Music Collection' Mean Only Up To 50GB?

Oct 21, 2010

I have a few questions in regards to the music streaming service offered.

1. Does "entire music collection" mean only up to 50GB?
2. I can't sync off my external USb harddrive. Is there a way to do this. my entire music collection is on the external HD.
3.If i move music to from my external HD to the ubuntu one folder am I actually copying the file to my netbook harddrive or is it only a pointer? I have limited harddrive space on the netbook.
4. If I have to actually move files from my ext harddrive to the ubuntu one folder I may need to delete folders because of limited space. Anyway to prevent syncing so the folders uploaded do not get deleted from cloud.

View 1 Replies View Related

Fedora :: Making An Rsync Backup More Robust?

Aug 24, 2009

I'm syncing a server over the internet with rsync, but it only works for a few hours before the backup fails with a "No route to host". I can restart the job and it'll will pick up where it left off, but is there an automated way to do this, or protect against a connection failure? I have about 170GB to copy over initially, but I can only get through about 4-5GB before the connection drops--manually restarting the sync everytime it drops will make the initial backup take days...

View 2 Replies View Related

Fedora :: Compressing Collection Of Text Files?

Nov 6, 2010

i have a collection of text files for 50 gbs..i want to compress these files and store them into dvds.i want to know the present popular optimized method to compress the files.let me know masters.

View 1 Replies View Related

Ubuntu Multimedia :: Music Player/jukebox For Large Collection?

Mar 29, 2010

What would be the best music jukebox that doesn't lag when there is a lot of music in the library (400+ gigs)?

I have tried songbird, but its pretty laggy. I like to import all my music and turn on shuffle and go through my songs.

View 3 Replies View Related

Ubuntu Multimedia :: Amarok No Longer Sees Music Collection?

Jun 27, 2010

I was just using Amarok 2-3 days ago and everything was working fine. Now every time I launch Amarok it shows 0 songs in my collection. Under my collection properties, it still shows I have the correct folder selected. When I try to rescan my collection, absolutely nothing happens. Also, when I quit the program, it stays in the top gnome bar. When I click on it, the exit option is gone and I have to manually kill the process.

I'm running 10.04 with the 2.6.34-5 kernel.

View 8 Replies View Related

Software :: Amarok Does Not Show Music Collection And No Sound When Web Browsing?

Aug 8, 2010

i'm on a ubuntu machine and i installed amarok, because i like it more than rhythmbox, so i configured the collection but the list doesn't show.. i click the option "update collection" but it does nothing i can play mp3 files if i right-click on them and open with amarok..., also when i'm using amarok i dont get sound on my web browsing lets say in ..... i get no sound. what can it be? by the way i'm using UBuntu not Kubuntu.

View 1 Replies View Related

Slackware :: Amarok Hangs At 10% When Scanning Large Music Collection?

Jun 6, 2010

After installing slackware 13.1 I start up amarok and when I go in and configure the settings and it starts to scan the folder and it either hangs at 10%, stops responding all together or crashes, the library is about 130 gigs of mp3s. I do not know where to start on this one. Amarok version 2.3.0

View 4 Replies View Related

OpenSUSE Multimedia :: When Try To Scan Music Collection / Amarok Don't Find Any Song

Feb 16, 2011

3 o 4 days ago I installed KDE 4.6 from KDE 46 repository and I upgrade my OS and some applications like Amarok. I had some problems so I erased .kde folder and reboot my computer. KDE 4.6 works fine and fast (in my opinion, faster than KDE 4.5) but I have problems with Amarok 2.4. When I try to scan my music collection Amarok don't find any song. I have a NTFS partition in my computer and I have all my music there. I mount the partition since fstab and in 6 months I didn't have any problem. I can use Clementine for exmple and scan all my songs perfectly, but not Amarok.

View 7 Replies View Related

Ubuntu Multimedia :: Find-out The Beats Per Minute (BPM) For Each Of The Songs In Music Collection?

Jul 10, 2011

I'm looking to find-out the beats per minute (BPM) for each of the songs in my music collection to put together various playlists and thought it would be as simple as adding that column of information to Rhythmbox or Clementine, but that fields shows-up as blank. Am I doing something wrong or do I need a scanner to add that meta information first? If so, do you fine folks know of a good one out there for a large collection?

EDIT: So, I installed Banshee because I've read it has a BPM scanner; however, it runs insanely slowly. It took from when I last posted this to now to scan my music collection and only has about two dozens songs' BPM computed. Is Banshee supposed to be that slow? Seems useless.

View 2 Replies View Related

Ubuntu Multimedia :: Recommend A Good Player For A Large (~900GB) Music Collection?

May 7, 2011

i've recently installed 11.04 and am giving banshee a shot. it seems pretty good although has crashed a few times. but when i import my music folders (about 900GB, 175,000 items .. and growing..) it takes days. that's not such a big problem because it only needs to be indexed once, i assume, ... but the UI is very slow now - so that clicking on an album can take several seconds to bring up the tracklist. also typing into the search box there is a large delay before the matches are shown. is this just to be expected for such a large db? i recall i had google desktop indexing and returning results almost immediately back on winblows.. other ones i have tried include rhythmbox , amarok, songbird .. but have not found any of them to be stable and simple enough to my liking.

can anyone recommend a good player , my essential requirement is a fast and efficient indexing - with tag support would be grand, but just based on filenames is ok too. drag and drop into a play queue like rhthmbox had would be nice.

View 9 Replies View Related

General :: Automation - Good Console-based Program To Automatically Manage Music Collection?

May 31, 2011

I like the convenience of having a music player that manages the placement of my music files based on the tags of the files, sorting into a root music folder with Artist/Album/01 - Track Title.mp3 sort of hierarchy. Previously, I was using Banshee for this feature, as even most of the other gui music players don't have this library management feature. Now, I've been trying to use console-based applications, and I have been using mpd + ncmpcpp fairly successfully. When I download new music, it is downloaded to a specific folder, and that's not part of my music collection. My previous workflow would be to open a gui filemanager at the downloaded folder, and drag and drop those songs onto banshee, which would then make a copy of those files in my library hierarchy. Currently, I'm doing the same thing, using Banshee only as a sort of import program, then closing it and updating my mpd database, and there we go.

If at all possible, I'd like to automate this to something where I can define a watch folder, and when some magical program sees new music in said watch folder, it identifies the correct place for the files to be copied into my heirarchy, renaming folders and files to the correct format. I know I've written a lot describing, but I don't actually think this too uncommon a task, and I'm wondering if anyone has a solution for my problem.

View 3 Replies View Related

Ubuntu Multimedia :: Media Cleaning Script - Scan My Music Collection's Meta Data

Jul 14, 2010

I'm looking for a way to scan my music collection's meta data to find songs that don't have leading Capitals.

Example...

I want to change the file: eric clapton - its in the way that you use it artist: eric clapton; title: its in the way that you us it to...Eric Clapton - Its In The Way That You Use It

Is there a way that I can query my music to find words that don't start with capitals and change them?

View 1 Replies View Related

Ubuntu :: Deleted Folders During Grsyc Backup On The Destination Hd?

Mar 31, 2011

i took a back up using grsync and stupid me, the "delete on destination" was enabled.So, i lost some folders and files that i used to keep only on the receiver hd to keep my host clean

View 5 Replies View Related

Server :: Backup / Restore - Importing LVM Setup At Destination

Jun 8, 2009

I am doing a LVM replicate to another server.
Example: server1.foo.com has / , /boot , swap and few LVM partitions. All are in /dev/sda disk of size 80GB. /dev/sda5 is a LVM partition which has only one vg00 and it has 2 LV's (/var and /usr) and a SAN storage connected to this server which has around 500GB of single partition(a LV partition) called /data and its under vg00 .

Now I have build another server called "server2.foo.com" with same RedHat OS version - RHEL 4 and want to import the same LVM setup at destination. Down time or unmounting the filesystem is not a problem (but am trying to reduce the down time as much as possible). Is there any way that I can take a proper snapshot of whole LVM disk and restore with same setup at destination without losing the data and the lvm file system layout?

View 4 Replies View Related

Ubuntu :: External Hard Drive Will Not Let Backup Files?

Jan 10, 2010

I've been working at this for the past 2 days now. My computer got some kind of virus or something that has caused it to loop at startup and continually reset. I run an XP OS on a Gateway. I desperately need to backup my files, because the person who had my backup absently deleted my stuff. I was able to boot up using an Ubuntu disc and I'm in it right now, I've found my files, I have an external hard drive. The problem:First, it wont let me paste into the hard drive. If I drag, it says "Error while copying to "/media/FreeAgent Drive". You do not have permissions to write to this folder." I've mounted the external drive, nothing changes.

I've gone in to properties, is says under permissions that the owner is root, folder access is "Access files" and at the bottom is says "You are not the owner, so you can't change these permissions." The drop downs where I need to change permissions is in gray, so I can;t change it.So next, I tried "gksu nautilus", went to the drive through there, and it let me use the drop down selection under permissions. I tried to change the folder access and I got this message: "The permissions could not be changed. Couldn't change the permissions of "FreeAgent Drive" because it is on a read-only disk." So I tried changing the file access to Read and Write. It didn't give an error, so I thought perhaps it finally worked. I hit apply, and tried to put my files in. Once again I got the message from before that said I didn't have permission. I tried to change the owner so it was no longer root and I got "The owner could not be changed. Couldn't change the owner of "FreeAgent Drive" because it is on a read-only disk."I'm getting so frustrated right now. These files are VERY important to me! The hard drive I have is a Seagate FreeAgent desk 500GB

View 9 Replies View Related

General :: Keep A Backup Of A Bunch Of Files On A Flash Drive?

Apr 29, 2011

I keep a backup of a bunch of files on a flash drive, so that whenever I change distributions I can just restore all my Android stuff (saves on re-downloading everything). One of these is the Android SDK.

In my ~/.bashrc I add the paths to some executables in the SDK, only if the directory exists, and only if the path is not already in $PATH. For the Android NDK this works fine, but for the SDK I get this:

Code:
snfo@snfo:~$ adb devices
bash: /home/snfo/Android/sdk/platform-tools/adb: No such file or directory
snfo@snfo:~$ ls -F /home/snfo/Android/sdk/platform-tools/adb
/home/snfo/Android/sdk/platform-tools/adb*

Everything else is fine though, just that one path is causing trouble.

Now, I've saw something similar to this before whenever you move an executable from one place to another. If you don't re-source your bash config it will continue to keep looking wherever it used to be located. But I've never moved these files.

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved