General :: Rsync To Copy Directory Tree From NAS Drive ?
Apr 10, 2010
I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
View 3 Replies
ADVERTISEMENT
May 14, 2011
when I installed 13.37 I created a local copy of the entire stable tree (source/ and all the rest) just to have all that stuff around to browse offline.
Now, to instruct myself, I'm trying to use rsync to keep this stuff up to date. But I seem either to have misread the rsync man page or ... well, I don't know. I am issuing the following command and getting the results seen below:
Code:
View 3 Replies
View Related
Aug 19, 2010
I have ubuntu 9.10 & windows 7 on my laptop. I have one 40GB Primary partition (C & 60 GB logical partition on my windows. I have given another 20GB for ubuntu.
I hibernated my windows7 & then started ubuntu 9.10. Then I copied a folder containing some PDF files from my ubuntu to that 60GB (D on windows. Then I rebooted the machine & choose windows from GRUB. the windows came up from hibernation but nowhere I was able to see that folder which I copied.
Since ubuntu supports (understands) NTFS file system it means when I copied that folder it should have updated the Directory Tree of NTFS on that 60GB (D but that folder is not shown.
When I restart the windows the folder appears (ofcourse because windows checks the file system again for consistency while in hibernation it does not).
View 3 Replies
View Related
May 24, 2010
I'm trying to setup rsync to backup a remote directory to my local drive.
I cd to the directory that I want to pull the files to, then I enter:
rsync -vrtW account@remote.com:~/public_html
I enter the password then it starts running. I get all the files listed, but none of them actually transfer. What am I missing?
View 1 Replies
View Related
Jun 1, 2011
How can I use rsync to copy ONLY the my home folder (and nothing inside of it, just the folder name) to another machine. I've tried things like
Code:
rsync -av /path/to/src /path/to/dest/
or
Code:
rsync -av -f"+ */" -f"- *" /path/to/src /path/to/dest/
This last option recursively (through the -a switch) copies only folders, including all subfolders. If I try
Code:
rsync -v -f"+ */" -f"- *" /path/to/src /path/to/dest/
nothing is copied (not even my home folder.
View 9 Replies
View Related
Jun 15, 2011
I am trying to create a simple bash script to rsync some folders within a directory stucture. I am using wild cards, in the rsync source directory structure, but my command always fails. I believe it is the way I am using wild cards within my for loop. Here is my command ;
Code:
for seq in `cat test.txt` ; do rsync -nvP /folder/folder/folder/folder/folder/**/$seq /folder/folder/folder/ ; done This always fails, where if I do a ls to the destination, to test the path, it always works.
View 5 Replies
View Related
Jun 11, 2011
How do you copy everything in the home directory to an external drive while running in rescue mode?
View 3 Replies
View Related
Jan 26, 2011
I just installed a new HD on my system with multiple HD's already. I have a drive with two versions of Ubuntu & would like to copy the complete drive to the new drive along with all the contents & partitions of the Ubuntu drive.
1 - Could I partition the new drive & just copy the contents using rsync?
2 -If I copy all the contents over could I just reinstall Grub & edit fstab & be good to go?
View 3 Replies
View Related
Sep 27, 2010
I am trying to use the cp command to copy some files around but want to preserve the entire folder structure. Example:
Then within the /backup location I want to see /backup/folder1/folder2/file.
I have tried cp -a and cp -p but neither give me the above.
View 3 Replies
View Related
Oct 29, 2010
Are there any other archiving tools than tar that preserve Linux file permissions and user/group flags? I want to create archives of a directory tree on my ubuntu box. The directory trees are large, a first run with tar cvpzf archive.tgz /home/foo/bar yielded a 5GB archive. I also want to keep all permissions and other flags and special files.I'm fine with a 5GB archive, however to look inside that archive -- since it is a compressed tar archive -- the whole 5GB have to be decompressed first! (Or so it appears when opening it with the archive viewer -- I'm happy to be corrected.)So I need a way to "backup" a directory (tree) while preserving full filesystem attributes and right and creating an archive with an "index" that hasn't to be decompressed for browsing its contents. An archive is either a single file, or a (small) set of files that carries full and complete information within this file/s. That is, it can live on any filesystem (size permitting), it can be burnt onto a DVD, you can split it (after which the point of this question is really lost - but still), ...
View 3 Replies
View Related
Feb 11, 2010
I am trying to ftp a whole directory tree using vsftpd as ftp server in fedora11
Code:
I am getting error while transfering a directory tree ...
View 1 Replies
View Related
Sep 30, 2010
This little fragment does the opposite of what I want and I am trying to figure out how to reverse it. I want to strip off the directory and echo just the filenames. I cannot predict how many levels deep the directory tree is.
View 6 Replies
View Related
Jul 19, 2011
I would like to overwrite files in a directory tree, recursively. The ones I would like to overwrite match the filename "x_alpha*.png" and have a size exactly 456 bytes. Is there any way to search for these recursively in a directory tree, and overwrite them with a reference file, for example "e:mydirgood.png"
I am using Windows 7, but I have UnxUtils, so I can use those too. What I am looking for is something like this, generated automatically:
copy /y e:mydirgood.png e:mydiracx_alpha0023.png
copy /y e:mydirgood.png e:mydirefgx_alpha0045.png
copy /y e:mydirgood.png e:mydirhx_alpha0248.png
View 2 Replies
View Related
Feb 9, 2010
I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6
/mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
rsync -av --progress --stats --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync -av --progress --stats --sparse --one-file-system --recursive --perms --links --delete /mnt/tmp/ /mnt/external/
I tried an fsck on the home partition :
fsck -f /dev/sdb6
This is the output from the last rsync :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "abcd.avi": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
[code]....
View 2 Replies
View Related
May 13, 2011
I am in my current directory. I want to copy a directory somewhere else into this current directory. Lets say I want to take it from direc1/direc2 and the directory I want to take is called demo.
Code:
That is what it shows in the man pages, but when I do that, it says cp: no match
View 5 Replies
View Related
Nov 8, 2010
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
View 2 Replies
View Related
Jul 28, 2011
I am performing a dry run using Rsync on 2 different boxes.While i'm doing that, Under destination directory, I want a specific directory x to be ignored for sync.Please let me know the exact pattern to ignore the directory.The current command I'm using is:rsync -avnc --delete $LOCAL_DIR $USERNAME@$DESTINATION_IP:$REMOTE_DIRunder DESTINATION_IP, I would want to ignore a particular directory under REMOTE_DIR.
View 1 Replies
View Related
Nov 30, 2015
I have a home storage of many drives which are seldom accessed, and extremely seldom written to. So, I made several RAID1 arrays with "write-mostly" drives in each one, so that all the drives spin down after a while, and if they are accessed, only one drive in each array has to spin up. This way, I hope to minimize the risk of losing anything due to mechanical shock or wear.
But it was not so easy. First, the write-mostly drives did spin up on every read request due to a bug in the kernel, so I managed to get the fix accepted into the newer kernels (which is my largest contribution to the Linux development so far. See [URL] .....) Now, if I read directly from the array, it works as expected (only the first drive spins up); but if I create a filesystem there (ext4) and read from it, the second drive still spins up anyway. (Does ext4 write something when it reads a file?.. I have it mounted with relatime, so AFAIK it shouldn't...)
Well, I thought, I'll just mount the filesystems read-only and remount them if I need to modify the data there, which is not very often, to say the least. Also, this way the drives don't have to spin up every system shutdown to unmount the filesystems; during a power outage, it used to take a lot of time for them to spin up one by one just to unmount, while my UPS had to survive long enough for them to finish. That's one more problem solved.
But I decided to improve even further. Sometimes I just want to see the contents of those arrays without even accessing some files (for example, when I accidentally click the wrong network drive button in Windows, or when I just want to see if I have something there or not). If I read the directory trees during startup and keep them cached, then those drives won't even have to spin up at those moments! The question is, is there a way to do it? I remember Linus boasting about our kernel being the best at caching the filesystem trees, and I know there is vmtouch [URL] .... which allows us to keep some files cached, but what about the directory tree itself?.. If I simply "find" all the files there, it seems to work, but they get evicted from cache sooner or later...
View 3 Replies
View Related
Sep 25, 2010
If transfer all files under a directory by rsync, what is the order that rsync determines to transfer the files one by one?At first it looked like rsync transfers files in alphabetical order, but later I found rsync skipped some files in the first sweep through the alphabetic order, and then went back to transfer files that were skipped in the first time and this time still in alphabetic order.
View 1 Replies
View Related
Apr 19, 2011
Is there a way to force rsync to not make directories in its destination directory; ie, to simply dump all of the files from the source directory directly into the destination without copying any of the folders that the files were originally in? I tried --no-dirs, but that seems to only be for empty directories.
View 1 Replies
View Related
Mar 23, 2010
Recently i am trying to check on the rsync speed for single file(2.4GB iso) directory ( 900MB directory with files inside ) When i run the rsync for single file: the speed i get is average 50MBps However, for a directory: average speed is 10MBps Is there any reason behind this ? i tried to google but unable to get the concept.
View 2 Replies
View Related
Jul 14, 2011
We have an rsync cron job set up to mirror all the files in a "..dashtdocsdocs" folder to the same folder on another server. It copies all the files over correctly and deletes any files in the "docs" directory that aren't in the sending directory, but it also deletes any files we put in the target directory's parent folder (..dashtdocs or other subfolders like ..dashtdocsimages) even though they've been excluded in the .rsync-filter file.
Here's our current rsync .sh:
/usr/local/bin/rsync --stats -qPzrtpl --delete --password-file=/var/run/.appprodrsync --log-file=/export/home
/webuser/logs/rsync-extranet-log -FF /export/home/ appprodrsync@appprod::dprweb_extranet/ > /export/home/webuser/logs/rsync-extranet-output 2>&1
[code]....
So for example server A has ..dashtdocsdocs and ..dashtdocsimages. Server B has ..dashtdocsdocs but if I manually copy the images folder over to ..dashtdocsimages, the images folder gets deleted from the target directory every time rsync runs.
A: ..dashtdocsdocs
..dashtdocsimages
B: ..dashtdocsdocs
..dashtdocsimages (<-- gets deleted everytime rsync runs)
I'd like to keep just the docs directory synched and update other folders manually, but they keep getting deleted. It looks to me like it's running a delete-excluded option, but that option wasn't used.
View 2 Replies
View Related
Jun 10, 2010
I've got a directory structure full of files. I want to convert them all into some other format, but I don't know how to keep the directory structure, is there a way so I can "set" which command to use on files instead of `cp`?
View 4 Replies
View Related
Apr 7, 2011
Why would rsync insert a user's home directory path in variable expansion when run via cron, but not when run manually. The gory details... Red Hat Enterprise Linux AS release 4 (Nahant Update 6) Linux 2.6.9-67.0.20.ELsmp The script (parts anyway, and simplified)...
Quote:
#!/bin/bash
. /home/bea/.bash_profile
echo rsyncloc=${rsyncloc} >> ${log}
[code]....
View 4 Replies
View Related
Jan 29, 2011
i would like to find and backup all *.mp4 files from /Pictures and its sub-directories and move them to a single directory on a remote. I can find and move the files but I don't want the directory structure...just the files to be placed in the remote directory.
To find my files I use
rsync -r -a -v -e "ssh -l user" --delete --include '*/' --include '*.mp4' --exclude '*' /home/drew/Pictures/ remoteserver:/Users/drew/mp4
but this creates all the subdirectories
I also tried
find ~/Pictures -name "*.mp4" -exec rsync -r -a -v -e "ssh -l user" --delete {} remote:/Users/drew/mp4 ;
This works but takes forever
View 3 Replies
View Related
Jan 18, 2011
I have a user account (no root access) on a Debian system. My disk quota for my home directory is limited to 250MB, of which I've used 150MB.
I would like to install a program that requires 95MB of space (let's call it prog-5.1) into $HOME/local/bin. Because the source files take up 1GB, I've had to place them in the /tmp directory (to stay under my limit).
Unfortunately when I run the config script, I receive the following:
Code:
The reason for this is that /tmp is mounted in noexec mode, meaning I can't execute code in /tmp directory:
Code:
I'm wondering if anyone has any ideas about running the config script in my home directory, and somehow pointing it to configure the files in the /tmp/prog-5.1/ directory.
View 5 Replies
View Related
Mar 16, 2011
I found it easiest to embed narp into the gtk+-2.0 installation because all I needed to do was tweak the Makefile.am and Makefile.in files and let configure do the rest.This is a hack. I've failed at getting my own makefiles to work.
Would anyone point me in the right direction on how to properly make an installer for Linux? Also, what is the correct/common directory tree for Linux software packages? Any more suggestions on proceeding with this project using community standards?
View 2 Replies
View Related
Jan 15, 2010
I have a 1TB USB external drive, currently formatted as fat32. What I need to do is copy two folders and all their subfolders, totaling about 500GB, to that external drive. The USB drive will have to transfer back and forth between RHEL, Windows XP, and Mac OSX computers freely.What format should I go with on the USB drive, FAT32 or NTFS?What rsync switches should I use? I know I don't want to use -a because I don't want any permissions restored. I'm guessing I'll have to run rsync a couple times to fully get all the files, so I need to be able to cancel an rsync, then have it pick back up where it left off, not start over and recopy every file again.
View 8 Replies
View Related
Jan 19, 2011
The problem was finding an applet to browse directory tree directly in the Gnome panel similar to that in "Recent Documents". The thread was closed without any solution posted there and also no solution available elsewhere (I searched for over 2 hours) in the forum.
Search for "file-browser-applet" in the Synaptic Package Manager. The applet with exactly that name is what you need. It allows to browse and open files on your computer from the GNOME panel. Once installed, "Add to panel".
View 1 Replies
View Related
Feb 17, 2011
I'd like to know if it's possible to automatically mount, and fire up rsync to sync a USB drive with a directory? Specifically, I'd like to copy as much data as the drive can hold and only delete the oldest files if space is needed. I would assume I'd do something like this with a script, but my problem, is where to start.
View 2 Replies
View Related
