Ubuntu :: Partition The New Drive & Just Copy The Contents Using Rsync?
Jan 26, 2011
I just installed a new HD on my system with multiple HD's already. I have a drive with two versions of Ubuntu & would like to copy the complete drive to the new drive along with all the contents & partitions of the Ubuntu drive.
1 - Could I partition the new drive & just copy the contents using rsync?
2 -If I copy all the contents over could I just reinstall Grub & edit fstab & be good to go?
I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
I want to copy my ubuntu install to a bigger hard drive, and am not quite sure what to do. According to my google searches, I need to run ubuntu from a live cd, then in a partitioning program copy the ubuntu partition to the new one, then resize it. Is that all? Do I need a linuxswap partition on the new hard drive? I have been using kde partition manager to arrange my new partitions. On one hard drive I have the partition I want to install ubuntu on(what type should this be? ext4?) and a partition to share between ubuntu and windows, and then will use my old ubuntu partition for installing windows xp.
i am putting a larger drive in my laptop, i have linux mint 10 KDE setup with all the software i need and running just the way i like it. is it possible to actually copy the entire partition to a external drive then place the partition back into my laptop with the new drive in it, and still have it all setup the way i had it?
basically so i dont have to reinstall everything and set it up again.if this is possible could you please explain how i can do it in the simpliest terms at all please.
I have a very specific issue that I am having trouble resolving. I have an old laptop and a new laptop with a smaller HDD. I want to copy the windows partition from the new lappy to the old bigger HDD so I have room for Ubuntu. All of my files are on a Maverick install on the old lappy. How can I get all my files and windows to the old HDD and into the new laptop. I am a little stuck on this one because of my limited options.
I would like to copy the contents of a directory into another. I don't want to copy the directory and all files and directories under it, but just the contents of the directory just as if it were a regular file. Doing cp -r target dest copies the directory and the entire hierarchy rooted in it. I get error if I do not include the -r option. (I am calling cp from within a C program.)
Is there a way to copy a directory (retaining the permissions and owners) without copying the contents of the directory?
If there is no such thing... then I need a way to determine if a target path is a file or a directory, and if it is a directory I need to make a new directory elsewhere that has the same name, owner and permissions.
Basically, I'm trying write a script to copy 200 GB of files over a network to a new server, and I'd like to do it by generating a list with the find command. That way, I can migrate large chunks of the files over the course of a week, and on the day of the migration generate a new list of files that changed in the last week and then copy just the chagned files over minimizing the down time. However, the list will contain directories that I can't just use the 'cp' command on because it will copy all the contents of the directory.
how to copy 3 dir's content to 1 dir by crontab?suppose i want to copy /home/ftp1/* /home/ftp2/* /home/ftp3/* to /ftpdatathree ftp user data to one folder after every one minute by crontab methodso it goes like*/1 * * * * /bin/cp -rf ??? /ftpdata
Is there a way to copy a directory without copying the contents, but preserving ownership, timestamp etc of that directory?
I've looked at the cp man page, but I don't think it supports it. I'm thinking one would have to write a script to gather the info, and then mkdir, chown and touch. Does this seam right?
My system is an Acer Aspire One Happy netbook. It runs Windows 7 OEM. I installed Ubuntu Netbook Remix, which works fine. Unfortunately, I put GRUB in the MBR (I wanted it on Ubuntu partition in order to use Win 7 bootloader). Acer netbooks seem to have a particular MBR (in order to run e-Recovery Management), which is different to the standard Windows 7 MBR.
Moreover, in the internet there are instruction to restore Acer MBR, but they do not seem useful, since I do not find the MBRWRWIN.EXE and RTMBR.BIN files on the hidden partition. Is there a way to tell GRUB to remove itself and restore the previous content of the MBR (something like the old "lilo -u")? Does GRUB make a copy of the content of the MBR before installing itself? Where is it, if it is so?
I am using CentOS 5.5 and openSUSE 11.0. I use vim editor but do not know more than 1 - 2% of its functionality. In openSUSE if i open 2 text files in 2 separate terminals, i can copy the contents of one file onto another by just left-clicking and dragging the mouse over the required text. But i am unable to do this in CentOS.
I even threw some video DVDs at it to make sure it wasn't the disc.
Code:
[pickens@acer1 Videos]$ dd if=/dev/sr0 of=POTC.iso dd: reading `/dev/sr0': Input/output error 5088+0 records in
[code]....
I am getting the same thing on my laptop running Mandriva, oddly enough. Two different drives, two different computers, two different distros and multiple DVDs.
I've created other users in my machine. now I want to add all my home directory contents and settings to the home directory of other users. how can i do that? Can I do it from /etc/skel directory?
I have a server that I wanted to transfer it to a newer one both of them have CentOS but the newer one kernel is more up to date I wanted to know is it possible just to copy some directory contents exactly to another for transferring the server data (for example /var /usr /bin /home /etc). I have one website on my server with its mysql database
I was wondering if there would be a way for me to copy the contents of a shared folder that's running on an XP system using shell script. Actually, there are two shared folders that I want to do this with. I want to be able to launch the script and automatically copy all the contents of each folder into two separate folders on my ubuntu desktop. For example;
Lets say that I have Shared Folders A1 and A2 on my networked XP system and I have Folders B1 and B2 on my Ubuntu desktop. I want the script to automatically copy the contents of A1 into B1 and likewise for A2 and B2. During the copying, I want the script set up such that any pre-existing files in the B1 & B2 folders will automatically get overwritten by the ones copied from A1 & A2.Is there any possibilities of me achieving this?
I'm having to rebuild my home server following a failed upgrade from 8.04 to 10.04 . All data lives in /shared on this server, the contents of which are mirrored weekly to a USB HDD which is mounted at /backup, using rsync:
I recovered the contents of the backup drive to the rebuild server's /shared directory using the cp command with the archive flag set to preserve ownership, timestamps etc. Everything looks fine to me. However, when I do a test rsync (adding -n to the command above) then it looks like rsync wants to recopy everything, and I'm at a loss to see why. For example, below is a test on the subdirectory /shared/backgrounds. The file attributes look identical:
Code:
chris@quadra:~$ ls -la /shared/backgrounds total 8112 drwxr-xr-x 2 chris chris 4096 2009-04-12 11:06 .
[code]....
Is there some condition that rsync can detect that I can't see? Is rsync sensitive to the way the HDD is mounted? (the USB HDD is actually mounted at /media/backup, and /backup is a soft link to this.)
I have a newly installed Dell Optiplex 755 with Ubuntu 10.04 (64-bit) and I am having serious issues with network copying. Whenever I start rsync och scp with larger amounts of data my system becomes practically unresponsive (all types of apps grey out) until those processes are completed. Cpu stays at below 10% and I have lots of free memory and such .
Using Ubuntu Server 10.04 LTS. I'm new to Ubuntu and testing Rsync. I successfully copied 3TB of data from a Win7 machine to an MDADM Raid5 array. All appears to be fine. Used a Win app for the copy. I then deleted a 250GB folder on the Raid5 array and recopied the data using Rsync. Rsync was executed via a Putty session on a WinXP machine. The source was an eSata attached drive (same drive used for the big 3TB copy) and the destination was the same Raid5 array. That copied just fine. I bit verified it with a Win7 app. Perfect.
I then used the following Rsync script to copy a single 26GB file from that same eSata drive back to (what I intended to be) the Raid5 array: Code: neil@ANTECUBSV:/mnt$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test sending incremental file list created directory /mnt/Test ./ C_VOL-S300-b001.spf sent 3020267622 bytes received 34 bytes 50760800.94 bytes/sec total size is 3019898880 speedup is 1.00 neil@ANTECUBSV:/mnt$ cd raid
Note that only about 3GB copied. No error messages were posted to the putty session. I made a mistake in the Rsync command, creating the Test folder directly in the mount folder rather than the Raid array, as I intended. That is a little strange, yes, but I would not think it would cause a partial copy? The /mnt folder is on my system drive, which had about 34GB available space before the copy, so comfortably would have had 6GB or so after. The eSata disk is mounted as /mnt/disk1 The Raid5 array is mounted as /mnt/raid
I then recopied the file to the correct intended destination on the Raid5 array, which has about 400GB free space (plenty). Code: neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/raid/Test sending incremental file list ./ deleting 2010-07-05 Backyard Birds/Thumbs.db deleting 2010-07-05 Backyard Birds/ C_VOL-S300-b001.spf sent 11105775462 bytes received 34 bytes 50366328.78 bytes/sec total size is 11104419840 speedup is 1.00 neil@ANTECUBSV:/mnt/raid$ df -h
Note that only about 11GB was copied, and this was confirmed with an ls -l command. Now I am correctly copying the file to the Raid array but it is still incomplete. I then copied the file back to the /mnt folder to see if the problem reproduces: Code: neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test sending incremental file list created directory /mnt/Test ./ C_VOL-S300-b001.spf sent 26327927554 bytes received 34 bytes 56558383.65 bytes/sec total size is 26324713984 speedup is 1.00 neil@ANTECUBSV:/mnt/raid$ cd /mnt/test
This time I got my full 26GB file. Why I might be getting inconsistent results? This is quite troubling of course. I'd also be interested in basic a command line Linux diff app (that does file directory as well as bit level checking) if one is available.
For the life of me I can not figure out what I am doing wrong with scp to copy a directory and its contents from a remote machine to my local host. I have no issues with getting a single file but would like to just save time and get the whole folder in one command.
Here is what I have tried:
scp user AT remoteMachine:/home/username/folderIwant user AT localMachine:/folderIwant this gives me a permission denied error and try again and received disconnect from localHost to many authentication failures
scp user AT remoteMachine:home/username/folderIwant . says can not find file or folder
I am sure this is something easy that I cant remember, and searches gives me local to remote not remote to local and trying to make the local to remote suggestions I read to work remote to local have not worked.
I'm trying to view directly a partition with a damaged filesystem on it (NTFS) and so far the hex editors I tried do not do that. I tried GHex for example and it complained that /dev/sda is not a file. The partition is unmountable as NTFS however since it is damaged so I couldn't mount it first. Are there any hex editors out there that would allow me to view the contents directly and copy and paste stuff in there?
I have recently purchased an external hard drive in order to backup my home partition. In my PC I have a "1.5T" drive with several partitions on it, containing OSes and the home partition. The home partition is 1.3T according to df, the external drive contains one partition that spans the entire disk,df reports it as 1.4T in size. Both partitions are ext3. When I use rsync to copy files from the home partition to the external partition, the external disk becomes full, despite the destination - supposedly - being larger than the source. I don't understand why copying files from one partition to a slightly bigger partition should need more space than on the source partition. Does anyone know what is happening ?
Details : I created the partition on the external drive with gparted; gparted reported it the already have several gigabytes in used space immediately after the partitions creation - I thought at the time that this must be normal. The home partition contains many files of all sorts, including lots of big audio and video files. If you are wondering, for all my important files this external disk is only secondary backup, as they are also backed up to the "internet".
These are the mount points :
/mnt/tmp/ : home partition, /dev/sdb6 /mnt/external/ : external partition, /dev/sdc1
I used rsync to copy the files, I know there are more efficient ways to do this, but I wanted to use the same command that I will subsequently run to sync the backup.
Next I tried adding the --sparse switch, as I was wondering if the problem may come form sparse files. I don't know however if rsync would go back and shrink the sparse file by just adding the switch and executing the command. I also added --one-file-system, for good measure. Here is what I ran next :
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32) rsync: write failed on "abcd.avi": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.6]
[code]....
Looking at the destination after a partial copy seems to indicate that the problem is not symbolic links being "expanded". I have not checked the source filesystem for sparse files, nor the destination to see if these files could be larger there, as this does not seem trivial.
Here is some additional info :
$ df /mnt/tmp/ Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb6 1415342836 1414173740 369096 100% /mnt/tmp
On my Ubuntu box, I have a mounted windows share connected via gvfs called graphics. I want to backup everything on a nightly basis from graphics to backupserver/graphics . If I use rsync, it will not copy files that have parent directories with funky characters in them (but the directories themselves will be copied!). Everything else gets rysnced just fine.
graphics/test/macdir/picture.psd ...when rsynced over to ... backupserver/graphics/
gives the error:
rsync: mkstemp "/home/administrator/.gvfs/drobo on x.x.x.x/linux_backups/graphics/test/ macdir/.picture.psd" failed: Operation not supported (95)
The directory macdir gets created but there is nothing in there. This happens for all files underneath dirs with funky names. cp -Rf works perfectly! Directory and child files all get copied over no matter how strange the characters get in the directory names.
I am trying to create a simple bash script to rsync some folders within a directory stucture. I am using wild cards, in the rsync source directory structure, but my command always fails. I believe it is the way I am using wild cards within my for loop. Here is my command ;
Code:
for seq in `cat test.txt` ; do rsync -nvP /folder/folder/folder/folder/folder/**/$seq /folder/folder/folder/ ; done This always fails, where if I do a ls to the destination, to test the path, it always works.
For reasons I won't get into, I need to copy directories so long as the average system load is low. Can someone help me write a BASH script that will copy the contents of a directory, but check to make sure the average system load is below X before copying each file, and if not, wait Y seconds and try again?