Ubuntu :: Is There Better Way To Do Such Marathon Copies?
May 30, 2010
Let's say you want to copy some files from one drive (ext3 over USB) to another (ext4 in local box). You have a few terabytes over a couple million files.You could just use Nautilus, which is what I did. It seemed to have completed normally (after a few days).I wanted to verify that everything copied accurately, but I'm not sure the best way to do this. If I used md5sum, the verification would take all week, so that won't do. I used "du -bs *" and compared the sizes of the folders at the root of the drives. Most of the folder sizes were off somewhat - less than 1MB in any case. For each folder whose size was different, the difference was a multiple of 4096 bytes. This makes me suspect that the information du is giving me isn't precisely the information I assume it's giving me.
Baobab gave me different sizes from what du gave me, and it also gave significantly different sizes between my source and destination drives.I wanted to do a "diff -r source destination", but I don't even have a guess as to how long that would take so I gave up after a couple hours. It wouldn't tell me everything I want to know, anyway (if files are missing vs if files did not copy correctly).
So the help I need is:
1.) Is there a quick way to reassure me that everything copied correctly?
2.) Is there a better way to do such marathon copies? (faster and more reliable?)
Whenever I start my netbook up lately, it shows that I have three copies of Ubuntu (at least there are three text copies.) Only the third one works, because whenever I load the first two, it does not get any farther than a command prompt-like text screen that says something like "Kernel Panic." I know that that is about the equivalent to Windows' "blue screen of death," and I would like to know how to get rid of the first two.
I do not know if this is exactly related, but it all started when I tried to do an update, and apparently Compiz is an unauthorized program (which is silly, since I got it from the Software Center. Now, in the third Ubuntu option, the Compiz is non-existant (I used to have multiple desktop areas where I could do work, and have gotten quite used to having many windows open at once, but now, even the icon that I used to use to switch between them isn't there.
I am trying to copy the files from my WHS disk to my Ubuntu Server disk. I have the windows disk mounted at /media/WINDOWS and I want to transfer to /storage so I ran;Code:sudo cp -r /media/WINDOWS /storageIt takes about 4-5 seconds and is complete, but there is about 500 GB worth of data there so I know it didn't really copy everything over. When I look at the files in console it shows them, but when I look at the /storage through SAMBA on my Windows machine, it only shows the directories.?
When I select the "Open with...Other Application" context menu item I get a dialog box with multiple copies of the available programs. All of the offending program entries are Wine programs.How do I remove the extra programs from this list.this is a small problem but kind of annoying..
I deleted Windows from my Notebook and am only rockin Ubuntu on this machine. On the (previously) shared Data Partition, there are dozens of GBs occupied by the Volume Shadow Copy Service backups (which I used with Windows) but now how do I get rid of these?In Windows you could either turn off VSS/Restore under System Protection or alternatively use the Disk Cleanup Utility. How do I do this with Ubuntu?
I recently inherited a server that I'm now looking after, and when browsing the running processes I came across a number of processes labelled:"/usr/local/apache-2.2.6/bin/.libs/lt-httpd -k start"
At any one time there are between 10 and 15 of them running.Googling the issue has been surprisingly unhelpful... can anybody even tell me what this process is, much less what sort of issue might result in multiple running copies? I'm running Debian Etch (release 4.0).
I am trying to get my network music player (Logitech Squeezebox) to connect consistently with Ubuntu. Most of my music (FLAC files) is on a separate NTFS hard drive as I am dual booting with XP.I have edited the Fstab file somewhat based on suggestions on the forum but I can not seem to consistently connect to the Squeezebox with my music files (FLAC). What appears to happen is I set up my mount point in the media folder as Music and configure this with my Squeezebox which scans the Hard drive and finds all my music. The player works and it all seems fine even after a reboot but after a couple of reboots I get an error message when booting that says "The disk drive for media/Music/Flac is not ready yet or not present".
When I go to the media folder it appears that the Music folder is duplicated with a folder called Music_ and after a resetting the Squeezebox scanner to the new Music_ folder the process starts over again and I eventually end up with another folder called Music__ and so on.My question is 1) can I delete these Music folders without erasing all my Music library , and 2) how do I correct this situationHere is the Fstab file:# /etc/fstab: static file system information.
rsync -r -v -e ssh root@nn.nn.nn.nn:/usr/local/websites/* /usr/local/websites and each time I run it it copies everything - all files. I thought rsync was only supposed to copy files that had been added or modified.
I made a Bash script that is fired by a Cron job every morning. It dumps an SVN backup on some Samba shared drive. I would like to know how I can make sure the job worked correctly without having to verify the shared drive every morning. Right now, I take the job's output, save it to a log file and send this file by email. But the ouput isn't so great.
If I initiate a file copy of more than a couple of GiB, the PC goes into a dramatic slowdown. Even selecting a different subdirectory through Nautilus can take 30+ seconds.Now, whilst I appreciate that file copying puts a load on the bus, DMA and, to a certain extent, the CPU, it seems unconscionable that it makes the PC effectively unusable until the copy has completed.Is there any way to (practically) lower the priority, or similar, of the copy process so that one can continue to use the platform during large file copies? Right now, I end up using a laptop adjacent to the PC whenever I have to copy a large file. This, for a mainstream operating system is, frankly, ludicrous.I'm aware that I could run Nautilus 'nicely' but I don't want to make changes which would compromise other aspects of the system. It would also be pleasant, for a change, not to have to read a couple of telephone directories of technical documentation in order to resolve the problem myself. This must be a general problem and, in my view, something which seriously compromises the usefulness of Ubuntu given the sizes of contemporary drives and files.
The i7 I'm using at the moment has 8 cores non of which go over 15% usage while the copy is progressing despite the OS being effectively frozen for long periods.Environment:Intel Core i7-2600 @ 3.40GHz; 16GB RAM; Natty (11.04) fully updated as at 29th July 2011; Kernel 2.6.38-10-generic, GNOME 2.32.1.; Primary drive has 563.9GiB free space, Running in Ubuntu 'Classic' (no effects) mode
Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes
I am trying to make a backup copy of a DVD without success. k3b simply stays there silent doing nothing and I have to terminate the application. Is there a way to make DVD copies if one needs to?EDIT:I am using the home partition for caching as it is the biggest one on the disks; /tmp is too small.
(Running openSUSE 11.1 with KDE 4.3). I finally got my iPod Touch (2nd Gen) mounted using ifuse and libimobiledevice, using the following command: Code: ifuse ~/iPhone And I was really pleased to finally copy some music files onto my iPod using gtkpod. I navigated to the ~/iPhone/iTunes_Control/Music folder and found the folders F00 to F49. Output of ls -l */*:
I'm almost certain the files have been successfully copied to the device, because I've unmounted it, using: Code: fusermount -u ~/iPhone and when I re-attached it, the files were still there.
So I just have a few problems: 1) gtkpod won't update the iPod database, even though a dialogue window opens and says it is doing just that. Does anyone know why this might be? 2) Dolphin doesn't recognise that the iPod is mounted? the command dmesg recognises it, but there's no fstab for it. I wonder if anyone knows how I could create a udev rule to automatically mount the iPod when I plug it in. And how can I make dolphin recognise it? 3) Amarok doesn't recognise that it is mounted. I have both 1.4 and 2. I presume this is for the same reason that Dolphin doesn't recognise it.
I am trying to run two different copies of vsFTPd service in the same server, one for IPv4 and the other one for IPv6. Because as I know that you cant run one vsFTPd server for IPv4 and IPv6 in the same time.
the title says it all. im using CLI streamripper with this command:
Code: streamripper http://<stream IP address> -d ~/Music -z -o larger -T it keeps a copy of every song it rips in /incomplete then copies it to another folder once it's finished ripping (but keeps the copy in /incomplete). how do i get it to stop this and delete the copy in /incomplete once its finished moving it to the new folder?
Difficult to describe, but basically the desktop background image has a slightly smaller copy of itself (roughly 70% of the original) overlayed in the upper-left hand corner, like a picture-in-picture type of effect. This occurs on both the login screen,d the wallpaper I have set under my account. It seems to be isolated to ONLY the wallpaper, and not affecting any icons or functionality of anything else that I've noticed. Logging out and rebooting have no effect on the issue.
It has been such time that I have been making back-up copies of my DVDs using dvdrip. Lately I tried to back-up DVD of Iron Man2 and was surprised to see so many titles under Table of Contents. 93 titles of which re all running 2 hours plus. And so I ripped the recommended one which is the longest running title. Surprisingly, the output is all screwed up. Meaning scene one is at the end along with some scenes from the middle of the film and so on. Is there any way to correctly choose which of the 93 titles is the right one please? Obviously, I can't go through the motions of doing it one by one. Or would it be more practical to just copy the whole DVD image and burn in another DVD instead of making a HD back-up?
I'm trying to set up rsync to only copy new songs from my computer to another. I'm using the "--ignore-existing" argument, but it appears to copy all files anyway. The client (source) is Windows 7 64-bit running DeltaCopy Client and the server (destination) is Synology DS410 (running rsyncd).
I'm using postfix and the always_bcc option to backup the emails which are passed through my MTA. The problem is that (with spamassassin and clamav running as virtual-smtp agents) I get three copies bcc-ed through to the backup account. Is there something I can do to stop the bcc from being carried out on the internal filters and only work on the final send or mailbox delivery?
I have a desktop and a laptop both running opensuse 11.2 with kde4. I have a samba share on my desktop. I tried opening a video on that share from my laptop (wirelessly) with Dolphin/SMPlayer. Here's what happened:
The video started downloading and the system tray notified me it would take 25 minutes. I thought that was too long (video is 350MB) so I checked the download speed and it was about 2 MB/s. It didn't make sense but I let it keep going.
25 minutes and 3.4 GB later, the download finally "finished"--according to the system tray. However, I checked my system monitor and something was still downloading at 2 MB/s. I confirmed with "df -h" that I was losing 2MB of space a second. At this point I only had about 700MB of disk space left so I rebooted (I wasn't sure how else to stop the download).
After digging around on / I found my video at /var/tmp/kdecache-londy/krun and it was 350MB. Then I found multiple copies of the same video, of varying sizes, on /tmp/kde-londy totalling 3GB.
I deleted the tmp files and tried it again. This time instead of clicking on the video to play it, I tried copying it to my laptop. Same thing started to happen but I didn't let it continue.
I have a 64bit linux server with 5 virtual hosts on it. When someone fills out a contact form on one of the sites...I get 15-20 copies of the same email. At first I thought it was the kids clicking send multiple times because the first emails were coming from the children's ministry "Email The Cast" section. But then I started getting multiples from the adult sites too. All contact forms are set to come to me.
What's stranger is that my registration section for one of the sites uses the SAME php script (different file) to email me a notification that someone has registered but I only get 1 copy of that.
As an example, I have two servers, sm-i222 and fileserv. sm-i222 is a Win2k3 system running cygwin. fileserv is a linux box running RHEL 4.7. sm-i222 maps /cygdrive/c to the c: drive and /cygdrive/d maps to the d: drive(actually a single 4TB RAID). from the sm-i222 server /cygdrive/c I call a small script from the crontab. The internal IP for fileserv is 10.0.0.7. See below.
These three lines perform well in that they make a full transfer of the fileserv:/home/ directory on fileserv to the appropriate place on sm-i222 using rsync. I use rsync instead of scp because I have to traverse subdirectories and symbolic links in the /home/... filesystem on fileserv. What I'm looking to do is use rsync to do an incremental transfer/backup of only the files that have changed since the last full backup. I'll manage the times I do this manually or in crontab. A colleague says this is do-able, but not how. Rsync.org says this is do-able but not how. Cygwin says this is do-able... see rsync.org. I believe what I'm looking for is a single rsync line like I have above that only transfers the changed files on fileserv to sm-i222.
I know that fork() copies the address space of the calling process. Say, however, i have a linked list allocated. Will the list be copied over to the child process's space? If so, i would have to free them in the child process as well as the parent process, correct? Or will the variables be copied but not be pointing to any valid address? Or would it just kind of not do anything?example:
Is there an open source application that can download e-mails from a generic POP mail server, keep a copy of the mails locally and then deliver copies to multiple accounts on a LAN? I have a friend who uses a group e-mail account, and more than one person is always hooked onto that account. The problem is they cannot leave e-mail on the server, as it eats out their webhosting space, but if someone downloads a message in Outlook, the other users do not see that message.
I want to take a graphics file and make 10 copies of it to the same directory, each with 001, 002, or some such designation at the end of each file name so they have discrete files names. Is this possible using cp?
