Debian :: Run Rsync Without Writing To The Boot Drive?
Mar 8, 2011
I am running Debian squeeze on a Seagate Dockstar. It boots Debian from a USB flash drive, and has two 1 TB usb hard disks hooked to it. The machine is used for only rtorrent and nfs. The flash drive contains only the system software and rtorrent. All of the data that rtorrent moves around is on the usb hard disks, including downloaded data, watch folder, and session folder. About once a day I run rsync with the source being usb_1 and the target being usb_2: # time rsync --progress --stats -a --delete /media/usb_1/ /media/usb_2 Since this machine boots from a usb flash drive, I'm interested in keeping the number of writes to the flash drive to a minimum. The machine used to boot from an Adata flash drive, which has no LED. That drive wore out after about five months. Completely dead, as in gparted would fail with an I/O error on write when trying to write a new partition table to it. No big deal - I restored the software to a new flash drive using an image I had made in Clonezilla.
The new drive (a Sandisk Cruzer 4 GB) has an LED and I assume this is showing me when the drive is being accessed, read and write. When the system is at idle, the LED sort of "breathes" off and on slowly. When rtorrent is downloading or uploading, it flashes a few times and goes back to breathing.
I'm assuming that once everything has been loaded and the system has been running a while that when the LED flashes, that means that data is being written to the flash drive. (Is this a good assumption?) With just rtorrent running, there's not much flashing, maybe 10 - 20 times/minute. However when rsync is running, the LED just flashes on and off like crazy. The daily rsync run takes anywhere from three to 30 minutes, depending on how much new data was downloaded each day. But no matter how long it takes, the LED just flashes like crazy the whole time.
I'm assuming this means that all the new data from the source drive is being written temporarily to the flash drive or something like that, and this is wearing the flash drive out prematurely. Does this sound likely? If so, is there a way to get rsync not to do this and to do its thing in RAM only and not write to the boot flash drive?
View 12 Replies
ADVERTISEMENT
Jul 25, 2011
for the last three days, I have been trying to get my drive ( LiteOn iHas624 ) to work. Unfortunatly, it seems that I can only use the burner with Nero4Linux or Brasero, but not K3B.DVD-RAM/+RW packet writing seems neither to work out of the box, nor by the setup as suggested in /usr/share/doc/udftools/README.Debian or the media4linux howto(just to name the most promising, i definitly read more on the issue).
I tried to format the DVDs in various udf revisions, as well as ext2 just to check, but without success.As I am close to the point where I would rule out the possibility to obtain package writing capability, I dare to ask for help.I would be very grateful for any hint or reading recommendation on the issue.
Yours, Benjamin.
View 1 Replies
View Related
May 12, 2011
I was just wondering is 2.5mb a second normal to write to a usb2 pendrive with the 'dd' command?
I know dd maybe having an influence on writing time, but it still seems really low for usb2 transfer?
Is there a command to measure the time a file takes to transfer from HD to usb pen? (not using dd).
View 11 Replies
View Related
May 17, 2011
I am using a new install of Kubuntu 10.04 LTS. My system is a 64-bit AMD desktop.
I use a small partition for my entire Kubuntu install, and use separate, larger partitions for my media and work files.
I opened up Dolphin (my KDE file manager) via GUI, navigated to my media partition, and attempted to create a new folder by
right-click->
Create New->
New Folder-> etc
The Create New in the menu is ghosted out, as if I have no permission to access it.
How can I write files to the drive?
View 14 Replies
View Related
Aug 22, 2011
restore a partition images on another machine to the unused space on my laptop. I'm using netcat and dd to restore the partition.
On laptop:
nc -l 192.168.192.254 9000 | dd bs=1024 conv=notrunc,noerror seek=<some block pos> of=/dev/sda
On W/S:
dd if=part.img bs=1024 | nc 192.168.192.254 9000
The problem is that fdisk -l displays start & end sectors of the partitions and I need to know blocks. How do I determine the block number after the last partition to start writing this image?
Quote:
cat /proc/partitions
Displays blocks per partition. I could add those numbers up but is sure would be nice to have a utility that just give you what you need.
View 4 Replies
View Related
Sep 8, 2009
I have a relatively common problem, but I don't seem to identify it's source. I have a SAMBA server on my LAN to which there are mapped a few shares as network drives in windows xp (as Y: ) and mounted as CIFS in linux [as /y]. The problem is that every time I save a file [either windows xp or linux] on the mapped drive / mounted folder, our IDEs alert us that the file changes right after the save. I am running SAMBA 3.3.2.
View 1 Replies
View Related
May 9, 2010
I was wondering how to get a script that i am writing with emacs over to possibly a flash drive or onto the desktop.?
View 4 Replies
View Related
Jan 15, 2010
I have a 1TB USB external drive, currently formatted as fat32. What I need to do is copy two folders and all their subfolders, totaling about 500GB, to that external drive. The USB drive will have to transfer back and forth between RHEL, Windows XP, and Mac OSX computers freely.What format should I go with on the USB drive, FAT32 or NTFS?What rsync switches should I use? I know I don't want to use -a because I don't want any permissions restored. I'm guessing I'll have to run rsync a couple times to fully get all the files, so I need to be able to cancel an rsync, then have it pick back up where it left off, not start over and recopy every file again.
View 8 Replies
View Related
Feb 17, 2011
I'd like to know if it's possible to automatically mount, and fire up rsync to sync a USB drive with a directory? Specifically, I'd like to copy as much data as the drive can hold and only delete the oldest files if space is needed. I would assume I'd do something like this with a script, but my problem, is where to start.
View 2 Replies
View Related
Feb 3, 2011
I am trying to backup my ~ directory to a portable drive. In the meantime rsync is giving me all kinds of messages like this:
[Code]...
This drive is neither plugged into my computer nor is it involved with my backup script:rsync $base_options -narv /home/noleks /media/Personal1/Linux_bak. I think the question is, does rsync have a queue somewhere of operations it did not complete? If so is there a way to rest this? It is possible something went awry with a previous sync on this drive but I cannot remember right now. Either way why is it trying to complete that sync when I'm asking it to do something else?
View 2 Replies
View Related
Jan 18, 2010
anyone shed some light into these errors I keep getting for multiple files when I run rsync in verbose mode to a FAT32 external hard drive?
rsync: stat "/mnt/usbdrive/Batch 2/od venezuelan" failed: No such file or directory (2)
rsync: recv_generator: mkdir "/mnt/usbdrive/Batch 2/joseph gangster" failed: No space left on device (28)
I have 588GB available on my external hard drive, and i'm only trying to rsync 200GB to it, so its not out of space or anything like that. I've tried rsync with -r -t --size-only --delete and -r -t --modify-window=1 --delete and both ways seem to give me the message. or should i just reformat as NFTS and start over? i already had another 200GB data set copied on here, was trying to add a second. its all JPG pictures.
View 14 Replies
View Related
May 30, 2010
I have a slow 3TB usb drive that I keep my video projects on, but when I'm working on a project I want to have the files on my main HD so that the video editing is less painful. So here is my question: is there a way to have a folder on my main computer that syncs files *as they are needed* to my main drive, then when they are edited sync them to the USB drive again?
So if I have a folder with 100 clips on my USB drive, can I sync just the directory listings to the local folder, then when I add a video to my project file and it is read by the video editing software, then just that file is synced to my local folder? Then, when I close the terminal with this app running, or some other signal the local folder is destroyed (all of the files are synced to the USB drive once they are changed).
View 4 Replies
View Related
Jan 26, 2011
I just installed a new HD on my system with multiple HD's already. I have a drive with two versions of Ubuntu & would like to copy the complete drive to the new drive along with all the contents & partitions of the Ubuntu drive.
1 - Could I partition the new drive & just copy the contents using rsync?
2 -If I copy all the contents over could I just reinstall Grub & edit fstab & be good to go?
View 3 Replies
View Related
Apr 10, 2010
I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
View 3 Replies
View Related
Jan 16, 2011
I've been trying to get a cold backup of a 1TB database this weekend, started the whole process Friday and still have yet to get a single device backed up. I'm using rsync to copy files from my /u17 thru /u29 mounts, and the usb is formatted ext3. Each time the rsync would start off fine but after about 30 minutes it would fail with any number of errors but the most prevalent is "Read only file system", "broken pipe". Here are samples:
rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown" [sender]: Broken pipe (32)
rsync: write failed on "<path to one of my .dbf files" failed: Read-only file system (30)
rsync: chown "<path>" failed: Read-only file system (30)
rsync: rename "<path of .dbf> -> <rename attempt>": Read-only file system (30)
rsync error: error in file IO (code 11) at receiver.c(305)
rsync: connection unexpectantly closed (16787 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c (359)
I've unmounted and remounted a number of times and kicked off the rsync again and it goes about 30 minutes and I get the same errors. This was all as user 'root', so I tried to do the rsync as user 'oracle' and I get the same thing. After looking into the device as it is recognized, it is being picked up by multipath. Would the fact that a usb device is being managed by multipath be a problem? Currently it is mpath15. How would I add usb devices to the mpath blacklist? The usb is being assigned /dev/sdbj but I'm worried that it would change at a reboot. I've searched the web for all of these errors and still no answer.
Note: I've also just tried to do a copy using 'cp' and got the same "Read only file system" errors. I can sometimes touch a file and sometimes I can't. I want to try and get this backup done this weekend.
View 9 Replies
View Related
Mar 6, 2010
I'm currently learning to use rsync to backup my music collection. I have a Firefox tab open to the rsync manual page(s) and have been reading man rsync and running experimental rsync operations.I've been doing this for the last 3-4 hours. I've used rsync for this purpose in the past with disastrous results. What was and is once again (due to a month and a half of file pruning) a 9000 file music collection had mysteriously grown to over 25,000 music files and 80GB of data! This was likely due to the fact that I didn't really know what I was doing with rsync and had never spent too much time learning about all the parameters, what their functions are and how they may relate to my goal.Here are the particulars:
* Source drive is a 500GB disk, /media/sata500/music/.
* Destination drive is a 250GB USB disk, /media/FreeAgent/music, connected to the same computer that houses the 500GB disk.
* I want to copy or backup files from /media/sata500/music to /media/FreeAgent/music.
* I do not want to create ANY duplicates of files that exist.
* I only want to add files to the destination drive if they are new on the source drive, like if I rip a CD and add the contents to the source. I want them copied over next time I run rsync.
Here's the rsync command in it's most recently used form, and probably very immature at this point.
Code:
rsync -t -r -vv --stats -i --log-file=/home/glenn/rsync.log /media/sata500/music/* /media/FreeAgent/music/
This appears to have copied all files and folders and I'm satisfied that my goal has been met with some success. To convince myself of this I ran the command and then once it was complete I added 2 new songs putting them in their respective folders on the source drive and ran the same command again. The resulting output was
[code]....
Two files transferred. Exactly what I want.Both folders now house 20,931 files and use 40.6GB. Identical as far as I can tell.What I'm concerned about are time stamps and play count data, etc. Anything that changes the original file. I don't want this data to cause a file to be transferred as I'm afraid that the new file will be created along side the old file of the same name thereby starting this whole music collection expansion thing all over again. I've invested a lot of time and effort to get it pruned down to where there are virtually no duplicates and albums are correct in that they contain the proper songs in the proper order.
View 14 Replies
View Related
May 24, 2010
I'm trying to setup rsync to backup a remote directory to my local drive.
I cd to the directory that I want to pull the files to, then I enter:
rsync -vrtW account@remote.com:~/public_html
I enter the password then it starts running. I get all the files listed, but none of them actually transfer. What am I missing?
View 1 Replies
View Related
Nov 30, 2010
I have a home network that includes a couple of computers {A, B, C, D}. currently, I have a cron jon that runs every minute and updates (using rsync) the hard drives of computers {B, C, D} with the contents of hard drive {A}. So far everything works great, as hard drive {A} barely has any information on it. Now, I am about to copy a lot of information (about 8 GBs) to hard drive {A}. Naturally, the cron job will run (as it runs every 1-min) and try to 'sync' the contents with hard drives {B, C, D}.
Given my network (100Mbit/sec), there is no way the cron job will be able to 'copy' the contents to hard drives {B, C, D} in one minute. It will take much more time. Does this situation create a problem? meaning, will cron re-run a new rsync instance 1min later, even though an existing rsync process is running and still copying information to hard drives {B,C,D}? Will my backups be hurt / slowed down tremendously because of this?
View 2 Replies
View Related
May 20, 2011
I have an OpenBSD and a FreeBSD system and a mac. I also have a Ubuntu server. What i would like to do is back up all these systems to an external hard-drive using rsync when the external usb disk is connected to my Ubuntu box.If i format the external usb disk with cfdisk and the create a non-bootable ext3 file system on this external disk and create and put all the necessary public keys on the Linux box then from the BSD's or the mac issue the command:
Code: #rsync --progress -avhe ssh --delete / user@ubuntuBox:/usb/disk/path/dir/ Will this back up the entire systems so that they can be restored in the event of an emergency? I should store each OS just in a separate disk file of the external usb drive each time right?? Because i would rather not have to format the external usb drive for each different OS. Would this work? and would the restoration command for these BSD's be:
Code:
rsync -avze ssh UbuntuBox:/usb/disk/path / I just need to know the basics. I'm sure given that i'll be able to automate the process. I don't want to clone the disks for forensics. I just want to have a way of restoring to a clean OS. This is the most basic question:All the howto's never mention whether or not you have to have an rsync server running on the machine your backing up to. So do you just push or pull from one end of the connection only or do you have to have a client at one end and a server at the other, as is traditional?
View 3 Replies
View Related
May 9, 2011
I have never used rsync before, only DD. But from what I have been reading, rsync is better becasue it will basically mirror your hard drive, thus being able to run the cloned software from the new hard drive. My problem is I do not know what is the best commands or even the basic commands to use in rsync. I am trying to make an image from a external hard drive to a usb drive. That way my chances of messing up he original software is not as risky becasue I'll just restore the image onto another hard drive. Does anyone know the best script to have rsync make an image file of a hard drive and place it on a usb drive and then restore it?
View 8 Replies
View Related
Jan 31, 2010
I don't understand this: I did Quote: rsync -avz --delete /media/PPT-WORDS centguy@centos52-64-dell:/media/ while I was at home connecting my portable hard drive to a desktop centos52 machine. On the next day, I came to the office and I connect my portable hard drive to another centos52 machine, do do
[Code]....
View 3 Replies
View Related
Nov 10, 2010
I created a small C++ program which starts a server in a separate thread and waits the user to press q using the standard input/output. Something like:
Code:
printf(...);
server->Start();
[code]....
View 8 Replies
View Related
Oct 26, 2010
I've installed Squeeze on a USB stick, but can't get it to boot. I've had this problem before and gave up last time. I installed on an encrypted LVM - here is the grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by /usr/sbin/grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
[Code]...
I added rootdelay=10 and switched root from hd1,1 to hd0,0 as suggested elsewhere. Still no go, i jsut get dumped into ramfs shell with an error message saying that /dev/mapper/crunchbang-root doesn't exist.
View 1 Replies
View Related
Aug 24, 2009
CentOS 5.2 64bit 2.6.18-92.el5xen. Use rsync with --link-dest for nightly backups, works well. Was recently asked to start weekly backups to an external drive for off-site storage. The regular syncing works but hard linking seems to be ignored. So the backup is long with no space saving advantage. Here is an example of the command being run:
rsync --stats -axzvl --numeric-ids --delete --link-dest=/mnt/DISASTERBACKUPS/austinBackups/backups/2009-08-21 /AUSTINBACKUPS/backups/2009-08-24 /mnt/DISASTERBACKUPS/austinBackups/backups/
View 4 Replies
View Related
May 20, 2011
I have an OpenBSD and a FreeBSD system and a mac. I also have a Linux server. What i would like to do is back up all these systems to an external hard-drive using rsync when the external usb disk is connected to my linux box.If i format the external usb disk with cfdisk and the create a non-bootable ext3 file system on this external disk and create and put all the necessary public keys on the Linux box then from the BSD's or the mac issue the command:
Code:
Will this back up the entire systems so that they can be restored in the event of an emergency? I should store each OS just in a separate disk file of the external usb drive each time right? Because i would rather not have to format the external usb drive for each different OS. Would this work? and would the restoration command for these BSD's be:
Code:
I just need to know the basics. I'm sure given that i'll be able to automate the process. I don't want to clone the disks for forensics. I just want to have a way of restoring to a clean OS. This is the most basic question:All the howto's never mention whether or not you have to have an rsync server running on the machine your backing up to. So do you just push or pull from one end of the connection only or do you have to have a client at one end and a server at the other, as is traditional?
View 1 Replies
View Related
Mar 6, 2011
I need to mount a USB drive at boot up. I tried this entry in fstab:
Code:
LABEL=/Root-FS /Root-FS auto defaults,auto,user 1 0
But that did not do it. If I do a 'mount -a', that works.
This is for an embedded Debian system.
View 1 Replies
View Related
May 31, 2010
As my proficiency with Linux improves slowly, I've been trying to find the answers for myself, but in this situation I must admit I find myself rather stumped. I have a perfectly nicely working Fedora 12 install on an 80GB SATA drive, and when it hit an error and wouldn't boot last week (easily fixed with fsck from the initial command line) I panicked and ordered a new 250 GB drive. It got here and I might as well use it, I thought to myself, so I went about trying to figure out how to move my install without having to reset all of my settings, programs and so on. I didn't want to mess with dd because I'm not so so clear on resizing my partitions once the copy is done (if someone thinks this is a better idea I'm open to suggestions.) After some poking around I found this set of instructions which I attempted to follow to the letter, but hit some snags. I understand this thread I am referring to may be a bit outdated, which is why (I assume) I hit a bump here
Code:
# mount /dev/hdy1 /boot
mount returns an error demanding I specify the file system type. At a loss, I barreled on until
Code:
[Code]...
To summarize, I partitioned and mounted my new drive using fdfisk and the instructions provided above, then used rsync to copy over all of the files, so as far as I know the new drive is ready to go, just not yet bootable. Opening the Grub.conf file in Kwrite (as root) returns a blank page. What do I do now?
As a side note, you can see that I am not too squeamish about the terminal, so I would prefer to find a "command line only" solution to this relatively simple (?) procedure.
View 5 Replies
View Related
Jul 15, 2011
just installed ubuntu on my win 7 laptop using the wubi.. after the install it said restart,so i restatred and booted up ubuntu which went sound,it then says finishing installation.. ubuntu comes up does the partions then started copying files then once it had done this the screen went black with loads of writing on it..it seems to just stay like this.. i left it 30 mins the restarted laptop again..laptop working sound booted ubuntu again and the same thing happened..
View 1 Replies
View Related
Sep 18, 2015
I have got a 1TB USB hard drive, which I partitioned to be 500GB NTFS and on the other half I installed Debian 8.1.0. During graphical install I selected to install the bootloader not to the MBR but also to the external drive. After completing the installation I wanted to boot into Debian, but it just started Windows, which is installed on my internal. Even after choosing the USB drive in the boot menu, Windows booted. I later installed the bootloader to my internal, then I could boot into both Debian and Windows, but only if my hard drive was plugged in.
View 14 Replies
View Related
Feb 20, 2016
I have a 2TB external Hard drive that nonetheless is being used for booting Debian off of. I have downloaded the "debian-8.3.0-amd64-netinst.iso" and have extracted it to my external hard drive. The letter assigned to this drive is "I". When I shut it down and enter the boot settings, it asks me for a name and a path for a new boot option. I have tried many different paths including:
Code: Select allI:setup.exe
I:autorun.inf
I:debian.iso
setup.exe
debian.iso
I renamed the original Debian download (debian-8.3.0-amd64-netinst.iso) to "debian" so I didn't have to type the long file name into the path. When I type in "I:debian.iso" as the path and restart it pops up with a grub prompt, in my mind that tells me that some part of the debian.iso file is corrupted.
Specs:
Dual Core i5-3317U, 1.7 GHz, Turbo boosted
8GB RAM
1TB Internal Memory
64-bit OS and processor
Windows 8.1 Default OS
View 5 Replies
View Related
