Ubuntu :: Video Projects On 3TB USB Drive - Persistent Rsync?
May 30, 2010
I have a slow 3TB usb drive that I keep my video projects on, but when I'm working on a project I want to have the files on my main HD so that the video editing is less painful. So here is my question: is there a way to have a folder on my main computer that syncs files *as they are needed* to my main drive, then when they are edited sync them to the USB drive again?
So if I have a folder with 100 clips on my USB drive, can I sync just the directory listings to the local folder, then when I add a video to my project file and it is read by the video editing software, then just that file is synced to my local folder? Then, when I close the terminal with this app running, or some other signal the local folder is destroyed (all of the files are synced to the USB drive once they are changed).
View 4 Replies
ADVERTISEMENT
Sep 4, 2009
I just did a fresh install of Fedora 11 and the only video resolutions available were 640x480 and 800x600.
The native resolution of the monitor is 1280x1024 and that's what I'd like to use.
I was able to get the display to the proper resolution with the following commands:
Code:
However, rebooting sets it back to 800x600. I have seen the edits to xorg.conf but that file wasn't created.
Is there a way to make the 1280x1028 resolution persistent without creating a xorg.conf file?
View 8 Replies
View Related
Mar 27, 2010
I've just installed UNR version of Ubuntu on one of my thumb drives and was wondering what I needed to do to for making it have persistent changes? I've seen the different tools and the usb creator they have on Ubuntu already but I want to do it myself from scratch. What files do I need to edit to make the drive persistent? I made the bootable thumb drive using UNetbootin.
View 9 Replies
View Related
Feb 16, 2011
Yes, I know, I know, there are lots of instructions out there on how to create a persistent Linux USB drive. However, I've been having a really hard time finding if it's possible to create the WHOLE thing persistent.For example, I want to turn my 8 GB thumb drive into a portable Debian Squeeze where I can install (persistent) apps and make root-level changes to the filesystem. Is this possible?
View 2 Replies
View Related
Jul 11, 2010
I am trying to install Linux (the distros I have attempted it with are Arch, Fedora, Ubuntu and Mint) on a USB drive and make it work like a removable hard drive, keeping programs and settings. I tried it manually at first, partitioning the drive with Fedora's "Disk Utility" and dd'ing a Fedora 13 iso over. I should note here that I have definitely configured the BIOS correctly, enabled booting from removable media and set it as the default with all other devices disabled, but that I have never actually booted from USB before with this motherboard. On bootup I got
Code:
DISK BOOT FAILURE, INSERT SYSTEM DISK AND PRESS ENTER
I then tried it with Ubuntu 10 and Ubuntu's "usb-creator". This was apparently successful, but on bootup I got:
Code:
missing operating system
DISK BOOT FAILURE, INSERT SYSTEM DISK AND PRESS ENTER
I downloaded UNetBootIn, but the application kept saying I needed "p7zip-full", which I couldn't find anywhere. I then got Fedora liveusb-creator, but whichever iso I give it I get this error:
Code:
Unable to find LiveOS on ISO I looked at the source code and it seems to be looking for a directory named LiveOS on the iso containing the files "squashfs.img" and "osmin.img" Here is the code (usr/lib/python2.6/site-packages/liveusb/creator.py, line 575):
Code:
def extract_iso(self):
""" Extract self.iso to self.dest """
self.log.info(_("Extracting live image to USB device..."))
[code]...
I couldn't find much about what LiveOS actually means and why I need it to create a bootable USB, so if anyone could tell me more about this that would be great. Is this (the .img files) the only thing distinguishing a "Live" OS from a non-Live one? I looked in my Ubuntu live CD and there was no such directory, but it works perfectly well. In case it would make a difference, the stick is 8GB and branded duracell, not sure what manufacturer it is.
View 8 Replies
View Related
Jul 27, 2009
I just tried Centos 5.2 Live starting from a 2 GB USB flash drive. Everything seems to run fine, fast, stable - except for that the persistent feature is not working. I created the USB from Windows using the Centos 5.2 LiveCD image and the current version of Live USB Creator (3.7), and declared a 256 MB persistent space.
This persistence feature had worked before with Fedora 11 but the system resulted unstable, kernel panic.... Now Centos has been solid for hours in a row... but the file where persistence should be reflected remains untouched with the initial creation timestamp. When rebooting, every change in config, file created etc gets lost.
View 2 Replies
View Related
Dec 30, 2015
I installed debian 8 on a 16 GB usb drive using this guide. I used a debian 8.2 64-bit image with mate. If I were to get a larger usb drive, would I be able to transfer everything from the 16GB drive to it? How?
View 1 Replies
View Related
Jun 11, 2011
I have a 16Gig usb drive and i've managed to get Backtrack up and running on it with persistence, but I really want to have on it is Mint, Backtrack 5, if possible Ubuntu and hirens bootcd. I don't really want to use unetbootin. how i need to set up the partitions, if it's possible to have both Backtrack 5 and Ubuntu with persistence since they both use casper-rw and what mint needs for persistence.
View 6 Replies
View Related
Oct 28, 2015
I'm trying to create a persistent live Jessie system on my 8GB USB drive.
If that matters, I'm currently on an Arch Linux system, and I partly followed what's on the relative wiki (Pages Create a new MBR for a USB stick, Manually create a USB flash installation and Install Syslinux), plus a CrunchBang post explaining how to make a persistent live USB out of any Jessie-deriving distro (like their BunsenLabs Hydrogen).
The problem is, even if Debian boots up more than fine, the system isn't persistent at all.
Here's what I did (I know some passages are redundant, but still...):
Downloaded the Cinnamon flavor of Jessie via torrentErased the old MBR
Code: Select all# dd if=/dev/zero of=/dev/sdb bs=512 count=1 && syncCreated a 1.1G W95 FAT32 (LBA) active partition and used the remaining space on a Linux partitionFormatted the first to FAT32 and labelled it "Debian64". Formatted the second to ext4 and labelled it "persistence"
Code: Select all# mkfs.vfat -n Debian64 /dev/sdb1
# mkfs.ext4 /dev/sdb2 -L persistence
Mounted the first partition and the iso
[Code] ....
View 5 Replies
View Related
Feb 3, 2011
I am trying to backup my ~ directory to a portable drive. In the meantime rsync is giving me all kinds of messages like this:
[Code]...
This drive is neither plugged into my computer nor is it involved with my backup script:rsync $base_options -narv /home/noleks /media/Personal1/Linux_bak. I think the question is, does rsync have a queue somewhere of operations it did not complete? If so is there a way to rest this? It is possible something went awry with a previous sync on this drive but I cannot remember right now. Either way why is it trying to complete that sync when I'm asking it to do something else?
View 2 Replies
View Related
Jan 26, 2011
I just installed a new HD on my system with multiple HD's already. I have a drive with two versions of Ubuntu & would like to copy the complete drive to the new drive along with all the contents & partitions of the Ubuntu drive.
1 - Could I partition the new drive & just copy the contents using rsync?
2 -If I copy all the contents over could I just reinstall Grub & edit fstab & be good to go?
View 3 Replies
View Related
Jan 15, 2010
I have a 1TB USB external drive, currently formatted as fat32. What I need to do is copy two folders and all their subfolders, totaling about 500GB, to that external drive. The USB drive will have to transfer back and forth between RHEL, Windows XP, and Mac OSX computers freely.What format should I go with on the USB drive, FAT32 or NTFS?What rsync switches should I use? I know I don't want to use -a because I don't want any permissions restored. I'm guessing I'll have to run rsync a couple times to fully get all the files, so I need to be able to cancel an rsync, then have it pick back up where it left off, not start over and recopy every file again.
View 8 Replies
View Related
Mar 8, 2011
I am running Debian squeeze on a Seagate Dockstar. It boots Debian from a USB flash drive, and has two 1 TB usb hard disks hooked to it. The machine is used for only rtorrent and nfs. The flash drive contains only the system software and rtorrent. All of the data that rtorrent moves around is on the usb hard disks, including downloaded data, watch folder, and session folder. About once a day I run rsync with the source being usb_1 and the target being usb_2: # time rsync --progress --stats -a --delete /media/usb_1/ /media/usb_2 Since this machine boots from a usb flash drive, I'm interested in keeping the number of writes to the flash drive to a minimum. The machine used to boot from an Adata flash drive, which has no LED. That drive wore out after about five months. Completely dead, as in gparted would fail with an I/O error on write when trying to write a new partition table to it. No big deal - I restored the software to a new flash drive using an image I had made in Clonezilla.
The new drive (a Sandisk Cruzer 4 GB) has an LED and I assume this is showing me when the drive is being accessed, read and write. When the system is at idle, the LED sort of "breathes" off and on slowly. When rtorrent is downloading or uploading, it flashes a few times and goes back to breathing.
I'm assuming that once everything has been loaded and the system has been running a while that when the LED flashes, that means that data is being written to the flash drive. (Is this a good assumption?) With just rtorrent running, there's not much flashing, maybe 10 - 20 times/minute. However when rsync is running, the LED just flashes on and off like crazy. The daily rsync run takes anywhere from three to 30 minutes, depending on how much new data was downloaded each day. But no matter how long it takes, the LED just flashes like crazy the whole time.
I'm assuming this means that all the new data from the source drive is being written temporarily to the flash drive or something like that, and this is wearing the flash drive out prematurely. Does this sound likely? If so, is there a way to get rsync not to do this and to do its thing in RAM only and not write to the boot flash drive?
View 12 Replies
View Related
Feb 17, 2011
I'd like to know if it's possible to automatically mount, and fire up rsync to sync a USB drive with a directory? Specifically, I'd like to copy as much data as the drive can hold and only delete the oldest files if space is needed. I would assume I'd do something like this with a script, but my problem, is where to start.
View 2 Replies
View Related
Jan 18, 2010
anyone shed some light into these errors I keep getting for multiple files when I run rsync in verbose mode to a FAT32 external hard drive?
rsync: stat "/mnt/usbdrive/Batch 2/od venezuelan" failed: No such file or directory (2)
rsync: recv_generator: mkdir "/mnt/usbdrive/Batch 2/joseph gangster" failed: No space left on device (28)
I have 588GB available on my external hard drive, and i'm only trying to rsync 200GB to it, so its not out of space or anything like that. I've tried rsync with -r -t --size-only --delete and -r -t --modify-window=1 --delete and both ways seem to give me the message. or should i just reformat as NFTS and start over? i already had another 200GB data set copied on here, was trying to add a second. its all JPG pictures.
View 14 Replies
View Related
Nov 30, 2010
I have a home network that includes a couple of computers {A, B, C, D}. currently, I have a cron jon that runs every minute and updates (using rsync) the hard drives of computers {B, C, D} with the contents of hard drive {A}. So far everything works great, as hard drive {A} barely has any information on it. Now, I am about to copy a lot of information (about 8 GBs) to hard drive {A}. Naturally, the cron job will run (as it runs every 1-min) and try to 'sync' the contents with hard drives {B, C, D}.
Given my network (100Mbit/sec), there is no way the cron job will be able to 'copy' the contents to hard drives {B, C, D} in one minute. It will take much more time. Does this situation create a problem? meaning, will cron re-run a new rsync instance 1min later, even though an existing rsync process is running and still copying information to hard drives {B,C,D}? Will my backups be hurt / slowed down tremendously because of this?
View 2 Replies
View Related
May 20, 2011
I have an OpenBSD and a FreeBSD system and a mac. I also have a Ubuntu server. What i would like to do is back up all these systems to an external hard-drive using rsync when the external usb disk is connected to my Ubuntu box.If i format the external usb disk with cfdisk and the create a non-bootable ext3 file system on this external disk and create and put all the necessary public keys on the Linux box then from the BSD's or the mac issue the command:
Code: #rsync --progress -avhe ssh --delete / user@ubuntuBox:/usb/disk/path/dir/ Will this back up the entire systems so that they can be restored in the event of an emergency? I should store each OS just in a separate disk file of the external usb drive each time right?? Because i would rather not have to format the external usb drive for each different OS. Would this work? and would the restoration command for these BSD's be:
Code:
rsync -avze ssh UbuntuBox:/usb/disk/path / I just need to know the basics. I'm sure given that i'll be able to automate the process. I don't want to clone the disks for forensics. I just want to have a way of restoring to a clean OS. This is the most basic question:All the howto's never mention whether or not you have to have an rsync server running on the machine your backing up to. So do you just push or pull from one end of the connection only or do you have to have a client at one end and a server at the other, as is traditional?
View 3 Replies
View Related
Apr 10, 2010
I have a WD world book edition 1TB NAS drive, and just purchased an acomdata 1tb drive and connected it to the NAS via USB. If I recall I think the WD NAS has a ext_ or some type of linux filesystem on it, and the acomdata has a ntfs filesystem on it.
What I want to do is copy over certain directory trees of the NAS to the USB attached drive. I usually use MS synctoy to sync folders from my windows pc to the NAS drive, and MS richcopy to make the initial transfer from PC to NAS. For this operation though, since it is taking place entirely on the NAS and its connected drive, I thought that rsync would be the best option, and it is available on my NAS drive.
Last night I entered in rsync -avr /movies/* /usb1-1share1/ to copy the entire "movies" dir to the drive, which shows up as usb1-1share1 on the NAS drive. It copied most of the directory tree ok, but a lot of the folders were empty, so this morning I tried rsync -Carv --ignore-existing /movies/* /usb1-1share1/ to try and get all the files missed, without recopying the 24GB that did make it across. This also managed to copy a few more GB over, but not everything.
I am running the command from an ssh session on the NAS using putty on my PC, in as user "admin" which should have all rights over these folders. There is a bunch of errors in the command window like this: rsync: failed to set times on "/shares/usb1-1share1/movies/classics/fulldvd/First Blood DVD/.VTS_01_2.VOB.RxdjWZ": Operation not permitted (1)
I want to restart another session and get the files it missed, but I want to find out what I am doing wrong first. Should I be doing this as root user? am I missing some switches or just plain doing it all wrong?
View 3 Replies
View Related
Jan 16, 2011
I've been trying to get a cold backup of a 1TB database this weekend, started the whole process Friday and still have yet to get a single device backed up. I'm using rsync to copy files from my /u17 thru /u29 mounts, and the usb is formatted ext3. Each time the rsync would start off fine but after about 30 minutes it would fail with any number of errors but the most prevalent is "Read only file system", "broken pipe". Here are samples:
rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown" [sender]: Broken pipe (32)
rsync: write failed on "<path to one of my .dbf files" failed: Read-only file system (30)
rsync: chown "<path>" failed: Read-only file system (30)
rsync: rename "<path of .dbf> -> <rename attempt>": Read-only file system (30)
rsync error: error in file IO (code 11) at receiver.c(305)
rsync: connection unexpectantly closed (16787 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c (359)
I've unmounted and remounted a number of times and kicked off the rsync again and it goes about 30 minutes and I get the same errors. This was all as user 'root', so I tried to do the rsync as user 'oracle' and I get the same thing. After looking into the device as it is recognized, it is being picked up by multipath. Would the fact that a usb device is being managed by multipath be a problem? Currently it is mpath15. How would I add usb devices to the mpath blacklist? The usb is being assigned /dev/sdbj but I'm worried that it would change at a reboot. I've searched the web for all of these errors and still no answer.
Note: I've also just tried to do a copy using 'cp' and got the same "Read only file system" errors. I can sometimes touch a file and sometimes I can't. I want to try and get this backup done this weekend.
View 9 Replies
View Related
Dec 3, 2009
i am trying to get centos 5.4 installed and bootable on my 16gb flash drive, with persistent overlay using and ext3 formatted drive.i want to be able to boot into centos and be able to have all updates from yum, etc, saved when shutting down for my situation i cannot use vfat.
View 9 Replies
View Related
Mar 6, 2010
I'm currently learning to use rsync to backup my music collection. I have a Firefox tab open to the rsync manual page(s) and have been reading man rsync and running experimental rsync operations.I've been doing this for the last 3-4 hours. I've used rsync for this purpose in the past with disastrous results. What was and is once again (due to a month and a half of file pruning) a 9000 file music collection had mysteriously grown to over 25,000 music files and 80GB of data! This was likely due to the fact that I didn't really know what I was doing with rsync and had never spent too much time learning about all the parameters, what their functions are and how they may relate to my goal.Here are the particulars:
* Source drive is a 500GB disk, /media/sata500/music/.
* Destination drive is a 250GB USB disk, /media/FreeAgent/music, connected to the same computer that houses the 500GB disk.
* I want to copy or backup files from /media/sata500/music to /media/FreeAgent/music.
* I do not want to create ANY duplicates of files that exist.
* I only want to add files to the destination drive if they are new on the source drive, like if I rip a CD and add the contents to the source. I want them copied over next time I run rsync.
Here's the rsync command in it's most recently used form, and probably very immature at this point.
Code:
rsync -t -r -vv --stats -i --log-file=/home/glenn/rsync.log /media/sata500/music/* /media/FreeAgent/music/
This appears to have copied all files and folders and I'm satisfied that my goal has been met with some success. To convince myself of this I ran the command and then once it was complete I added 2 new songs putting them in their respective folders on the source drive and ran the same command again. The resulting output was
[code]....
Two files transferred. Exactly what I want.Both folders now house 20,931 files and use 40.6GB. Identical as far as I can tell.What I'm concerned about are time stamps and play count data, etc. Anything that changes the original file. I don't want this data to cause a file to be transferred as I'm afraid that the new file will be created along side the old file of the same name thereby starting this whole music collection expansion thing all over again. I've invested a lot of time and effort to get it pruned down to where there are virtually no duplicates and albums are correct in that they contain the proper songs in the proper order.
View 14 Replies
View Related
May 24, 2010
I'm trying to setup rsync to backup a remote directory to my local drive.
I cd to the directory that I want to pull the files to, then I enter:
rsync -vrtW account@remote.com:~/public_html
I enter the password then it starts running. I get all the files listed, but none of them actually transfer. What am I missing?
View 1 Replies
View Related
May 9, 2011
I have never used rsync before, only DD. But from what I have been reading, rsync is better becasue it will basically mirror your hard drive, thus being able to run the cloned software from the new hard drive. My problem is I do not know what is the best commands or even the basic commands to use in rsync. I am trying to make an image from a external hard drive to a usb drive. That way my chances of messing up he original software is not as risky becasue I'll just restore the image onto another hard drive. Does anyone know the best script to have rsync make an image file of a hard drive and place it on a usb drive and then restore it?
View 8 Replies
View Related
Jan 31, 2010
I don't understand this: I did Quote: rsync -avz --delete /media/PPT-WORDS centguy@centos52-64-dell:/media/ while I was at home connecting my portable hard drive to a desktop centos52 machine. On the next day, I came to the office and I connect my portable hard drive to another centos52 machine, do do
[Code]....
View 3 Replies
View Related
Nov 17, 2010
Thought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
View 6 Replies
View Related
Jan 7, 2011
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
View 9 Replies
View Related
Aug 24, 2009
CentOS 5.2 64bit 2.6.18-92.el5xen. Use rsync with --link-dest for nightly backups, works well. Was recently asked to start weekly backups to an external drive for off-site storage. The regular syncing works but hard linking seems to be ignored. So the backup is long with no space saving advantage. Here is an example of the command being run:
rsync --stats -axzvl --numeric-ids --delete --link-dest=/mnt/DISASTERBACKUPS/austinBackups/backups/2009-08-21 /AUSTINBACKUPS/backups/2009-08-24 /mnt/DISASTERBACKUPS/austinBackups/backups/
View 4 Replies
View Related
May 20, 2011
I have an OpenBSD and a FreeBSD system and a mac. I also have a Linux server. What i would like to do is back up all these systems to an external hard-drive using rsync when the external usb disk is connected to my linux box.If i format the external usb disk with cfdisk and the create a non-bootable ext3 file system on this external disk and create and put all the necessary public keys on the Linux box then from the BSD's or the mac issue the command:
Code:
Will this back up the entire systems so that they can be restored in the event of an emergency? I should store each OS just in a separate disk file of the external usb drive each time right? Because i would rather not have to format the external usb drive for each different OS. Would this work? and would the restoration command for these BSD's be:
Code:
I just need to know the basics. I'm sure given that i'll be able to automate the process. I don't want to clone the disks for forensics. I just want to have a way of restoring to a clean OS. This is the most basic question:All the howto's never mention whether or not you have to have an rsync server running on the machine your backing up to. So do you just push or pull from one end of the connection only or do you have to have a client at one end and a server at the other, as is traditional?
View 1 Replies
View Related
May 31, 2010
As my proficiency with Linux improves slowly, I've been trying to find the answers for myself, but in this situation I must admit I find myself rather stumped. I have a perfectly nicely working Fedora 12 install on an 80GB SATA drive, and when it hit an error and wouldn't boot last week (easily fixed with fsck from the initial command line) I panicked and ordered a new 250 GB drive. It got here and I might as well use it, I thought to myself, so I went about trying to figure out how to move my install without having to reset all of my settings, programs and so on. I didn't want to mess with dd because I'm not so so clear on resizing my partitions once the copy is done (if someone thinks this is a better idea I'm open to suggestions.) After some poking around I found this set of instructions which I attempted to follow to the letter, but hit some snags. I understand this thread I am referring to may be a bit outdated, which is why (I assume) I hit a bump here
Code:
# mount /dev/hdy1 /boot
mount returns an error demanding I specify the file system type. At a loss, I barreled on until
Code:
[Code]...
To summarize, I partitioned and mounted my new drive using fdfisk and the instructions provided above, then used rsync to copy over all of the files, so as far as I know the new drive is ready to go, just not yet bootable. Opening the Grub.conf file in Kwrite (as root) returns a blank page. What do I do now?
As a side note, you can see that I am not too squeamish about the terminal, so I would prefer to find a "command line only" solution to this relatively simple (?) procedure.
View 5 Replies
View Related
Apr 12, 2011
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
View 6 Replies
View Related
