OpenSUSE :: File Number Anomaly Between Twonky And Rsync
Apr 27, 2011
In short, using rsync to upload master database of music files to NAS which runs Twonky as media server. Rsync reports many more files being considered than Twonky reports as being available ie. 15K reported by rsync vs 13K flac and mp3 files reported by Twonky.Is there another easy way to count the number of files on NAS and in master database. I can mount NAS directory locally with NFS on same machine as master database.
View 1 Replies
ADVERTISEMENT
Jun 14, 2010
I've ripped some CD's to MP3. The track name on my HD is "trackname.mp3" I then transferred them to my mp3 player and they show up there again as "trackname.mp3" So.. ..instead I renamed the file names on my HD to remove the ".mp3" at the end. I did this for a several individual songs and a couple of albums and transferred them across to my mp3player. So far so good... they now list on the player as just "trackname"
But now I get to an album where this no longer works. Once I remove ".mp3" from the file name on my HD two things happen, 1. the icon is no longer an mp3 icon and 2. it just lists under properties as an "application. And of course the song wont play.
Can someone explain why I can do this for some CD's but not for others? They were all ripped the same. Is there a way around it as I don't particularly want the songs on my player to all end in ".mp3"
View 9 Replies
View Related
May 3, 2010
I have been trying to get this setup working for a day now, I have:
QNAP NAS box running Twonky 5.1.3
I can browse to it and use the web browser to see all of the media I have uploaded to it. I have a ubuntu lucid clean insalled system with rhythbox and the coherence DLNA plugin. I cannot get the rhythmbox to see the media on the server. Very briefly at one point I saw the server name appear as RB started, but it disappeared immediately. I have something like 5K songs up mostly in MP3 format.
View 6 Replies
View Related
Apr 9, 2010
I have a 2 node RH 5u4 64-bit. I have installed and configured the latest Veritas CFS (Cluster File System) which also uses Veritas Cluster Server. File system if VxFS. Storage is on EMC Symmetrix arrays with Veritas Mirroring between the arrays.We have noticed that running 'du -hs' on the shared directory/filesystem where it takes about 3 minutes on one node and 30 - 45 minutes on the other node.I've been running strace on 'du'.'du' runs an 'lstat' on each file (66,000+ files). On the slower node, the ave time spent in 'du' is about .001 seconds longer, which accounts for the 30-45 minutes. Also, the standard deviation is much larger, which means to me that the lstat times are all over the place!Another interesting thing is that iozone profiling shows that the i/o rates from both nodes are darn near identical, with no anomalies at various buffer & file sizes! And, iostat looks really good as does 'vxdmpadm iostat show
View 3 Replies
View Related
Mar 17, 2010
I formatted 56 GBs of my hard disk space into fat32 and it seems that 27 gbs of it is used! Though direct checking from the volume itself shows nothing like this, gparted still insists 27 gbs are used.
View 4 Replies
View Related
Jun 5, 2010
I have a script that I am using to try to measure network throughput over the ethernet ports. This is at work in order to test the performance of our onboard ethernet ports (planar) versus our pci-e (and daughter) ethernet pluggable cards.The script is very long (and calls other scripts), so I will try to only show the parts that seem to be producing this weird little bug:
Code:
#Robert de Bock's "A shell script to measure network
#throughput on Linux machines" served as the basis for
#this part of the script. Thank you for your help Robert!
[code]...
View 4 Replies
View Related
Oct 31, 2010
Code:
cat ${SOURCE}/{start,universal,index,end}.txt > ${SERVER}/index.html
cat ${SOURCE}/{start,universal,02042010,end}.txt > ${SERVER}/02042010.html
[code]....
View 3 Replies
View Related
Sep 17, 2009
I'm looking for a way to insert the number of lines in a file to the start of the aformentioned file. This should be simple but as I am not used to scripts in Linux, I am finding it tough going. I can find the number of lines in a file easily enough via
filesize=$(awk 'END {print NR}' $1)
but as for inserting this into the first line, i'm failing to do so. I've tried some of the other approaches on these forums but none so far have been able to do so.
I've tried:
sed '1i$filesize' $1
but sed i requires a string, not a variable so no go I've also tried:
mv "$1" "${1}.bak" 2>/dev/null || touch "${1}.bak"
cat $filesize "${1}.bak" >"$1"
but again with no luck as cat seems to need an input stream Just to recap, i want to insert a line at the start of a given file that holds the number of lines the original file has.
ie the file:
a
b
c
d
e
should become:
5
a
b
c
[code].....
View 3 Replies
View Related
Mar 10, 2010
i am trying to transfer a file from my live linux machine to remote linux machine it is a mail server and single .tar.gz file include all data. but during transfer it stop working. how can i work and trouble shooot the matter. is there any better way then this to transfer huge 14 gb file over network,vpn,wan transfer. the speed is 1mbps,rest of the file it copy it.
rsync -avz --stats bkup_1.tar.gz root@10.1.1.22:/var/opt/bkup
[root@sa1 logs_os_backup]# less remote.log
Wed Mar 10 09:12:01 AST 2010
building file list ... done
bkup_1.tar.gz
deflate on token returned 0 (87164 bytes left)
rsync error: error in rsync protocol data stream (code 12) at token.c(274)
building file list ... done
code....
View 1 Replies
View Related
Oct 21, 2010
I have 2 different mounts. One points to a local windows share(NTFS ->Samba) and the other one points to a PPTP VPN connection sharing(I belive that is NTFS too). I use "cifs" scheme in my fstab to mount these. And I use my Debian box to copy between these 2 mounts. I have started using Rsync for that purpose, I think that it works fine for now. My main problem is that it looks like Rsync cannot figure out if the files are same or not in source and target folders when I use these mounts. Most of the time Rsync copies the same files and folders over and over again even though those files and folders are on the target.
I am wondering if there is a way to make this scheme work? Being on a Vpn connection(slow) a Windows box, Rsync could have save a lot of my time if it could have recognized the files and folders that are same on both ends
View 4 Replies
View Related
Jul 1, 2011
I have successfully backed up my files using a script to a remote server with a log file output.However the log file is appended each time.I wish to have a different log file each time with date and time and have yet to figure that part out.
View 2 Replies
View Related
Mar 18, 2010
I'm running rsync in daemon mode on CentOS 5.4, and I'm seeing inconsistent times reported in the log file.
View 2 Replies
View Related
Nov 17, 2010
Thought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
View 6 Replies
View Related
Jan 7, 2011
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
View 9 Replies
View Related
Dec 1, 2010
The rsync module "opensuse-full" which worked well so far seems currently broken. It tries to mirror a huge number of additional stuff (factory?) but fails with "permission denied". Any place where I could report this?Command:
Code:
rsync -rlpt -hi -stats rsync.opensuse.org::opensuse-full /drive/repo
Errors:
[code]....
View 5 Replies
View Related
Apr 12, 2011
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
View 6 Replies
View Related
Sep 18, 2009
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
View 1 Replies
View Related
May 23, 2011
I made a shell backup script that uses Rsync and I am trying to get rid of the password prompt because it will use a CRON to run. I have set my variable in my shell script at:
PASSWORD_FILE=rsync_password
And the password in that file only takes up 1 line.
However when I run (ignore $DESTINATION)
rsync -aRvz tmp $DESTINATION --password-file=$PASSWORD_FILE
It still gives me the prompt. How can I accomplish this? I cannot allow a prompt and I do not want to have to use keys.
View 1 Replies
View Related
Mar 24, 2010
I'm always getting this error when I run the rsync command:
Code:
when I try to navigate the file:
Code:
it gives
Code:
View 2 Replies
View Related
Mar 25, 2011
I have a backup sh file that I have been using for a long time. It has always worked. 2 Days back I switched to a different pc and now suddenly the script don't work.If I run it manually in the terminal it works. But when it execute with cron it doesn't copy any files to the backup destination. It starts but doesn't copy anything.Can someone help me as to why it works manually but not with cron ?
View 9 Replies
View Related
May 16, 2011
I have an Ubuntu machine running NFS4 server and a plugapps (arch linux) machine connecting as the client. The plugbox is running an rsync job to backup the home directory from Ubuntu to a local USB HDD.
All of the files in the destination have owner nobody and group nobody.
Ubuntu /etc/exports:
Code:
/home 192.168.2.1/24(rw,sync,no_subtree_check,no_root_squash)
plugbox /etc/fstab
[Code].....
how I can mantain the file owners. I have the UID's and passwords sync'd between the two machines for both root and the user who's home dir is being backed up.
View 9 Replies
View Related
Mar 23, 2011
I'm going to be using this command to back up my files:
Should I change anything or is it ok?
View 4 Replies
View Related
Feb 16, 2010
I'm using rsync for a backup-sript at the moment and want to keep all files. The files are always unique, so I want to rsync without delete any file on the destination.
I've tried with --no-delete and --max-delete=0 but nothing seens to work. Is there even a possibility to do so?
View 2 Replies
View Related
Mar 30, 2011
I am looking for a way to control the order in which files (or more specifically, certain subtrees) are updated for a very large rsync update done daily. There is a finite time frame and finite bandwidth to do these updates, and they usually do NOT complete. I want to control the order so that more important files are copied earlier.
What I have been doing is breaking up the rsync runs into a few steps where the first step is a very limited set of subtrees using --include and/or --exclude, and each step thereafter is increasing the amount, until the last step is the whole tree to be updated. This is not working because the time it takes to transfer the file list is large (varying from 10 minutes to 40 minutes depending on step). I really need to get this back down to doing a single rsync run to reduce that wasted time.
I cannot ultimately split things because there are a lot of hardlinks that need to be retained (or else the destination space and bandwidth would be exceeded).
I was looking at rsync's batch mode. My thought was if I could get a list of what needs to be updated, I could then sort that list and do just 1 more rsync run to do the updates in the order I want. But it turns out the batch mode file is in some cryptic binary format. I really have no idea if that could have worked.
The order selection is not simply based on the names in the top directory, so I cannot simply just list those in the preferred order on the command line (otherwise, yes, that would control the order). If I do list directory names from various depths, that list gets flattened into the destination directory, which would not work to replicate the whole tree exactly.
View 6 Replies
View Related
Feb 3, 2010
I am using rsync to backup data from a file server to a external device, the data is accessed from windows boxes via samba with their own usernames, user1, user2 etc... Rsync is failing to copy the file permissions when sending to the external device, I have tested sending to a ftp server and a usb hard drive and received the same error, see below:
Code:
root@Fileserver:~# rsync -avz --delete-after /shared/fileshare/ /backup/backup
building file list ... done
./
manager/
rsync: chown "/backup/backup/manager/cronman.txt" failed: Operation not permitted (1)
public/ .....
sent 339 bytes received 104 bytes 886.00 bytes/sec
total size is 4593 speedup is 10.37
rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]
When I backup to a local directory the permissions are copied correctly.
View 4 Replies
View Related
Dec 7, 2009
I am unable to transfer file using rsync. I have installed rsync and ssh package.. When I try to transfer file , its giving an error rsync -v -e ssh a usw3tvx@lalqau40:ssh:lalqau40: Name or service not knownrsync: connection unexpectedly closed (0 bytes received so far) [sender]rsync error: error in rsync protocol data stream (code 12) at io.c(463) [sender=2.6.8]I tried with simple format... nothing works...Even I couldnt ssh with hostname. However I can connect with Ip address... I
View 1 Replies
View Related
Jul 21, 2010
use rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
View 2 Replies
View Related
Sep 25, 2010
I want to copy a big file from my harddrive to a removable drive by rsync. For some other reason, the operation cannot complete in a single run. So I am trying to figure out how to use rsync to resume file copying from where was left last time.
I have tried use the option --partial or --inplace, but together with --progress, I found rsync with --partial or --inplace actually starts from the beginning instead of from what was left last time. Mannually early stopping rsync and checking the size of the received file also confirm what I found.
But with --append, rsync starts from what was left last time. I am confused as I saw on the manpage --partial, --inplace or --append seem to relate to resuming copying from what was left last time. Is someone able to explain their difference? Why --partial or --inplace do not work for resuming copying? Is it true that for resuming copying, rsync has to work with option --append?
Also if a partial file was left by mv or cp not by rsync, will rsync --append correctly resume the file copying?
View 1 Replies
View Related
May 26, 2011
Using Ubuntu Server 10.04 LTS. I'm new to Ubuntu and testing Rsync. I successfully copied 3TB of data from a Win7 machine to an MDADM Raid5 array. All appears to be fine. Used a Win app for the copy. I then deleted a 250GB folder on the Raid5 array and recopied the data using Rsync. Rsync was executed via a Putty session on a WinXP machine. The source was an eSata attached drive (same drive used for the big 3TB copy) and the destination was the same Raid5 array. That copied just fine. I bit verified it with a Win7 app. Perfect.
I then used the following Rsync script to copy a single 26GB file from that same eSata drive back to (what I intended to be) the Raid5 array:
Code:
neil@ANTECUBSV:/mnt$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test
sending incremental file list
created directory /mnt/Test
./ C_VOL-S300-b001.spf
sent 3020267622 bytes received 34 bytes 50760800.94 bytes/sec
total size is 3019898880 speedup is 1.00
neil@ANTECUBSV:/mnt$ cd raid
Note that only about 3GB copied. No error messages were posted to the putty session. I made a mistake in the Rsync command, creating the Test folder directly in the mount folder rather than the Raid array, as I intended. That is a little strange, yes, but I would not think it would cause a partial copy? The /mnt folder is on my system drive, which had about 34GB available space before the copy, so comfortably would have had 6GB or so after.
The eSata disk is mounted as /mnt/disk1
The Raid5 array is mounted as /mnt/raid
I then recopied the file to the correct intended destination on the Raid5 array, which has about 400GB free space (plenty).
Code:
neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/raid/Test
sending incremental file list
./
deleting 2010-07-05 Backyard Birds/Thumbs.db
deleting 2010-07-05 Backyard Birds/
C_VOL-S300-b001.spf
sent 11105775462 bytes received 34 bytes 50366328.78 bytes/sec
total size is 11104419840 speedup is 1.00
neil@ANTECUBSV:/mnt/raid$ df -h
Note that only about 11GB was copied, and this was confirmed with an ls -l command. Now I am correctly copying the file to the Raid array but it is still incomplete. I then copied the file back to the /mnt folder to see if the problem reproduces:
Code:
neil@ANTECUBSV:/mnt/raid$ rsync -r -a -v -e ssh --delete /mnt/disk1/Test/ /mnt/Test
sending incremental file list
created directory /mnt/Test
./
C_VOL-S300-b001.spf
sent 26327927554 bytes received 34 bytes 56558383.65 bytes/sec
total size is 26324713984 speedup is 1.00
neil@ANTECUBSV:/mnt/raid$ cd /mnt/test
This time I got my full 26GB file. Why I might be getting inconsistent results? This is quite troubling of course. I'd also be interested in basic a command line Linux diff app (that does file directory as well as bit level checking) if one is available.
View 9 Replies
View Related
Aug 10, 2009
When using rsync to copy from one NFS mounted filesystem to another mounted NFS, rsync displays "stale NFS file handle" while attempting to issue a chmod after each file is copied. The files appear to be written successfully but the owner and group show up as "99." The source file system is mount USB ext3 drive and the target is a Buffalo TeraStation TS-XL/R5.
View 6 Replies
View Related
