Ubuntu Networking :: When Transferring Large Files Using Scp Between Desktop Running Maverick And Other Servers?
Nov 12, 2010
sometimes when transferring large files using scp between my desktop running maverick and other servers running Ubuntu, Debian or CentOS, I get the following error message: 77% 258MB 11.3MB/s 00:06 ETAReceived disconnect from xxx.xxx.xxx.xxx: 2: Packet corruptI've found a seemingly related bug report on launchpad here: but the provided "ethtool" fix did not help. I'd be most grateful for any ideas on how to solve this issue. Some more info:Linux lotus 2.6.35-22-generic-pae #35-Ubuntu SMP Sat Oct 16 22:16:51 UTC 2010 i686 GNU/Linuxlspci | grep eth -i00:19.0 Ethernet controller: Intel Corporation 82567LM-3 Gigabit Network Connection (rev 02)
Every time I attempt to transfer over a large file (4 GB) via any protocol, my server restarts. On the rare occasion that it doesn't restart, it spits out a few error messages saying "local_softirq_pending 08" and then promptly freezes. Small files transfer fine.
Relevant information:
Ubuntu server 10.10 Four hard drives in RAID 5 configuration CPU/HD temperatures are within normal range
This problem is not exclusive to Ubuntu, I've experienced it in Windows and OSX as well, but it seems that almost every time I transfer a large number of files (i.e. my music collection) between my desktop computer and laptop via my external hard drive, I end up losing files for no reason. I usually don't notice the files are missing until later on, because I am never informed of any data loss. Now, every time I make a large transfer of files, I just do it two or three times to ensure that I don't lose any files.
I'm experiencing connection problem when transferring a large file from Windows 7 (Home Premium) to my Ubuntu 11.04. The transfer starts, but after a couple of seconds the connection drops and all the shares are unavailable. I'm also unable to connect to the server over ssh, and the only thing I can do to restore the connection is to reboot the server. The strange part is that this was never a problem a couple of weeks ago, and I've not done anything to the setup on either machines besides installing security updates.
I want to transfer an arbitrarily large file (say >20GB) between 2 servers. I have several considerations:
Must use port 22 (ssh) because of firewall restrictions Cannot tax the CPU (production server) Memory efficiency Would prefer a checksum check but that could be done manually Time is not of the essence
Server A and Server B are on the same private network (sharing a switch) and data security is not a concern, Server A and Server B are not on the same network and transfer will be via the public internet so data security is a concern, My first thought was using nice on an scp command with a non-CPU-intensive cypher (blowfish?). But I thought I'll refer to the SU community for recommendations.
I have just rewired my LAN using Cat 6 cables. Download- and Upload speeds to my main machine are great :However, it seems my LAN speed is very slow. when I copy large files from my main desktop pc to my mediaplayer, I only reach a speed of 3,7 MB/s in Ubuntu 10.10 using SMB protocol to connect to mediaplayer. I use a Sitecom WL351 Router that works fine as far as I can see. Using my dualboot Windows 7 I can copy the same large files using same setup (same pc to same mediaplayer) I can copy with almost 10 MB/s. I appreciate a little protocol overhead in Ubuntu, but this seems to be a little too much
I've tried FXP and rsync, none worked and I can't find much info on how to get them working, just alot of google results with people in the same situation as me!
I want to transfer files from 1 remote server to another remote server... a fairly easy task in which I'm sure there's an easy solution, I just can't find it.
While the redhat posts indicated selinux, when I checked it with sestatus (after first installing sestatus), it came back disabled. So what is locking down the /var/www directory?
I am looking to set up a home network server on an old PC. I want to be able to access files on the network hard drive in the PC, stream video or music from the "server", and use the computer as a regular desktop connected to TV as a monitor. I tried setting up an FTP server and that worked for transferring files but failed to get a good connection for streaming or fast transfers.
Was less then 1MB a second. I tried setting up a shared folder but ran into several permission errors between the host PC and the laptop that that connected to the shared folder. which way to go with this to get what I want out of it. I know if can be done just not sure how to do it.
I've got a quadcore PC, 2gb ram pc running ubuntu 11.04 with samba set up on a shared folder - the plan is to share files and media between myself and my housemates. However, I do not have the router attached by ethernet, I am connecting to the router (in another room) by a d-link wireless dongle.
I can access the share without real issue from my windows laptop and other computers in the house share I'm part of, and I can happily transfer a text file here and there, but it seems to be anything over 20mb will not go. All the files I am trying to transfer are >1gb and it's driving me mad.
After some decent transfer rates of around 1mb per second it just freezes up. The screen attached to the actual server goes blank and the host becomes unreachable by ping.I'll be happy to provide any command results or anything to get this working. any assistance would be welcome!
I've been using Ubuntu on my fileserver for quite a while now, and I've always really had this problem, but I want to finally address it and get it fixed. At seemingly random points (when my fileserver is under stress - typically while I'm writing lots of data to it), my fileserver will crash. It generally completely crashes, not responding to any further file requests or any of my SSH commands, and must be reset hard (typically by flipping the power switch). After such an occasion, I end up with some corrupted files. It seems to corrupt a large array of files (it's not an isolated issue - for example, it corrupts files that were not being accessed anywhere near the time it crashed, including files that had never been accessed during that period of uptime). The files don't get completely smashed, but they're definitely corrupted (artifacts in images, skips in audio and video files, often complete failure of binary files such as virtual hard drives or disc images).
I'm using Ubuntu Server 11.04, but similar issues to this happened for me in 10.04 LTS (in fact, I upgraded to try to solve them). I'm using mdadm to create an 8-drive raid6 array. The drives are 1.5 TB each, mostly Samsung HD154UI, but with a WD drive in there too (sorry, I can't find the model number at the moment). The hard drives themselves appear to be working fine - SMART reports no issues with any of them, mdadm says they're all up, and I have no reason to believe that the drives are at fault here (although I can conduct further tests if necessary). I've posted about this problem before here and here. In these cases, the issues seemed to be with XFS - in fact, I switched from XFS to ext4 on my RAID array because I simply believed XFS to be unstable. Unfortunately, this issue occurs with ext4 as well, so I'm fairly certain it's an mdadm issue. Here is the output of "cat /proc/mdstat", for those interested:
Desktop, laptop, both static IPs, can ping each other no issue. 10.04 LTS.Am attempting to transfer files using Filezilla (which worked when I tried it about two years ago!) and the overall response is 'No route to host'. I have scoured the interwebs and have found no solution. Pretty sure I'm putting in the correct details.
I tried using ssh between my netbook and desktop, but it was going to take around 30 hours to transfer 39GB over the home network. Also SSH is very sketchy and often drops connections.I've been messing with it all day and I'm quite frustrated.What I'm looking to do is use my netbook as more of a primary computer and the desktop as a storage computer. Not quite a server, because I'd like to still keep a GUI on it. I'd like to be able to keep my music and movies on the desktop and stream them to the netbook (SSH sucks for this, always drops connections). I've already set up the web client for Transmission bit torrent client so I can torrent on a machine that's almost always on and connected.
Is there a better setup for all of this? I like the netbook because of the portability; I like the desktop because it's always connected (for torrents) and it has a larger storage capacity. It would be mainly used around the house. I would like to back up a file or two while abroad, but I'm not looking to stream music while I'm across town or anything.
I have Fedora 12 (with all the latest patches, including the 2.6.31.6-162 kernel) installed on a new Supermicro SYS-5015A-H 1U Server [Intel Atom 330 (1.6GHz) CPU, Intel 945GC NB, Intel ICH7R SB, 2x Realtek RTL8111C-GR Gigabit Ethernet, Onboard GMA950 video]. This all works great until I try to transfer a large file over the network, then the computer hard locks, forcing a power-off reset.
Some info about my setup:
[root@Epsilon ~]# uname -a Linux Epsilon 2.6.31.6-162.fc12.i686.PAE #1 SMP Fri Dec 4 00:43:59 EST 2009 i686 i686 i386 GNU/Linux [root@Epsilon ~]# dmesg | grep r8169 r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[code]....
I'm pretty sure this is an issue with the r8169 driver (what I'm seeing is somewhat reminiscent of the bug reported here). The computer will operate fine for days as a (low volume) web server, and is reasonably stable transferring small files, but as when as I try to transfer a large file (say during a backup to a NAS or a NFS share), the computer will hard lock (no keyboard, mouse, etc.) at some point into the transfer of the file. It doesn't seem to matter how the file is transferred (sftp, rsync to NFS share, etc.).
I am trying to share files from a desktop running Ubuntu 11.04 so that they may be seen from a Laptop running Ubuntu 11.04 which are on the same network.
I have tried installing Samba, which worked fine until i replaced windows on the laptop with Ubuntu. then the Laptop would not recognize any computers on the network. as i tried to search through the forums and solve the problem myself i think i may have also edited my smb.conf file in a way that it will no longer allow me to share the folder ( which used to be shared) on the desktop.
EDIT: if it helps when i right click on the folder, and try to create share i get: 'net usershare' returned error 255: net usershare add: cannot convert name "everyone" to a SID. Invalid parameter.
I have once tried transferring files from Redhat 9 web files to Centos 5.5 /var/www/html/ directory. Ok... LAMP is installed and working fine. httpd / apache is running and mysqld is installed without root password. Now I wan't to transfer it again to secure the database from Centos 5 to fedora 13. But when I tried browsing the web to check the web files I can only see these:
I am a novice in the world of cloud and recently managed to configure Ubuntu 9.04 Cloud (using kvm, eucalyptus and other packages) successfully at my college for my project work. The problem is that i can only manage to view the running instance using rdesktop from any remote machine. Is there any way to do this other than rdesktop/logs? Secondly, I want to develop a application on the lines of google docs as a part of my project. Is it possible to install apache server on this virtual instance, and host a website? How will the client access this website? Which frameworks would be required or do I have to develop one?
I've resolved to move onto Ubuntu, but I haven't done so quite yet. One question that sprang to mind was the transferral of files from the Windows OS to the Linux OS on the same machine, if it's possible. I'm thinking something like the dual partition, though I suppose Copy/paste wouldn't work.
Another possible solution was using dual partition and then Windows SCP, but everything I've read on the topic indicates that SCP works by finding the (different) host address of the computer; thus it would read my machine as one, even if I'm running Ubuntu on it as well.
how to transfer these friggin' things.
Part of the reason for my query is that I am quite strapped financially and am unable to backup my files on an internet service or on discs.
I want to convert my wife's computer from MS XP 2003 to Ubuntu 10.10. I have saved "My Documents" to an external hard drive. How will I go about transferring these files to Ubuntu? Do I just pick individual files and go to "Save As"? Can I just transfer them en mass?
when I try to transfer files between flash drives, it starts to copy normally and then after few seconds it starts to lag. Transfer rate drops with each second by 0.2 MB down until 5 MB/s where it completely hangs. This happens to every single flash drive I put in and this is annoying because I have external HardDrive connected over USB port and I keep all my data in there, so when I try to transfer the files from the hard drive to the flash drive - it lags and hangs. Same thing if I want to copy from internal harddrive to flash drive. Although everything is fine when I copy from the flash drive to the internal harddrive and everything is fine when flash drive or the external harddrive is alone (no other flash drives or something is connected).
I am also using Ubuntu 10.04. I have tried transfering files on my netbook running ubuntu, because I thought that it might be a USB port issue but the same thing happened on the netbook too...
On an Apache2 server someone else setup, I have a folder with drwx--x--x permission and the php file can still write in the folder. But on my own setup, I need to set the same folder to drwx--x-wx. Inside the folder, I have a index.php that runs just by setting rwx--x--x but on my own setup, I need to allow read permission for others/group before it can run: rwxr-xr-x (or else I get a blank page). I tried changing the folder and files to root but it's the same.
The computer and phone are paired, and transfer starts, but after 4096 bytes it says:Transfer finished with an error: Error sending object. I'm running 64-bit Lucid and Maemo 5 on the phone. Here are some of the things I've tried, following the docs, and the errors I get:
Code: ~$hcitool info 3D:F7:2A:60:7A:40 Requesting information ...
I have two different Linux operating systems (on two separate partitions) on my laptop. One is open SUSE and the second is PCLinuxOS and I frequently have to update files and folders from one to the other (the consequences of trying to keep both current!). When I am using openSUSE and want to access folders/files on the other OS, openSUSE asks for SU authorisation to access the other . . ./home file. Until recently, I could open ALL of the folders on the (other) /home file but now I can open some folders but not others. The Dolphin screen shows a warning at the bottom that "Could not enter folder /media...... " ! I can then only access the folder by going into the menu and using the 'File manager -Super User' mode ..AGAIN.
I suspect this action is necessary because I have had to previously open these specific folders in 'Root mode' to be able to 'paste' the moved file! I would like to avoid this (double) 'messy' action - how to change/avoid this My system is a 32bit openSUSE 11.4 with KDE 4.6.0 and the 2nd OS is also KDE 4.6
I am using the diff command with the -r option, to compare a large number of files and files in subdirectories. My main interest is to find out which files have been changed, and not what the actual changes are, and since a lot of files has been changed, it would be a lot easier to view the file names only. Is there and option for diff that might do this, or does there exist a similar tool/command that could do the job?
I downloaded Amarok 2.4 today and am attempting to use it to sync my music to my iPod Touch. .mp3 files transfer perfectly, but for some reason any .m4a file is being reassigned a .mp4 extension and being relegated to the 'Video' app on the iPod. I do realise that an .m4a is just .aac audio in an mp4 wrapper, why Amarok and/or the iPod is deciding to treat them like a video instead? It's doing it both to my own ffmpeg-encouded .m4as as well as unprotected .m4as from the iTunes Store (relics of my time on Windows). Anyone know what might be going on with this? Ubuntu 10.10 NE, Amarok 2.4, iPod Touch 2G (... possibly 3G, but I'm pretty sure it's a 2G) running firmware 4.2.1
I am trying to download a EBCDIC file from z/Os to UNIX using FTP in ASCII mode. The problem I am seeing is; when UNIX FTP Client issues a get; UNIX CPU goes to 100% CPU utilization when transferring a file in ASCII mode. I have done packet traces using wireshark and noted that z/OS Server is translating the file to ASCII before putting it on the wire, however when UNIX FTP Client gets the file I believe that the UNIX FTP Client is again translating the file to ASCII causing the 100% CPU utilization. I need verification that this is what is happening, and is there a way of getting around this other then transferring the file in binary mode then doing a EBCDIC
I have a linux box that I plan to use primarily as a server. I also have another machine that dual boots Windows/Linux.I would like to have a way to backup my filehe Windows/Linux box onto the Linux server. In other words, I am assuming the hardrive on the Windows/Linux box could fail at any time and I want to have a backup of important files.Should I set up an ftp server to do this? Are there any security issues that I need to be concerned about if the files contain sensitive information etc?
I am running a linux server and am familiar with scp somewhat, and have used it successfully in the past. But that was talking to another linux server. Thing is, here I need to talk with a Windows server. How do I do that. In addition, their server may require a VPN, which seems to add another layer of complexity.
I've discovered that Dolphin seems to lose random files when copying many large folders.
I first noticed this a few months ago when I tried to copy my music library from one folder to another on the same HDD. It consisted of around 600 folders and 6500 files. During the copy there were no errors but after the copy I found that some of the newly copied folders were missing files. I put it down to human error or a glitch.
Yesterday I tried to copy 13 folders containing rips of some of my DVDs. Each folder basically had one film of either 700MB or 1.4GB. Again no errors showed up during the copy but I found 3 of the newly copied folders were empty.
It's not so critical with music or films but I can't afford to lose work data like this.
Has anyone experienced or seen a similar problem with Dolphin? I'm going to have to do some more extensive testing but this is not good.
The first time I noticed the problem I was running KDE4.3.4 (I think) and now the latest was with KDE4.4.0.
