I have to transfer large amounts of data (>80 GB) over ssh using rsync. Everything is working fine, but the DSL connection where the backup data is sent from will drop once every 24h for up to 3 minutes (switching providers is not an option).
How do I:
Automatically restart the transfer when the connection is back up? Make sure there are not by accident two rsync commands running at the same time?
I want to copy a big file from my harddrive to a removable drive by rsync. For some other reason, the operation cannot complete in a single run. So I am trying to figure out how to use rsync to resume file copying from where was left last time.
I have tried use the option --partial or --inplace, but together with --progress, I found rsync with --partial or --inplace actually starts from the beginning instead of from what was left last time. Mannually early stopping rsync and checking the size of the received file also confirm what I found.
But with --append, rsync starts from what was left last time. I am confused as I saw on the manpage --partial, --inplace or --append seem to relate to resuming copying from what was left last time. Is someone able to explain their difference? Why --partial or --inplace do not work for resuming copying? Is it true that for resuming copying, rsync has to work with option --append?
Also if a partial file was left by mv or cp not by rsync, will rsync --append correctly resume the file copying?
My ThinkPad X32 doesn't shut off the backlight when I suspend it. It also doesn't resume properly. Shutdown works normally. I'm using OpenSuse 11.3 Gnome. Had the same issue with an install of Linux Mint Debian Edition (201101). It is a well-known Debian issue that has evidently trickled down through Ubuntu to Mint. It also exists in Fedora. In search of a distro that would handle suspend/resume without problems, I first tried Scientific Linux. Suspend/resume worked flawlessly, but I was not happy with the package manager in SL, which leaves it to the user to resolve dependency issues that are automatically handled by Synaptic in Mint/Ubuntu/Debian and by the Package Manager in OpenSuse.
OpenSuse 11.3 Gnome had the same problem - the backlight remained on upon suspend, and resume didn't work at all. I solved this issue by making a simple change in a start-up file in OpenSuse. I'm hoping the solution will work in Mint as well, as it is my favorite distro which I have used for three years. The solution:
1. With OpenSuse: open a terminal and type gnomesu gedit /etc/grub/menu.lst. When prompted, type your root password and click OK.
2. Gedit opens with the menu.lst file displayed. Down the page a bit, you'll see several sections that begin with "###Don't change this comment"
3. The first section gives details for the default boot operating system. Somewhere in this long spec, you'll find "vga=0x314" or vga=(something different for your system).
4. Just before or after this part, type "nomodeset" separated from the vga spec by a space.
5. Save the file and reboot. In OpenSuse, suspend/resume now works normally.
someone could let future readers know where the menu.lst file is located under Linux Mint. In the past, I've found it frustrating to try to edit the menu spec with Mint 10 versions, which use Grub2.
I started an upgrade last night (from Jaunty to Karmic) using a "do-release-upgrade" command on a remote server via ssh. However, this morning, I discovered that the ssh connection between my machine and my server was broken.
I can see that the do-release-upgrade process is still running (well, a giant /usr/bin/dpkg is in the process list) and waiting for a response.
Is there any way I can take control of that session from my current session?
I'm using Scientific Linux on a laptop, connecting to a debian server on my LAN via a basic BT HomeHub router. When starting a file transfer with ftp or rsync from the laptop my connection is dropped, every time, after a couple of seconds. I have to reset my network adaptor and restart my network manager (wicd in this case) in order to get network connectivity back. I had a very similar problem before - [URL]. On this occasion my workaround was to replace NetworkManager with yast. I am now using SL though so no yast available, and I am not using NetworkManager any more anyway.
View 3 Replies View RelatedSo, when I booted up, the wired connection was automatically detected and in use. Whenever I suspend my computer and then resume it later I lose the wired connection. It is detected (seen) by Networkmanager, but I can never connect. I found a way to restore the connection, but it is manual and I was wondering if I could automate this process every time I resume. Details follow:
Laptop: HP dm1z, Natty 64bit. ethernet card:
Code:
#lspci|grep Ethernet
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller (rev 02)
Likely culprit (according to dmesg)
[Code].....
The rsync module "opensuse-full" which worked well so far seems currently broken. It tries to mirror a huge number of additional stuff (factory?) but fails with "permission denied". Any place where I could report this?Command:
Code:
rsync -rlpt -hi -stats rsync.opensuse.org::opensuse-full /drive/repo
Errors:
[code]....
This is a recent problem, and I can't pinpoint any change/upgrade that would cause this. Rsync transfer from Client to Server:
sent 11756196 bytes received 1032741 bytes 138258.78 bytes/sec
total size is 144333466390 speedup is 11285.81
Pinging back and forth from each machine is fine. No Ifconfig errors Client, but Server has RX packet errors.
eth0
Link encap:Ethernet HWaddr 00:11:25:37:ee:44
inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::211:25ff:fe37:ee44/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:41786 errors:2157 dropped:0 overruns:0 frame:2157
TX packets:34138 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:55615449 (55.6 MB) TX bytes:4737538 (4.7 MB)
What's the next step?
Dell Precision Workstation running Suse 11.1 & KDE 3.5 as default O/S, with Suse 11.3 KDE4, Ubuntu 10.04 & Win XP Pro as secondary O/S's.Medion PC with Win XP Pro for those windows apps I have to use (against better judgement but because my clients require it!)
On Friday 6th I was having a play with Apache2 running on the Dell Precision on Suse 11.1. I have had a local development webserver for some time to test websites before uploading them a host server. I had wanted to test the DHCP & DNS functions for some time, and this seemed like a good opportunity. But I ran into some problems and decided to un-install the DHCP & DNS. I intended to install a VM and re-install the Apache server on it, to make it easier to troubleshoot.Unfortunately my internet connection on that box is now broken, but only with the Suse 11.1 install. The other 3 O/S's all work fine, and the rest of my machines are all good. In fact the Suse 11.1 install can communicate across the LAN, and print to both the network printers, it just won't connect to the Internet. Turning off the firewall doesn't do anything, and reconfiguring the NIC in YaST doesn't help.
I suspect that there is a config file somewhere with faulty info, but I can't find it. If searched everything that was changed on that date.I have been searching this forum, LinuxQuestions forums, and Googling for 2 days with no real success. I found some interesting threads which I thought would help but so far no luck. I can ping all the interfaces on my LAN, but anything outside is blocked. I thought it might be in the BT HomeHub but that works fine and all the boxes can connect except with the Precision running 11.1.
I just installed 10.10 on a desktop machine, dual boot with Win XP. Everything went well, it connected and did all the system updates etc. Then one time I rebooted and it wouldn't connect. I've tried rebooting again several times, tried unplugging the modem/router from the power and from the computer then reconnecting everything and starting it up again, no joy. I booted from the Live CD and it connects without any problem, likewise from Win XP. My Network Connections shows 'Auto eth0' and below is the ifconfig output.
Code:
jim@ubuntu:~$ ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:13:8f:b8:ad:fc
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:42 Base address:0x6000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:240 (240.0 B) TX bytes:240 (240.0 B)
I am attempting to write a server application in C on a linux machine which listens for TCP connections and transfers data. I am trying to detecton the server side when the connection is broken. The closest thing that I got to work was looking at the return value from sending data. For example the server�s job is to mainly read data from the socket but to test if the connection is still up the server sends data periodically back to the client. I look at the return value from send() to determine if the connection is broken e.g.
Code:int ret = send(session->clientSocket, &data[sentCnt], count - sentCnt, MSG_NOSIGNAL)I found that this does not immediately return an error when the connection is broken. The reason for this is because even though the connection is broken send() is still successful because it is able to put it on the network buffer. To fix the issue I did the following things;
Code:
//set send timeout
struct timeval timeout;
[code]...
I logged on to one of my servers and ran a yum update, while the update was in progress my SSH connection got disconnected.
I logged in through a new connection when connectivity was restored and looked at running processes. It looked like yum was hung so I killed it (big mistake and one I won't repeat).
Now, when I try and run yum, I get the following:
Quote:
yum update: There was a problem importing one of the Python modules required to run yum. The error leading to this problem was:
Please install a package which provides this module, or verify that the module is installed correctly.
It's possible that the above module doesn't match the current version of Python, which is:
If you cannot solve this problem yourself, please go to the yum faq at:[url]
I have tried hunting around for the correct version of libelf without much luck.
Is there anything I can do to get yum working again?
The server is running Centos 5.2
I am writing a code in Python where a socket client changes data with the server. That works nice if the connection is up. However it is also supposed to work offline. So I need to be able to detect if the connection is up before sending data, but I was not able to do so. A summary of the code is like that:
Code:
try:
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.connect( (self.host, self.port) )
connected = True
except:
connected = False
if connected:
data = "some data byte"
try:
socket.send(data)
except:
connected=False
socket.close()
If I unplug the cable after the connection is ready then the socket sends data and does not detect the connection failure. What is amazing is that the client detects the connection failure just after the cable has been plugged again. Of course there is a loop in the code above and always that connected==False a new connection is made. How could the program detect the connection failure before sending data and then lead the code to an exception?
I have three machines networked to my desktop which run a bunch of simulations in parallel. As they're running, I connect to them via SSH and screen to keep an eye on the runs and look at the output. They stay usually connected for days at a time. The SSH servers and client are running Fedora 14. Yesterday one of my coworkers accidentally yanked the plug on one of the servers while it was running. When I powered it up again, I started getting some odd connection problems. I couldn't connect to it via SSH initially because I got the Remote Host Identification Changed (RSA host key changed) error. I deleted the key in .ssh/known_hosts, which allowed me to connect, but it denied my password. I then logged into that machine locally, restarted sshd, and removed .ssh/known_hosts again. Now I can log in via SSH without problems. However, the connection dies with a "Write failed: Broken pipe" error every few minutes (as opposed to the other two machines, which stay connected indefinitely).
So my questions are:
1.) why would a power loss affect the behavior of the SSH server?
2.) why do I keep getting broken pipes now?
Thought I'd post it here because it's more server related than desktop... I have a script that does:
[Code]....
This is used to sync my local development snapshot with the live web server. There has to be a more compact way of doing this? Can I combine some of the rsyncs? Can I make the rsync set or keep the user and group affiliations? Can I exclude .* yet include .htaccess?
When I run rsync --recursive --times --perms --links --delete --exclude-from='Documents/exclude.txt' ./ /media/myusb/
where Documents/exclude.txt is
- /Downloads/
- /Desktop/books/
the files in those directories are still copied onto my USB.
And...
I used fetchmail to download all my gmail emails. When I run rsync -ar --exclude-from='/home/xtheunknown0/Documents/exclude.txt' ./ /media/myusb/ I get the first image at url.
I have a tiny shell script to rsync files between two servers and remove the source files.
This script works fine, when it has been initiated manually or even when the rsync command is executed on the command line.
But the same script doesn't work, when I try to automate it through crontab.
I am using 'abc' user to execute this rsync, instead of root, as root login to servers are restricted in all of our servers, by us.
As I mentioned earlier, manual execution works like charm!
When this rsync.sh is initiated through crontab, it runs the first command(chown abc.abc ...) perfectly without any issues. But the second line is not at all executed, and there is no log entry i can find at /mnt/xyz/folder/rsync.log.
I just tried to sync files from one server to another. After the sync process, I found the files are bigger than original ones.
I looked up the web and found someone mentions the rsync daemon. So I have to run the daemon on one server before I run the rsync?
The command I used is rsync --partial --progress -r source destination
use rsync to cp such files and dirs under /var/www/html/mydir directory but these two files(/dir4/1.html /dir4/2.html) cant rsync to dest mechine.
rsync configure file,below...
I'm using Ubuntu 10.04 LTS server and Postgresql 8.4. I have a .sh script that is run by cron every other hour. That works fine. The .sh script includes an rsync command that copies a postgresql dump .tar file to a remote archive location via ssh. That fails when run by cron; I think because it is (quietly) asking for the remote user's password (and not getting it). I set up the public/private ssh key arrangement. The script succeeds when run manually as the same user that the cron job uses, and does not ask for the password. I am able to ssh to the remote server from the source server (using the same username) and not get the password prompt (both directions), so why doesn't rsync work? I even put a .pgpass file in the root of that user's directory with that user's password, and the user/password are identical on both servers.
I think the problem is rsync is not able to use the ssh key correctly. I tried adding this to my script but it didn't help.
Code:
Here is the rsync command embedding in the .sh script.
Code:
Here is the cron entry:
Code:
I am happily using SSH's X11-forwarding to run some applications and then the connection was disconnected. Running ps ax from a new SSH connection shows that the applications are still running. How do I "resume" or recover the GUI for the applications to prevent data lost?
View 1 Replies View RelatedCan a Ubuntu (or any other distro) wake up from standby/hibernate at a preset time?
I actually don't want the PC to consume 200W all the time for just doing something every hour.
How can I resume a stopped job in Linux? I was using emacs and accidentally hit ctrl-z which blasted me back to the console. I can see it when I type 'jobs'
[*****]$ jobs
[1]+ Stopped
emacs test_queue.cpp
When I left work yesterday I left vim open with the files I was working on. I'm working from home today, so I VPN'ed in and when I go to edit those same files, I get the warning that there's another current vim session open with those files. I know I could just force logout my other user, but that would kill the existing vim session. I think my changes would all be saved in the .swp file. Instead, is it possible for me to take that existing vim session and open it with my second login? That way it would for sure preserve all my changes, and I wouldn't have to open all the same tabs again.
View 5 Replies View Related(yes a,vim a,copying data etc) via any terminal .please dont give me kill fg,bg at some instances kill -STOP pid,kill -CONT pid .
View 2 Replies View RelatedI was trying to get this feature: wake up my htpc from s3 with my remote control and the solution is to modify /proc/acpi/wakeup and a descriptor in /sys. Here are the details: I'm using a Microsoft IR receiver for MCE remote that appears as dev 2 of bus 2 in lsusb
root@htpc:~# lsusb
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 002: ID 0471:0815 Philips (or NXP) eHome Infrared Receiver
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 006: ID 045e:0714 Microsoft Corp.
Bus 001 Device 005: ID 045e:0715 Microsoft Corp.
Bus 001 Device 004: ID 045e:0707 Microsoft Corp. Wireless Laser Mouse 8000
Bus 001 Device 003: ID 045e:070c Microsoft Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub....
That's USB0 I have to enabled, why not usb 1 or 2 (bus 2 in lsusb)? Moreover why are all the disabled/enabled preceded with a star and S4 and not S3 mentioned? Nevertheless that wasn't enough to get it work. I looked in gconf-editor in apps/gnome-power-manager/general but I have no can-suspend or something similar... (I'm running on 10.10, with 10.04 I could suspend only once, afterwards the computer didn't go to suspend, just black screen then login screen). So I looked in /sys/ and found that 'cat /sys/bus/usb/devices/2-1/power/wakeup' (notice the 2.1 as bus 2 device 2 (0,1,...) gave 'disabled' so a echo enabled > /sys/bus/usb/devices/2-1/power/wakeup and now I can wake-up with the remote when I want.
What I don't understand:
Why USB0 in /proc/acpi/wakeup ?
Why have to change in /proc and /sys ?
Is it possible to automate this to get it work even if I change the usb port the receiver is plugged in ?
After suspending my laptop, and then resuming the screen has black and white jagged wide zig zags across the screen. I can move the mouse and login fine, it's just the graphics are all screwed up. Has anyone had this problem before? Is this X? Is it a driver issue, my info is in my sig.
View 4 Replies View Relatedwhich cropped up today after no changes to the system: I can successfully ssh into my ubuntu 64-bit 9.10 machine (via OS X with iTerm), but after about 30 seconds, the connection drops with the message "Connection reset by peer Connection to xx.xx.xx.xx closed".
If I then try to immediately ssh back in I get: "Write failed: Broken pipe". If I try to immediately ping the machine instead, I cannot ping it until the ~10th try. Eventually I can log back in again, only to be kicked off after between 5 and 60 seconds. I called the network people and their are no known issue right now with networking. The networking on the ubuntu machine is otherwise fine, it seems. In the /var/log/auth.log file, it simple says:
Quote:
Feb 9 14:19:20 harriet sshd[4134]: Accepted password for xxx from xx.xx.xx.xx port 55105 ssh2
Feb 9 14:19:20 harriet sshd[4134]: pam_unix(sshd:session): session opened for user xxx by (uid=0)
Feb 9 14:19:28 harriet sshd[4046]: pam_unix(sshd:session): session closed for user xxx
The solution I come across to a similar problem is to add a "ClientAwakeInterval" number to sshdconfig, but this (as expected) doesn't fix my problem.
I have printer in CUPS that due driver problems (hp 1010) form time to time goes into pause. I would like to write a shell script that will be once per hour resuming printer in cups.But I have no idea after googling for couple of minutes how to resume printer from shell command line.
View 5 Replies View RelatedMaybe some torrent client for Linux can understand the metadata generated by a Windows client. Or maybe there is a Web/Desktop client that works in both systems. Is there any way to do that?I use uTorrent for Windows, and I haven't used any torrent client on my Ubuntu 11.04 yet. But if the solution uses other client for Windows, it will work for me.
View 2 Replies View Related
