Server :: SCSI I/O Error And Unstoppable Apache2 Process?
Apr 30, 2009
I currently have a massive problem with my main webserver: one of the SCSI drives where I keep my htdocs seems to have failed (sd 0:0:0:0: [sda] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK end_request: I/O error, dev sda, sector 7797601).
Since I have no backup (dumb!dumb!dumb!) of it, I tried to copy what I could on the fly, but that failed miserably. The next idea was to umount the volume and try remounting it R-O, but I have a bunch of apache2 processes that are apparently reading/writing on it and I can't seem to find any way of terminating them. I tried via init script, killall and kill -9 but they just don't respond. Until I can terminate those processes, I won't be able to unmount the drive and try saving what's left of the data.Oh and the cherry on top of the cake: the server is 1100 miles away so I can't even access it physically..Is there any way I didn't think of to get rid of those apache processes?
i m facing same error in most of the HCL servers. the problem is that it throws error while booting and sometimes not throws error. the error is :-
Feb 13 13:17:25 fe13s kernel: Adapter 0: Bus A: The SCSI controller was reset due to SCSI BUS noise or an invalid signal. Check cables, termination, termpower, LVDS operation, etc.
Feb 13 13:17:30 fe13s kernel: Adapter 0: Bus B: The SCSI controller successfully recovered from a SCSI BUS issue. The issue may still be present on the BUS. Check cables, termination, termpower, LVDS operation, etc
Feb 13 13:29:15 fe13s kernel: Adapter 0: Bus B: The SCSI controller successfully recovered from a SCSI BUS issue. The issue may still be present on the BUS. Check cables, termination, termpower, LVDS operation, etc code....
I have suse10 64 bit installed. I am setting up a svn server on it. After installation and adding the modules ,while reloading the apache2 it's throwing the error as: HTML Code: httpd2-prefork: Syntax error on line 113 of /etc/apache2/httpd.conf: Syntax error on line 31 of /etc/apache2/sysconfig.d/loadmodule.conf: Cannot load /usr/lib64/apache2/mod_dav_svn.so into server: /usr/lib64/libsvn_subr-1.so.0: undefined symbol: apr_memcache_add_server
Starting web server: apache2[Wed Dec 09 15:36:40 2009] [warn] NameVirtualHost XX.XX.XX.XXX:80 has no VirtualHosts(99)Cannot assign requested address: make_sock: could not bind to address 68.178.232.100:80 no listening sockets available, shutting down Unable to open logs failed!
it seemed like the most accurate place. Also apologies for any inaccurate terminology, I'm a bit new at this. Running Apache2 on top of Debian 5.0
Anyway, I have irssi set to output logs to a folder accessible by my web server. User permissions are all set up, so it writes to the folder just fine, but when I access the server index in a web browser (i.e., page that says "index of /[directory] at the top) I cannot see the the directory or the logs that irssi is making. I can ssh in and see the folders and files in the terminal, so they are being created. How can I set it so these are viewable through the web server? I tried restarting the server, no effect.
I have suse10 64bit and I was setting up SVN server on it. After all required setup while reloading apache2,its giving the error:
Code:
httpd2-prefork: Syntax error on line 113 of /etc/apache2/httpd.conf: Syntax error on line 31 of /etc/apache2/sysconfig.d/loadmodule.conf: Cannot load /usr/lib64/apache2/mod_dav_svn.so into server: /usr/lib64/libsvn_subr-1.so.0: undefined symbol: apr_memcache_add_server
Installation of SUSE 11 server now when I try to configure apache2 (opening port 80) I get a "cannot adjust apache2 service" error??? Can anyone shed some light on this.
For a bit of background info, the end result i'm trying to achieve is for files that get uploaded via a php script to be owned by the user account that owns the script rather than by the apache user.
I have installed a LAMP server (Debian Lenny + Apache2 + PHP Version 5.3.1-0.dotdeb.1). On this machine, my web application (PHP + MySQL) exists in 2 flavours:
- Dev - Prod
In /etc/apache2/sites-available, I have ims & ims-dev, one for each instance. Their contents are identical in terms of settings, except for the paths (dev/prod):
I am in the process of writing a web-application using apache,mod-python and cheetah. I installed apache2, mod-python and cheetah and also enabled the userdir module in apache2 so that i can host the webpages inside my public_html folder.
The public_html folder has a folder named 'site' which gets displayed when I type "url" in the browser. There are subfolders inside this 'site' folder and the site folder also has an index.html page inside it.
But when I click on the site folder in the browser, I get a 'Requested url /~myusername/site/ not found'. There are files inside the folder with 777 permissions and still I get this error.
I use Ubuntu Jaunty with the following configuration - 'Apache/2.2.11 (Ubuntu) PHP/5.2.6-3ubuntu4.1 with Suhosin-Patch mod_python/3.3.1 Python/2.6.2 Server at localhost Port 80'.
These are the lines I have added to the /etc/apache2/sites-available/default file
Code:
<Directory "/home/myusername/public_html/site/"> SetHandler mod_python PythonHandler mod_python.publisher PythonDebug On PythonAutoReload On
Since May 12,2009. Our system lifekeeper has the error log "lifekeeper error: DEVICE FAILURE on SCSI device '/dev/add'", but it ran normally. Until last week, it failover to the standby server. The disk still running, the error still come out.
In my understanding, the way /proc/scsi/scsi gets populated, /proc/paritions also gets populated in the same fashion. i.e. the description for first entry of /proc/scsi/scsi can be seen in the first entry of /proc/partitions and same for rest.
So, With this assumption, in my project, I used to relate first entry of /proc/scsi/scsi with first entry of /proc/partitions to get its total size and same for all entries.
But, I observed some differences in following scenario, where
1) The first 4 entries in /proc/scsi/scsi are SAN luns attached to my system and for which the actual device names in /dev/ are sda,sdb,sdc and sdd.
2) The last 4 entries are the internal HDDs on same system. In /dev/, their respective device names are sde,sdf,sdg & sdh.
(Output attached at end of the thread)
But in /proc/partitions, the device order is different.
You can see their respective sizes in /proc/partition output as well.
So, my question is, in this particular scenario, I can't relate the first entry of /proc/scsi/scsi with first entry of /proc/partition. i.e. scsi0:00:00:00 is not /dev/sde, because it is actually /dev/sda.
It seems that my assumption is wrong in this scenario.
Is there any way or mechanism to figure out actual device name for an entry in /proc/scsi/scsi in /dev/ directory?
How can my application should relate /proc/scsi/scsi entries with their respective device names and sizes?
I have been working on this for a few hours now and still haven't been able to track down whats going on. I just switched servers and put a fresh debian install on with apache2 + custom suexec + php5(fcgi). now when i upload a file to the site it wont upload, so i tracked down the issue. its the /tmp directory that root tmp. its at 100% so i restart the /etc/init.d/apache2 process and the /tmp directory is automagically cleared to 20% used and i can upload again. so i figure ok its got to be a setting. i looked in all the conf files conf.d the php5 conf files and no where do i have a /tmp directory set. so i think OK I can build a cron to clear the tmp directory every 10 mins till i find the issue, but alas, when i go into the tmp directory There are NO files to delete!!! wtf. holy crap. and all that goodness, this is crazy, how can you have a tmp directory that is not being written to, from all i can see, so i think ok it might be a hard link, i checked, in all root and sub root directories no hard links to tmp, no soft links to temp either. and if i sit in my /tmp directory and do a df -h after some uploads its going from 20% so 23% and up, but i do ls -alh and nothing, then ls -al and watch bytes, nothing , so this is very strange. has anyone encountered something like this? is it possible that this is a hard link thats hidden somewhere or some other extreme function i have never heard of?
When I enter "cat /proc/scsi/scsi" I'm returned with "cat: /proc/scsi/scsi: No such file or directory". I've tried this on two different installs on two different machines.
Some of our workstations have LTO's attached and they seem to drop off every now and again, the only thing which picks them up again (besides a reboot) is the famous rescan-scsi-bus script from here
The thing is that I'd like non-root users to be able to run this script, which in turn needs root to /proc/scsi/scsi
Is set up a postfix, dovecot + clamav server as described in the wiki. I also use the freshclam update script in /etc/cron.daily/freshclam. But my problem is i get this error message every day:
/etc/cron.daily/freshclam:
ERROR: Problem with internal logger (UpdateLogFile = /var/log/clamav/freshclam.log). ERROR: /var/log/clamav/freshclam.log is locked by another process
I've lost 6 CD-R's to this error. I have no idea what the hell could be wrong. I do not have a SCSI interface in this unit, but no matter, when ever I try to burn a cd it starts and then spits out this error; "SCSI error on write(16,16): [5 21 02] Invalid address" The log from the first attempt to burn that fails. Just before this I had burned a CD and the only problem was it couldn't eject it at the end.
I am trying a release of Ubuntu has have run into a problem that does not make much sense. I am receiving the following error: Syntax error on line 20 of /etc/apache2/sites-enabled/portal: Invalid command 'ProxyHTMLEnable', perhaps misspelled or defined by a module not included in the server configuration fail! I believe the modules required for this is apt-get install libapache2-mod-proxy-html and it shows under the available modules and it was enabled with a2enmod proxy-html.
The line from the config is ProxyHTMLEnable On. The release is the latest download with updates from, 32bit.
I was stupidly trying to setup apache for virtual hosts using two different tutorials and something got mucked up along the way. Apache will now not start and I just need a some help fixing it. My error logs show nothing and running "sudo apache2ctl configtest" said "syntax ok".
I think what i need to do is update the certifcate for the apache2, but I'm not sure how to do this, where to put it, and then which of the thousand apache config lines needs to be changed
i have a linux machine installed with RedHat 4.7 x64 bit. toady our machine just hanged and the only thing we can do was to do an hard reboot.i saw that when she hanges all her disk looks offline. in /var/log/messages i only saw this error message:
we have 6 disks in RAID 5 in this machine- SAS disks. i've search in the net and looks like the emulex driver told the SCSI to do a reset for some reason. how can i check this issue? machine is online now and everything looks ok but i'm afraid i have a problem perhaps with my disks and this is a production machine so it cant happened again.
Need help on how the scsi and multipathing works in Linux. From the docs i have read, i understand that by the use of multipathing we can assign multiple paths to a SAN partition. If there is a problem then one path will failover to other path.However, i am not clear on how linux recognizes the SAN partitions using the multipath drivers. For e.g. I have a HP Proliant server on which we have the following mounts:
Filesystem Size Used Avail Use% Mounted on /dev/cciss/c0d0p3 59G 11G 46G 20% / /dev/cciss/c0d0p1 494M 27M 443M 6% /boot
I created a new disk on our scsi san. I then ran the following command: echo "- - -" > /sys/class/scsi_host/host1/scan. did that command for each host. and in dmesg, it shows it found a device /dev/sdg. but when i do a fdisk -l. It never lists /dev/sdg. I just did this other day on another server and it worked fine like that. This is RH 4.8.
Client has a server running 5.5 (I think) and it they moved locations. This server is used in other locations (state) via ssh tunnel as well so they can all access files.
When attempting to boot up I get screenshot 1 (superblock errors)[url]
They supposedly have a backup on a hot swap scsi and I want to know how/if I can restore it using that backup if I cant fix the superblock boot/error issue.
I am setting up a new server and am in the midst of testing RAID. This is an Ubuntu 9.10 server. RAID1 (/dev/md1) is spread across 12 one-terabyte SCSI disks (/dev/sdi through /dev/sdt). It has four spares configured, each of which are also one-terabyte SCSI drives (/dev/sdu through /dev/sdx). I have been following the instructions on the Linux RAID Wiki ([URL]....
I have already tested the RAID successfully by using mdadm to set a drive faulty. Automatic failover to spare and reconstruction worked like a champ. I am now testing "Force fail by hardware". Specifically, I am following the advice, "Take the system down, unplug the disk, and boot it up again." Well, I did that, and the RAID fails to start. It outright refuses to start. It doesn't seem to notice that a drive is missing. Notably, all the drive letters shift up to fill in the space left by removing a drive. The test I did was to:
[code]....
Is removing a disk from the bus a reasonable test in the first place? Meaning, is this likely to happen in a production environment by other means than a human coming by and yanking out the drive? Meaning, is there a hardware failure that would replicate this event? Because, if so, then I don't know how to recover from it.
I am running fedora 8 on a Dell 2950 with a qlogic fibre channel card.is attached to a Dell/clarion AX100.I had a drive assigned to this server which had been working for approximately a year now.Recently during a power failure things turned ugly. Rebooting didn't fix things.I unassigned/reassigned the drive, rebooted, still no good.Funny thing when I unassigned the drive and reboot the system.still see /dev/sdb but no partitions and fdisk -l does not display /dev/sdb.I even did "echo 1 > /sys/block/sdb/device/delete" and the device no longer showed in /dev, but after a reboot it is back.I pulled and reseated the fibre card, same
