Debian :: Logs & Apache Load - Takes Up So Much CPU For Some Of The Requests
Mar 30, 2010
I noticed i have quite a few logs that end with .[number] for example "syslog.1" "mail.info.1" etc, why is this and why are they there since almost nothing is logged in them ??
Question 2: on my server im running a script like imagebam and imageshack with hosts images so i have quite a few apache requests to my server. I am wonder why apache takes up so much CPU for some of the requests? in Htop some requests take up 1.2% CPU while other take up 3-5% etc, so the total load is about 1.50 0.58 0.84 to 2.61 1.08 1.14 with about 128-150 apache requests all the time while sometimes the CPU load can be almost 0 with the same ammount of requests. is this normal? what could cause this in apache ?? the server is just running apache2. MYSQL is running on another server.
Question 1: Hey. I noticed i have quite a few logs that end with .[number] for example "syslog.1" "mail.info.1" etc, why is this and why are they there since almost nothing is logged in them ??
Question 2: on my server im running a script like imagebam and imageshack with hosts images so i have quite a few apache requests to my server. I am wonder why apache takes up so much CPU for some of the requests ? in Htop some requests take up 1.2% CPU while other take up 3-5% etc, so the total load is about 1.50 0.58 0.84 to 2.61 1.08 1.14 with about 128-150 apache requests all the time while sometimes the CPU load can be almost 0 with the same ammount of requests. is this normal? what could cause this in apache ?? the server is just running apache2. MYSQL is running on another server.
my server into the public internet and must have my webserver [2.2.9 (Debian)] act secure. But this does not look very easy [I am searching, reading and working on it already the whole day ]. I read the apache docs, but there is a lot of stuff, which is different in Debian [lenny, 5.0.6]. Apache ignores the host-header given by the browser: [URL] are all served, but should be blocked. I new new to apachy, but on my IIS this works as expected. All browsers act equal [so no browser header problem].I configured two VirtualHosts, an excerpt:
In a squeeze box, I installed awstats and it's working like a charm. Its cron job update the awstats database every 10 minutes (as it runs as root). But I would like to be able to update the statistics from the browser as well. So I setup everything as required and I gave "read" access to "others" to every apache log file. Now, a couple of questions came to my mind:
1. Am I compromising server's security giving "read" access to "others" to apache log files?
2. Instead of giving "read" access to "others", I could add www-data user to adm group (as apache log files are owned by root:adm and permissions are rw-r----). Is this more secure than giving "read" access to "others"? 3. If the option would be giving "read" access to "others" at the end, a log file would be owned by root:adm and its permissions be rw-r--r--. As apache rotates its log files, when Apache create a new log file, does it preserve the permissions (rw-r--r--) or create it with the default permissions (rw-r-----)?
I've been scouring the message boards and trying different things for two days and haven't found a solution. I set up Ubuntu 10.04.1 a few days ago using the server iso and selected the LAMP installation option. I will be using it as a sandbox to try out things from a PHP/MySQL book I purchased.
I have installed Gnome since sometimes I just can't get the command prompt to let me do things and Gnome may at least tell me why. I also installed PHPMyadmin.
My problem is that I can work with this thing all I want in any way I want - HTTP, SSH, SFTP, etc. - from within my home network but Apache refuses to respond to an outside request. Since it's only a test box it usually wouldn't be that big of a deal but I am going out of town for a couple of days and would really like to start working with my new book.
I am 99% sure my ISP is not blocking any ports. I have a ComTrend ADSL modem with router. I have given my Apache server a static IP on the NAT (xxx.xxx.x.101) and set up a dyndns address so I don't have to keep remembering the IP address.
Here is what I have tried so far: Setup a virtual server (port forwarding) on the router to to direct traffic on port 80 to internal IP .101The router told me its interface is using 80 so it would move itself to 8080 Made sure to also add port trigger for port 80 Made sure to save/reset the router Used my iPhone to connect via 3G - didn't work using IP or dyndns name Used iPhone to connect to 8080 and router responded
Set port forwarding/triggering for 8080 -> 80. No joy there, either Tried changing listening port to 8000Set port forwarding and triggering to allow port 8000 Changed ports.conf to read NameVirtualHost *:8000 and Listen 8000 Changed first line of /etc/apache2/sites-enabled/000-default to <VirtualHost *:8000> Restarted Apache service
Apache responded to dyndns.com:8000 from home machine No response when trying same on iPhone (sorry, it's the easiest way for me to test from outside my home network)
At one point, I added "ServerName localhost" to the otherwise empty httpd.conf file but that didn't seem to do a darn thing.
Many of the posts I have perused are at least a couple of years old and have included information on taking action with files that are not there or are no longer where they were. I have interpolated where I can but so far nothing has worked.
I am running Centos 5.3. I ran no updates, performed no installs, nor changed any configuration immediately prior to this issue. My problem is this: when I run the command startx (default runlevel 3), it is a long time (5-10 minutes) before Gnome startx, and once it does start applications will not run. Also, when I try to use sudo (from any environment, even ssh), it is a long time (5-10) before the command is executed.
I cannot say for sure, but it seems like this is an intermittent problem. Sometimes X takes a long time to start, but once it starts it will launch programs. Sometimes X takes a long time to launch, but once it starts it will only launch certain programs. Though presently X always takes a long time to start, and I cannot successfully launch any programs.
A while back a had a similar problem to this (x taking long time to start, sudo taking long time to execute) and it ended up being a DNS problem. Unfortunately, I cannot remember exactly what it was and I stupidly did not document it. Maybe this is also DNS related, I don't know.
I don't know what log files to look at for problems with X, Gnome, and sudo taking a long time to start.
I'm trying to figure out how I can get a request count per CIDR/24 from an apache log in combined format - e.g.: Code: 220.127.116.11 - frank [10/Oct/2011:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "[URL] "Mozilla/4.08 [en] (Win98; I ;Nav)"
I'm stuck using BASH for this and I generally write everything in Python, or even Perl if necessary so BASH isn't the most comfortable for me. I've got enough to extract the IPs and get a count I just need a slick way to come up with a sum on a per CIDR basis.
I'm running wordpress and have gone to the painstaking effort of setting up the ftp server on my 5.3 machine so it can do its own updates and download plugins. However, I've found that if I try to download anything, I get a "unable to resolve host..." error from the script. If I watch Iptraf while making the request, I see all the port 53 requests going to 127.0.0.1. Pings are too.
However, if I ssh in, I can ping and wget and whatever I want all day long and all the DNS requests to go the router (Clarkconnect 3.2 gateway machine). The web browser in the terminal works fine too. What is it about the php scripts that is causing them to misdirect pings and dns requests? I've never seen anything like it and can't find anything on the web about it either.
I'm working on a thorny mod_rewrite problem. I have a mac connected to my LAN running MAMP (Mac/Apache/MySQL/PHP). I request a non-existent file:
Code: http://192.168.1.2:8888/careers/db/1.html I see this in the mod_rewrite log file:
Code: 192.168.1.102 - - [14/Nov/2009:13:46:07 --0800] [192.168.1.2/sid#807df8][rid#8ec850/initial] (2) init rewrite engine with requested uri /careers/db/1.html 192.168.1.102 - - [14/Nov/2009:13:46:07 --0800] [192.168.1.2/sid#807df8][rid#8ec850/initial] (1) pass through /careers/db/1.html Note that the requested uri is /careers/db/1.html
If I change just the file extension on my request to PHP like so:
Then the request uri is totally different now. Here's the rewrite log:
Code: 192.168.1.102 - - [14/Nov/2009:13:47:23 --0800] [192.168.1.2/sid#807df8][rid#8fc850/initial] (2) init rewrite engine with requested uri /Applications/MAMP/htdocs/careers/ 192.168.1.102 - - [14/Nov/2009:13:47:23 --0800] [192.168.1.2/sid#807df8][rid#8fc850/initial] (1) pass through /Applications/MAMP/htdocs/careers/ Note that the requested uri now has a full path which does not include the actual filename, /Applications/MAMP/htdocs/careers/
What the heck? More info. If I request [URL], I can actually access p1.php. The requested uri is /careers/db/p1.php. The problem appears to be because the filename starts with a number. I can also request [URL] and get thru to 1.php with requested uri /careers/db//1.php. Does mod_rewrite think /1 refers to a backreference or something? Why can apache handle the html file request properly and not the php file request?
On my server I provide OCR file conversion service but the problem is when a user uploads a file and it's being converted then if you open another tab and try to load the site it won't respond until that conversion is completed. In other words until the PHP script finishes execution apace doesn't serve any other request to the same browser.
Here is my apache configuration:
ServerTokens OS ServerRoot "/etc/httpd" PidFile run/httpd.pid
You can check what I mean if you try to upload and convert a file and while the file is converting try opening the site in another tab.
I've had a VPS running Ubuntu 9.10 x64 server, hosting 3 websites of mine for a few months now. This problem has been happening for a while. Every once in a while, probably every 2 or 3 days, I'll wake up in the morning, and apache won't be responding, no web pages will load. /etc/init.d/apache2 status, reports that apache is functioning properly. Every time I simply have to restart the daemon and things run fine for another few days.
I thought maybe it was a memory issue, so I lowered the MaxClients in the prefork module from 50 to 30 a few days ago, but the same thing is still happening. My VPS has 512MB of ram, burstable to 1GB, and according to Virtuozzo, there was only one night of high traffic where I even came close to that soft limit. I've checked my syslog, and there's absolutely nothing in there about apache. I've checked apache's error.log as well, and there's nothing in there that would indicate a problem either.
A Linux (CentOS5.3) server is setup with apache reverse proxy. The reverse proxy server is opened to outside and an internal server is mapped to ProxyPass configuration. SSL certificate is also installed on the Apache reverse proxy server. The problem is, it is extremely slow in serving http requests through reverse proxy. There is no problem with server resources or bandwidth. When the internal server is directly accessed through Internet, there is no delay. The backend server and the reverse proxy server are also on the same switch (same subnet). When I searched the Net, there were recommendations to enable cache in Apache. I did so as follows in httpd.conf.
Title describes the problem rather well. After selecting Linux 2.6.23-22-generic 64-bit in the GRUB bootloader, I am staring at a black screen with a white flashing underscore for almost 2 minutes (usually between 1min40sec and 2min). The Linux kernel is loading here, and it's taking forever!
Once Plymouth starts it barely even manages to flash into view before it's done and I get to login, so bravo @ Canonical.
But seriously, I need that boot time cut down A LOT.
Since I'm assuming it has quite some relevance, here's all my installed programs: (So if you know any offenders you can point them out)
EDIT: I can see it also lists packages I've removed after installing them. Removed packages are listed as "deinstall" and are, obviously, NOT installed or functioning, but their config files remain I assume.
I use fedora 13, recently updated. I used to have an issue with wine fonts but it got solved once i upgraded. What isn't solved with the upgrade is the awful long time it takes wine to load anything. any suggestions? It's hard to find a solution on the internet for this issue cause if i search wine takes to long to load it returns how long does wine have to stay in a cellar.
I did used the Alien-Arena yesterday as usual and did work fine from start to finish! When I launch it today, it is taking about 4 minutes to load the main screen. I did a reinstall and with no success! Completely removed it with synaptic and reinstall... but the problem still remains! Something went wrong and I can't figure out what
I'm running Ubuntu 9.10 and Windows XP on my system. After installing Ubuntu, it boots normally about once, and then the next time I try to boot, I have to wait a few minutes with it saying "GRUB Loading." I have read other threads about this that say that it is clearly a GRUB2 problem, and something about Windows overwriting something with the MBR; although I haven't booted into windows once in the process of installing Ubuntu (multiple times) or afterward. GRUB version 1.97~beta4.
Does anyone know of any software that can monitor the Apache logs for certain phrases or keywords then send an alert when found? For example I know an attempt to hack has been made when I see log entries like this....
/admin/ /admin/phpadmin/ /phpadmin/
But by the time I see it, the attempt has long since failed or succeeded. What I need is a way for my server to alert me WHILE someone is entering these phrases. I realize there may be a "hit" to performance but my server is not that busy anyway (except for hackers).
On Lucid, using the nVidia proprietary drivers, the panel takes about 15-20 seconds to load after loging in. Everything else works fine, I can use my desktop shortcuts and open other programs, just the panel leaves an empty space at the top and bottom, until it loads.
I don't have this problem on a laptop with Intel graphics running Lucid, nor when using Nouveau drivers on this computer. I read somewhere that it could be due to gdmsetup not having it's config file, but even after trying the suggested workaround (opening gdmconfig and changing settings so the config file is made) the problem persists. If I run "killall gnome-panel" the panel disappears (not surprisingly) and then briefly pops back up before disappearing again for about 10 seconds.
Could this be something to do with Compiz? With the Nouveau drivers Compiz is disabled, so Compiz is the only thing I can think of that affects the panel differently between the two graphics drivers.
I just installed linux mint a couple of days ago and everything went great but now the grub screen takes about 20 seconds to a minute to load. It's not a huge issue but kind of annoying since the one that ubuntu installed took a second and I was at the menu. I imagine that it has to do with the background picture but i'm not sure if that would slow it down that much.
I am new to web server support. I have a request from my management to modify the logging slightly. Effectively I need to redirect a custom string from our http response into the apache access logs. When a user navigates to our site they receive a "dye" number that is associated with them. This number follows them to whatever cluster they are directed too. The string is formatted as such, com-company-dye: d0a2#6dfce. I need that that header dye to appear in the access logs so we can use that dye number as a key for troubleshooting issues though out our various monitoring systems.
I recently upgraded to Ubuntu 10.04, and Iv been using the ATI/AMD propitairy driver FGLRX and it worked fine inn 9.10.But now, the Bootsplash is ugly, and it takes ages for the OS to load, I didn't have this problem with 9.10.I try to remove it, but it chucks out an error stating a package doesn't exist.
For internal security reasons I need to prevent ourmcat logs from writing to the webserver local disk. We set up a separate logging server with rsyslog and need to pipe the log data to it.I am trying to work out how to configure tomcat to send all log data to the logging server via the rsyslog client (running locally) via a named pipe. We are on CentOS 5.6, Tomcat 6 and rsyslog 5.8.1. I need to know: 1) do we use the default logging library or log4j2) where is this configured in the tomcat config3) is there any code that would need to be written to achieve this
I don't mind that apache logs "rotate". By that, I mean that periodically the most recent log is renamed "*.1" and the older logs are bumped up a number In my particular situation, I do mind that the log that was "*.4" is deleted rather than being renamed to "*.5" when that periodic renaming happens.
I know most of the other /var/log files have the same behavior. What does this? Can I change it so my apache logs are "rotated" up but aren't deleted? I know this will take some hard drive space, but I have a lot of it.
I've just installed a fresh copy of 11.4 and the updates are driving me crazy. My internet connection is up, but the download stops and goes, and when I try using firefox, it takes a minute to even start to load any page. I have the system monitor up and it is telling me that the network is not receiving anything except for these bursts of data aprox. every 30 seconds.