Programming :: SHM Memory Counted As Cache Memory With Kernel 2.6.18?
Aug 25, 2010
Is that possible that SHM shared memory is counted as cache memory on Linux with kernel 2.6.18?If find it really odd since this memory is not file backed, but I have a piece of code that loads data using shm_open+mmap, and it generates an amount of cache memory in /proc/meminfo that corresponds exactly to the amount of shared memory (I load that data from a file but I am using posix_fadvise(fd,0,0,POSIX_FADV_DONTNEED) to ensure this file is not cached and I made sure that it is working as expected). As far as I know SHM memory was not tagged as cache memory with kernel 2.6.9.If it is the case it is really unfortunate since normally cache memory can be considered to be part of the "available" memory since it can be flushed promptly but this is clearly not the case with SHM memory... Is there an easy way to get the total amount of used SHM memory on a system?
I am new to C and linux. My code below does arbitary writes but I cant figure out where or how it does it.
I am calling the insertNode() function with seq = 'MISSISSPPI$' and alphabets = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ$'
Code:
Weird behaviour I should mention is that when I check for NULL pointer in node->child[index], the unassigned values are not null anymore, they point to arbitary memory.
I've installed my debian sid about one month ago (first xfce, next gnome) but noticed that it's kind of really slow. The upgrades take ages, launching (and using) firefox takes so much time,... In comparaison to my ubuntu, archlinux (on the same computer) or previous installation of debian there is clearly a problem somewhere.Today I tried to do a "top" sorted by mem usage : 3.5% xulrunner-stub, 2.1% dropbox, 1.4% aptitude (doing upgrade), 1.4% clementine,... nothing terriblebut still I've 2.7Gb or RAM used (more than 50%)
$ free -m total used free shared buffers cached Mem: 3967 26851282 0 79 1938
(Ubuntu Linux server, 64-bits)I was troubleshooting a problem with a file (~3.0 GB) which I had just downloaded, but it was failing the integrity test, when I discovered something really unusual.First this is the MD5 of the file after download, which didn't match the expected value:
This was really unexpected. Since I have a lot of RAM, I suspected this was the effect of caching and something was going awry with it. I decided to retry with the whole file from disk, for my surprise:
~% sudo sysctl -w vm.drop_caches=3 # This linux command invalidates vm.drop_caches = 3 # everything in the memory cache. ~% md5sum media.iso 2992aa6270f6e1de9154730ed3beedc1 media.iso
I redid it and now it seems to stay consistent, although this still isn't the value I was expecting. Certainly, the contents in memory cache were different from the contents on disk.This is the big problem.To fix the download, I created a torrent on the source machine and opened it in the target machine. Five 1MB chunks out of ~3.0GB failed integrity check. I used the torrent to fix these file chunks and how the file integrity is ok.The problem now is to determine where the data got out-of-sync.
I tested the memory with memtest86+, all but the bit fading test. I was expecting to see some failing memory module, but there wasn't anything. Everything is ok.Filesystem is Ext4, over LVM2, over a 3-disk RAID5 array.Ext4 is considered stable, and if data were inconsistent between disks, mdadm would have warned. But there is nothing in the logs. S.M.A.R.T. error logs are clean, the disks are new (have less than 30 days of "power-on-hours").I'm looking for information about any data-loss bugs in my current kernel (2.6.35), but there doesn't seem to be anything, as far as I looked.what else I could check, or where exactly could be the defect/bug?It is a Ubuntu 10.10 64-bit, Core i7 930, 6 GB non-ECC RAM.
Update: I confirmed that the files are being correctly written to the disk, the pages are being altered after they are read from disk, while in memory. I did a lot more memtests (I left it doing bit fade test overnight),and still nothing. All memory modules seem ok.Some more tests:
~% md5sum media.iso cc8bcf1ce67ff7704eadc2222650c087 media.iso ~% cp media.iso tmp[code]....(direcat is a version of cat that reads with O_DIRECT, that is, bypassing page cache)There is a clear pattern: it always happens to the 2nd byte in a 16-byte alignment. In that byte, almost always the bit 4 (LSB) flips to one, but there was one instance where bit 2 flipped to zero.
i am running Ubuntu Lucid x64 as a fileserver that shares its files via SFTP, NFS and Samba. Currently the hard disks are configured to go to standby if they are not needed. This works perfectly as long as no one browses the shares or my HTPC is running: That one repeatedly looks through the shares for new music or movies. In other words my problem is that the disks are spinning up a lot more often than they should have to. Additionally the spin-up time delays the response time while browsing. Since the machine has a lot of unused RAM i want to tell the kernel that it should keep the directory structure in memory. That way the disks would not need to spin up every time someone browses through the directories.
It seems that something is causing the memory cache to be cleared when the screensaver launches. Then when moving the mouse or hitting a key to stop the screensaver it takes forever for the already launched and minimized applications to redisplay. I am using the ubuntu classic desktop.
Now time to rant.
Natty has so many bugs it is impossible to work around them all. This memory issue is one of the worst. The dead zone caused by the static application switcher, and the application switcher plugins for compiz (and probably any override redirect window) is another big one. There is not a single window decorator for compiz that works right. I have encountered two packages with binaries that won't even run (emerald and xpdf). The nvidia drivers seem to have issues (this may be related to this memory clearing issue).
Without a doubt more bugs than any other release of Ubuntu to date. I have worked around most of them by recompiling, and etc., but this is a real pain.
This is all aside from the idiotic switch to unity, when it is clearly not ready for release. The general direction of Ubuntu is really starting to look bad. By the way, what is up with the decision to uglify the desktop with gray scale icons (I know this is an old one, but I haven't really ranted on Ubuntu yet).
I have a Linux server running CentOS with 2GB RAM. I am searching in google and also in this forum but could not find any luck yet. I am searching the way that we can receive notification by system to a specified email when the memory reached a specified value we defined.
Is there any tool or sort of shell script we set via crontab every minute to check that?
I am using malloc and frees a lot in my program. It shows its allocated but when i remove it doesnt show as the memory is removed(I am using the top command to view VIRT memory usage). If this continously grows what would happen to my program (Will it go out of memory?)
I am looking to buy some memory for my netbook. Currently I have 1 GB of DDR3 memory. However, the specification says that 2 GB of memory is the max. However, when I do the following it says that 4GB is the max:
I have a computer with 16GB of ram. At the moment, top shows all the RAM is taken, (NOT by cache), but the RAM used by the various processes is very far from 16GB.I have seen this problem several times, but I don't understand what is happening.My only remedy so far has been to reboot the machine.
I am looking for free database that has low memory usage and innodb and memory like engins that has C API and support trigger and client/server support for using in embedded linux systems.
we found that if we use 'top' to show the memory usage of a server (SuSe Linux 10), we can get virtual memory usage as well as 'Resident memory' usage. For virtual mem or a particular process, it is around 1.1GB, which is large but for resident memory, it only consumes 300MB. Are there anyone who knows what the differences are? I would also like to know whether the difference (1.1GB - 300MB) = 800MB are actually available for use by other applications in the system.
I am monitoring physical memory in a server I administer, and my hardware provider told me they had increased physical memory size to 4Gb... However, using several tools (free -m; top; dmesg | grep Memory; grep MemTotal /proc/meminfo I discovered that I actually have 3Gb, not 4... But, my doubt comes from the fact that dmesg | grem Memory tells me I have 3103396k/4194304k available The first number is effectively 3Gb, but the second one, is 4! so, why I am looking at this two different numbers?
I have had a fresh install of Ubuntu 9.10 and installed some software after that.Since third some, some process is eating half of my memory.I have checked processes running in system manager but everything is normal.Maximum is consumed by compiz which is about 26 mb, seems very normal.I did restarted my computer several times, and in the start for 5 mins, its fine after that again my cpu fans runs at very fast speed and my one cpu is used up 95 % (I have dual core).Please help me out, this invisible thing is driving me crazy.I am attaching my htop screen shot (sorted by cpu %), now the cpu is not used by completely but fan is still struggling hard and fast.
I am writing an application that wants to access periphals registers outside the standard (allowed) memory area.
Doing so gets me "segmentation fault".
I know, this is natural behaviour.
One way of getting around this is writing the module which has to be loaded by linux. I will consider this some time later.
For now, I want to come to some quick result and allow linux or gcc compiler to write to those memory areas of periphals. Is there a direct way to do so?
This is my first post in these forums. I'm still quite new to Linux (using Mint 9) so please bear with my not-very-articulate question(s)When I boot up and open up a tty terminal I get a message saying "Memory corruption detected in low memory." I've done an extensive google search about the issue and it seems not uncommon. I ran a memtest with no errors returned, so I'm sure that there's nothing really wrong with the memory; apparently it's a bug in the kernel that's causing this.
I found from command 'top' that 8GB memory are used. However, using command 'ps' with some options to grep the running processes and then summing up the memory used by the running processes are less than 2 GB. Where has the used memory gone ?
I have been setting up a vps I got out with bhost.net, with CentOS installed. I've been learning and have set up everying I need with the exception of ftp/sftp.
Using yum I installed vsftpd and ran into problems, thinking it was something I might of done I did a fresh install of CentOS and I still recieve the same problem on a fresh install so it is nothing I have done to the server.
The problem is when connecting via a sftp client I get an out of memory error. This error is listed in the putty faq ( url ) under A.7.5, there is a brief explaintion of the cure under A.7.6.
there is mention of a login script but I don't know where this is located. I'm a novice at Linux but by no means incompotent when it comes to computing.
I'm fairly new to linux Red Hat. We are running Rhel 3 on our VM's. We ran into a issue, (Bug 121801 - athlon-smp kernel does not support >4GB of RAM. what the stepos are to upgrade the existing kernel to the new i686? .
i have installed kernel-xen to my centos 5.4 server ( with yum install kernel-xen ). My server has got 8192 mb pyschal ram with PAE kernel; but after i installed xen kernel to system, my ram is being 7919664KB....how can i fix this ? i want see 8192MB RAM with xen kernel
I have a server for virtualization (Xen) and when I do a free -m it does not show all the memory (which should be 64GB), it shows 32GB (a considerable difference, isnt it?).
My hardware is a Dell R410 server: Dual Xeon 5530 64GB DDR3 4x SAS 500GB in RAID 10
I tried to compile Xen 4.0.1 but I can't solve a compile error that it gives me (if you want to take a look: [URL]) that is the end of the compilation including the error. But, I installed Xen from yum directly after reformating the HD and it works fine now.
My kernel version: 2.6.18-194.26.1.el5xen x86_64 CentOS 5.5 64 bit [root@server1 xen-4.0.1]# cat /proc/meminfo MemTotal: 33554432 kB
