General :: System Do Fragmentation On Virtual Address Space?
Apr 21, 2010
I have millions of small chunks of new's and delete's in my program. Even though no memory leak is detected through valgrind or purify, memory grows. But the growth is degrading over the number of times of the execution of the tasks. Can i think that fragmentation plays a role in this?
As i undertsand - out of 1GB of the virtual Address space for Kernel from 3GB to 4GB of the process address space, Kernel image (code, data, bss, stack, heap) resides staring @0x0 address. Vmalloc area starts either at the end of Physical ram size or at 896M. This 896M cap is mandated to ensure that minimum of 128MB is reserved as vmalloc_reserve for vmalloc,kmap etc.
Is the understanding correct? Now trying to map Physical Zones into this 1GB address space
Initial 16MB is mapped to ZONE_DMA 16MB - 896MB is mapped to ZONE_NORMAL 896MB - 1024MB is mapped to ZONE_HIGHMEM
Does this mean that Kernel image is residing in ZONE_DMA area? Any call to vmalloc() in kernel code will return address beyond 896M? insmod of any LKM will internally invoke vmalloc() to obtain contiguous area - where will this code physically located along with rest of kernel code in ZONE_DMA or in ZONE_HIGHMEM?
I am using malloc and frees a lot in my program. It shows its allocated but when i remove it doesnt show as the memory is removed(I am using the top command to view VIRT memory usage). If this continously grows what would happen to my program (Will it go out of memory?)
In a 32-bit system, max memory addressable is 4GB. Now Linux kernel does memory mapping division of 1GB for kernel address space and 3GB for user address space. That means 4GB of virtual address space is divided between kernel (1GB) and user (3GB).
Q1. All virtual mapping utilizes the available physical RAM without any division? I mean to say that if RAM is 512MB then a page in kernel space can lie any where RAM (leave aside old PCI dma accesses)? (How this fits to fact that kernel memory is non-pageble)
Q2. If a process is created in user space, it has visibility 4GB address space or 3GB address space?
Im an academic (university networks and security lecturer) studying/teaching network and operating system security, and inspired by the work of Hovav Shacham set about testing ASLR on linux. Principley I did this by performing a brute force buffer overflow attack on Fedora 10 and Ubuntu 9. I did this by writting a little concurrent server daemon which accidently on purpose didnt do bounds checking.
I then wrote a client to send it a malicious string brute forcing guessed addresses which caused a return-to-libc to the function usleep with a parameter of 16m causing a delay of 16 seconds as laid out in [URL] Once I hit the delay I new I had found the function and could calculate delta_mmap allowing me to create a standard chained ret-to-libc attack. All of that works fine. However .... To complete my understanding I am trying establish where I can find the standard base address for ubuntu 9 (and other distros) for the following, taken from Shacham:-
Quote:
[code]....
/proc/uid/maps gives me some information but not the base address ldd also gives me the randomised starting address for sections in the user address space but neither gives me the base address. Intrestingly ... when a run ldd with aslr on for over (about) 100 times and checked the start point of libc I determined that the last 3 (least significant) hex digits were always 0's and the fist 4 (most significant) where between 0xB7D7 and 0xB7F9. To me this indicated that bits 22-31 were fixed and bits 12-21 were randomized with bits 11-0 fixed. Although even that doesnt define the boundaries observed correctly.
Note: I am replicating the attack to provide signatures to detect it using IDS, and for teaching purposes. I am NOT a hacker and if needed to could reply from my .ac.uk email address as verification.
When I create virtual ethernet interface and do a ping -I <v_int> <host> the outgoing address is the one of the physical interface and NOT the virtual interface.Is there no chance that trafic will go out with virtual interface address??Incoming trafic is done well i.e. responds to the virtual interface have the virtual address.
My problem is that I have 2 modems and want to check both default gw behind the modems. If I do a "normal" ping both are reachable over default route even the modem which is not the default route will not work because ping goes over the working modem.So I have 2 routing tables and want to route the virtual interface to one modem the other to the other modem
I need a small shell based program that prints the mac address of physical ethernet adapter from it's firmware. I need this utility for license generation and appliance activation. I have tried several example but none of them is flawless, The easiest method I have found is to parse the output of "ifconfig" command but it has also some drawbacks.
1. Firstly program should differentiate between physical and virtual adapters. Physical means installed on board(wired or wireless) or installed additionally. Virtual adapters are those created by VPN or created by virtualization apps such as VirtualBox/VMWare etc. I am not interested in virtual ones.
2. In case of more them one physical adapters(wired and wireless), it should print the mac address and description(name & vendor) of both/all adapters.
3. If media is disconnected then also it should be able to read the mac address and description(name, vendor) of card.
4. This one is bit complex. I know that 'ethtool' can show you the universal mac address but it's limited to use only 2 types of drivers and won't work in all cases.
Suppose a process creates multiple shared mem segments as part of it job.Now is it always possible that on restart, that process can reattach to those previously created shared mem segments to the same virtual address spaces (returned on first time creation) ? My intention is use pointers over shared memory.So on restart I want those pointers absolute value to remain valid.
I heard that the ext file system does not causes file fragmentation. Could some one explain how this is achieved. And how come a file does not divides in to fragments as compared to Windows based Filesystems.
A process is trying one access to memory, for example through an array (ex.: vect[0]=123. What happens?
Here below what I guess but I'm not sure and accept any comment (please, distinguish between "the system" and "the CPU" in case).
Let's suppose swapping to disk disbled.
We have two scenarios: without and with cache.
If no cache is present in the system: 1. The CPU must discover the phys addr of vect[0] virtual addr. To do that, has to read from 3 (or 2 depending on the system?) pages tables, stored in memory as well. 2. The CPU writes to the final address.
These mean 4 memory accesses.
If cache is present: 1. Like above but, if the pages tables are in cache, we have 3 accesses to that. 2. If the req. page is not in cache, it's reads from ram and transferred to it. Afterwards, cache is written. In the best case we have 4 cache accesses.
I have read a couple of articles on how dynamic linking works (those stuff about got, plt and lazy binding), and I am still not sure why you need to do dynamic linking in such a complicated way.Suppose your program uses a function in a shared library that needs to be linked dynamically at run time (like a printf). Why can't you statically decide the virtual address of the function at compile time? After all, all you need to do is to enter the page table entry corresponding to the address of the function if the library has been already loaded to a physical page frame.
I setup a virtual server with these configurations:
1. Mail server: POSTFIX / DOVECOT 2. Webmin / Virtualmin
I am getting email to desired folders locally but I am not getting email from outside to desired users. Emails are bounced back with message "Address not found in virtual alias table".
Recent log errors are:
Sep 8 04:25:49 server1 postfix/smtpd[28002]: cannot load Certificate Authority data Sep 8 04:25:49 server1 postfix/smtpd[28002]: warning: TLS library problem: 28002:error:02001002:system library:fopen:No such file or directory:bss_file.c:122:fopen('/etc/pki/postfix/root.crt','r'):
I am in verse to test "Multicast Packet filtering".I want to setup Virtual Machines to be servers with bind option set to a multicast group address of 225.0.62.87.Then I want to configure the client VM, connecting to the multicast group address and setting the TTL as needed.
Logical Memory Space of 4GB is divided in to 3GB User Space and 1GB Kernel Space. Always. Correct?
1. How can we change it? (just changing value of PAGE_OFFSET is okay?)
2. If system have only 256MB of memory (embedded system) and suppose Kernel Modules eat away all the memory during boot. User space will be left will no memory. Is this case possible?
i just installed Fedora 12 (via vbox 3.1.4). stupidly i started with 8G and, of course, ran out of space on the / filesystem. i added 8G to the drive which now shows as 17G.based on the below outputs, it looks like i have 8G more space available on the drive. based on reading other posts it looks like i need to use parted to increase the size but not sure how to do that.
i want to add space to my file system, i want to slice some space from my home directory coz it has a lot of space...i am just adding applications, but im runnint ofspace....what shall i do, without reformating or reinstalling my ubuntu 10.10?
I want to add an ipv6 address in linux. i am going to use the following command. # /sbin/ifconfig <interface> inet6 add <ipv6address>/<prefixlength> Can anyone explain how this command will works? Is that prefix length value is fixed to 64 or is it vary?if so,can you explain how it could be?
My HP system suddenly stopped taking ip-address for wifi. I tried to give it static ip-address, it took it. But still I am unable to connect to internet. I tried this both with wicd and network manager.
I want to install CentOS with only 250 MB Space.How can I do that? What package can I select?I have been challenged by Manager to wants me to restrict < 250 MB for just OS installation.
Being new to Linux I am not sure of which Virtual software to use in my Opensuse Linux? I want to try FreeBSD again but I want to use some type of Virtual Machine .
I have RHEL running in Virtual Box and my native OS is Win 7. From a purely educational standpoint, I want to be able to access RHEL from Win 7 over SSH. I download putty but don;t know how to make it do what I want. Ideally, I would like to use the linux command prompt at a minimum and preferably access the GUI apps also. IS this possible? I am just trying to explore and expand my linux knowledge.
We have designed a board with Cirrus Logic(arm) processor, A Flash memory and some other peripherals have been connected to that. While building kernel we have selected MMU support. We have written few custom drivers for keypad,LED,LCD. But I would like to know how virtual memory mechanism can be helpful here even though there is no any hard disk has been connected. Where will be the virtual memory reside.