Red Hat / Fedora :: Ram Drive With Atomic Write-to-disk
Apr 12, 2011
I'm only somewhat new to linux but I still don't have a real grasp of it's deep innards and I had a fairly outlandish idea that I'm wondering whether is possible/plausible or not.I want to run a game server on CentOS that has a very high dependency on fast writes-to-disk. Disk writes are pretty much the single bottleneck in this server.First I looked at allowing a high queue of writes to pile up before it flushes them to the disk, but I read that this causes fsync, which is still used commonly, to take a very long time.
I've been thinking about the possibility of running the server on a RAM disk, but I still want changes to be saved to non-volatile storage. Not all at once, but have it actively write the changes to disk. The hope is that this would smooth out the peaks and valleys of write activity and improve overall performance, but I have not seen this idea discussed anywhere.
So my question is, is there any plausible way to continuously copy writes to a RAM disk to a physical drive without slowing down the speed of the writes to the RAM drive below the speed of said RAM? Or is there a better way to obtain this sort of performance, short of investing in expensive equipment?
View 1 Replies
ADVERTISEMENT
Aug 26, 2010
Introduction : We have a C++ application in RHEL 5.4 platform. We are using TCP/IP socket programming as well to send and receive some sort of messages. We are using socket write and read command for this purpose and we are getting some run-time write issues in between. By doing various debugging and strace operations, we came to the conclusion that issue happens in some write attempts as follows.
Detailed Description : In the simplest case, consider I have a server and a client. Server writes some messages using write command and client is supposed to read the same data.Major code snippets in Server side is as follows [It is not feasible to extract the actual files and application codes as a whole, below are just the major commands used in server side]:
sockfd = socket(AF_INET, SOCK_STREAM, 0);
int reuse=1;
setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR,& reuse, sizeof(reuse));
[code]....
View 3 Replies
View Related
May 19, 2010
have two shared variables a,b which are related to each other.When multiple applications share these shared variables, access to them needs to be an atomic operation, otherwise the relation may break. So to ensure mutual exclusion, I'll put their modification under a critical section protected by lock.
Code:
critical_code
{
P(mutex)
a := something
b := something
V(mutex)
}
Lets say my hardware/OS/compiler supports atomic variables. Then I modified my above code as follows.
Code:
code
{
atomic a := something
atomic b := something
}
Can this code ensure mutual exclusion, when accessed by multiple applications?
View 2 Replies
View Related
May 22, 2011
is there a way to write/unpack .qcow2 hard disk image directly to real hard drive in Linux?(I know it's possible to unpack .qcow2 to .raw and then dd to drive, but I'd like to skip .raw since its large)
View 2 Replies
View Related
Apr 18, 2010
I am trying to scrape all of the info off of an old Fedora disk so that I can referb the box and put new stuff on it, but for some reason I can't get the permissions (even as root) to write to the disk.
It is currently mounted under /media/backup and I can browse it all I want, but still can't write.
View 5 Replies
View Related
Mar 21, 2011
I have a netbook I'm not using and which I transformed into a server with Apache, Tomcat6, Netatalk, Webmin, BIND9 and Tor.
Problem is, the disks never stop spinning because all of the programs write a few kb at least every few seconds to disk, even when nobody is connected to it.
My question is: Is there a way to have the computer boot from disk like normal (maybe even a squashfs), keep ALL CHANGES to ram and then save to disk when either the ram is full (unlikely because the server is rebooted every few days) or at shutdown?
I thought about a mixture of ramfs and unionfs but I'm not good enough yet...
View 4 Replies
View Related
Jan 4, 2011
df -h [URL] I did the following command to find everything is in /usr or /var, then tracked it down to /usr/lib and /usr/share as the main offenders, but out of all the directories none are more than 1mb or so.
du -sh /* | sort -gr | head -n 5
I tried to uninstall firefox, which is what got me in this mess in the first place, the log claims it will remove ~240 mb but failes on a "E: Write error - write (28 No space left on device)" [URL] If I could juggle something onto an external hard drive so I can uninstall firefox I would be out of the wood. Failing that I believe a new install is in order.
View 1 Replies
View Related
Apr 6, 2010
I've got a bare bones Ubuntu 10.04 set up (xorg, openbox, usbmount). My (vfat32) stick drive mounts, and I can see what is in the one directory on the drive, but I can't write to the drive unless I use sudo. I tried the obvious step of attempting to change permissions on the drive..
Code:
keith@quiet:~$ sudo chmod -R 777 /media/usb0
[sudo] password for keith:
[code]...
View 9 Replies
View Related
Jan 25, 2009
now i have the kde session, now but the ntfs disk drive icons are gone
View 1 Replies
View Related
Oct 31, 2009
I have been having mysterious problems with my comp recently and I think it might have to do with my OS not releasing filespace. Previously, my OS partition was full, then I deleted/moved some files, but now it says that still no space is available:
[root@cluster log]# df -h (simplified):
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 7.9G 7.6G 0 100% /
/dev/md0 459G 110G 327G 26% /export
[root@cluster log]#
The OS being /dev/sda1 (I am 99% sure, didn't set it up originally) and is CentOS4. As you can see, I have only used 7.6GB, and I should have 300MB available. Only thing I can think of is that when it was full, I moved matlab from there to the /export drive and added in a symbolic link to where it was on the OS drive so it would still work ok. Could this be why the space is not being freed up? We are in the process of installing a 16TB drive so we can free up some space or expand the partition, but somebody else here at work is handling that, so some other option that I could do would be best.
View 5 Replies
View Related
Jan 17, 2011
This probably is something simple, and may have already been addressed on here.I have a 1.4MB floppy disk image file that I would like to mount as a drive.
View 5 Replies
View Related
Apr 9, 2009
I have got a hold of a extra hdd along with a hdd enclosure. I have tried looking for information on how to install linux on to one but haven't been completely successful on my search. So I turn to all of you. I was also wondering if its possible to have it were I can use it on multiple computers so I can use it for computer repair.
View 3 Replies
View Related
Feb 2, 2010
I have a Fujitsu Lifebook A3130, and I have semi-recently installed Fedora 11. I am still a fairly new linux user, and would like to continue to use this great open source software, but it seems my DVD drive fails to play DVD's. The specs are here, but unfortunately they don't give much detail as to the model number of the drive in question. I insert the disk, and it pops up with the following dialogue:
You have just inserted a Video DVD. Choose what application to launch. Select how to open "disk name" and weather to perform this action in the future for other media of type "video DVD". It then suggests opening the disk in "Movie Player", and provides a check box for "Always perform this action". I hit OK, and it seems to try to open Movie Player, but then comes up with this error:
An error occurred
Could not read from resource.
Then it provides an "OK" button. When I click the button it goes back to Movie Player, and proceeds to not show any video.
I just tried to open the disk inside of Movie Player, via the "Movie" menu, then clicking "play disk 'disk name'", but once again I get an error:
Totem cannot play this type of media (DVD) because it does not have the appropriate plugins to be able to read from the disk. Please install the necessary plugins and restart Totem to be able to play this media.
And again it provides an OK button, which diverts back to the main Movie Player window.
View 3 Replies
View Related
Feb 7, 2010
1. While installing Fedora 12 i got this error "Isolinux disk error 20; AX=4280, drive EF". I run Ubuntu 9.10 on same system but now getting Error on installing Fedora 12 using a writeable CD to install that. A friend of mine install fedora from the same CD. I am using Pentium 4 2.40 GHz, 512MB Ram, Nvidia G-Force MX-440 SE. and a complete Partition of approx 10GB.In frist attempt CD was not able to boot, but after 3-4 trys CD was booted for couple of times and now it was not booting again. In that 2 trys, i got the same error as mentioned above.2. Before installing this I install ubuntu and it was running fine, but a little problem with that. Whenever I boot my system all things remain same but monitor settings went on 800*640. I usually set them to 1024*768 (Maximum my card can suppot).
View 2 Replies
View Related
Dec 16, 2010
I had a set of rather unusual problems on installing a new 500Gb hard drive on my F14 system, I've solved them, but they were that unusual that I thought I should share them in case anyone else gets the same thing! I'd been experiencing intermittent faults on one of my drives, (lock ups for no reason, occasional boot ups that failed due to ' disk unreadable errors' and other odd errors). I assumed that the drive was failing, but it always showed 'Heathy' on disk utility! This was the disk with the OS on, plus my main 'data' disk had some bad sectors, so I thought I'd buy a nice big 500Gb and reinstall the whole system.
I backed up all my data to an external 'USB' drive, opened the case, (a big old under the table 'desktop', why do they still call them 'desktops'?) shoved in the new SATA drive and rebooted, intending to format the new drive to EXT4 and partitioning it before installing F14 again! OOOOOOOOOW! Major drive failure, missing OS, whole list of SDB errors! Control D, to reboot, BIOS only sees one drive; SDA (I have three, two IDE and one SATA) plus the one I just put in makes four. I go into BIOS and discover that it not only cant see any of the other drives, but my two DVD drives are missing too!
Now I've been building my own systems and mucking about with computers since before you could buy them, and Ive never seen a problem like this one! At first I thought the new drive had screwed my system, for on removal, the problem persisted! Then I noticed that the IDE connector in drive SDB, (the long 40 pin job) was just slightly out of line with the back of the drive, pushed it firmly in and what do you know, everything works! We moved recently and I think the vibration was enough to loosen the connector to give intermittent faults, and pulling the cables about to get the new drive in, pulled it out further......
View 10 Replies
View Related
Jun 14, 2011
I downloaded the latest version of wubi and when I click to run i get the error "pyrun.exe - No Disk. There is no disk in the drive. insert a disk into drive DeviceHarddisk2DR2".
View 2 Replies
View Related
Mar 12, 2011
When I'm recompiling apache and php, it doesn't go through and I get this error:
Code:
checking for atomic.h... no
/usr/include/mysql/my_global.h:364:24: error: asm/atomic.h: No such file or directory
[code]....
View 5 Replies
View Related
Apr 5, 2011
I am using kernel 2.6.32.21, and my hard disk is West digital WD10EARS-00Y, 1TB. This disk is just for data, I made 2 partitions on it, each has half. And I have another small disk for system. I am using ext3.
this is my fdisk /dev/sdc
Device Boot Start End Blocks Id System
/dev/sdc1 1 60800 488375968 83 Linux
/dev/sdc2 60801 121601 488384032+ 83 Linux
[Code]....
I ran some dd to test the write throughput on /dev/sdc2. If I run it in /data2, I got around 70MB/s. If I create some directory, say /data2/dir/, and run dd again, I might get 60MB/s. Sometimes I still get 70MB/s, sometimes I get 60MB/s, differs for directories.
I wonder if this is because the allocation policy of file system, ext3, or this is from my hard drive?
View 7 Replies
View Related
May 13, 2011
I need to insure the "writer" to named shared memory (small size, 160 bytes) completes his task such that the many "readers" always get the latest data. The machine is a multi-processor (8 CPU, 24 Gig RAM), non-real time system. The system has multiple processes that are also multi-threaded. Thankfully, there is only 1 "writer" and many "readers". There is no semaphore or mutex locking. "Readers" must not block each other and especially not block the "writer".
By design, it is expected from time to time a "reader" will be in the middle of reading when the "writer" begins to update the data. I want to protect against the case of the "writer" being interrupted and a "reader" completes it's read before the "writer" wakes up and completes it's changes. In this case the "reader" will get corrupt data, some new and some old.
View 7 Replies
View Related
Jan 13, 2010
Is there anyway in Linux that can achieve atomic increment/decrement for an integer variable without being interrupted? It means that the thread should not chance for other thread to run until theincrement/decrement is completed.
View 11 Replies
View Related
Apr 1, 2010
I have formatted to EX3 in gparted all went fine, however I cannot rename the disk or write anything to it as access is locked to root Furthermore a file has been created called "lost and found" its locked and 46GB in size - what is this?
View 14 Replies
View Related
Jun 20, 2010
I have Ubuntu 10.04 installed on my Armada laptop with a Teac CD-W24E internal and a Plextor-DVDR PX504A via USB. I use K3B for writing CD data projects to 700mb CD-RW 1-4x discs. K3B writes ok on both drives if the inserted disk is a blank disk. If I insert an appendable disk with lots of space left, it asks to blank it before writing. I find no way to write on an appendable disk with K3B. Is this a software limitation or am I missing something?
View 9 Replies
View Related
Mar 17, 2010
Well appears to be that i have two partitions..one of 300gb i did on windows time ago and one of 100gb where i place my DEBIAN LENNY 5.0, the problem is that i cannot write into it..only i can copy stuff from it.
View 13 Replies
View Related
Nov 17, 2009
Ive installed Gaussian '03 on fedora Core 10, but I'm unable to run it. It aborts and i get the following error
Code:
Erroneous write during file extend. write -1 instead of 4096
Probably out of disk space.
Write error in NtrExt1
View 3 Replies
View Related
Jan 5, 2011
I managed to have the ATI Catalyst driver (10.12) installed on a machine with F14 x86_64, Radeon HD 57xx and I also installed the ATI SDK Samples for OpenCL. ATI Catalyst reports OpenGL fine, glxinfo is fine, all OpenCL samples that do not use OpenGL are working fine. I used the "install to hard drive" feature of the F14 x86_64 desktop disk. Problem: which headers and libraries do I need to install in order to use OpenGL in an application? I would also like to have the OpenCL working on the system for another application.
Previously I had a small application working with SDL + Mesa library on FC9 using software rendering, but after F14 installation and ATI card addition the app is not working. Files gl.h, glu.h are missing. I checked the system and I found only a glew.h header. I tried linking against glew but it complains about glu.h (file not found).
I looked on the Internet and I found a post that someone suggested installing xorg-x11-devel, but there is no such package. I tried installing libX11-devel...f1.x86_64 but the gl.h/glu.h were still missing. I tried compiling the latest Mesa library with make linux-dri-x86_64 and it complains about egl. All dependencies have been checked (dri2proto, X11 version, libdrm, kernel version). After Mesa installation ATI Catalyst is no longer reporting OpenGL and glxinfo is not working. Am I supposed to install just the Mesa library without the ATI driver? Am I supposed to use glew only with the ATI driver?
View 2 Replies
View Related
Sep 1, 2010
According to [URL], gcc 4.3 supports atomic built-in function "GCC can now use load-linked, store-conditional and sync instructions to implement atomic built-in functions such as __sync_fetch_and_add" However, I am still using gcc 3.4 on my Redhat EL4. How do I get this built-in function installed?
View 2 Replies
View Related
Jan 31, 2010
I have this hdd and another with XP in it but when ever I chnage to UBuntu, the clock goes out of whack and, makes me manually change it whereas in my XP I have an atomic clock installed that I just click on 'Ping' and she's right. I've looked at heaps of downloads for similar on here but, I haven't found a free one other than a trial type that I'm not interested in, that works so, why is Linux so difficult in this regard? I will accept all the help I can get on here because I like the thought of being able to get away from Microsoft.
View 2 Replies
View Related
May 18, 2010
Upgraded from 8.04 LTS to 10.04 LTS, but can't get the sound working.Have downloaded the 10.04 Ubuntu CD image to installwhen I try to follow the directions at[URL]BurningIsoHowto for burning the image............it says to Right click on the ISO image file and choose Write to Disc. I have no WRITE TO DISC option when I right click.Will some kind soul please tell me how to burn the downloaded CD image to a CD?
View 6 Replies
View Related
May 18, 2011
I am doing a program that reads data from a gps and some other devices and writes some files with all the information. When I run it normally it works fine, but if I run the program in the background (with the ampersand) files are not created until I bring it to foreground or close it. I am confused, the program should run the same way with and without the ampersand. (Could it be that the main process that creates all the threads does not create them when it is executed in background? It seems like if the program is stopped until I take it to the foreground).
View 4 Replies
View Related
Oct 26, 2010
I have encountered this problem of memory usage is increasing as the during the my program is being run until 1Mb is left then it stays at that.A part of the program is this:
Code:
#define WRITE_BUF_SIZE (1024*1024)
void post_data(const void *data, unsigned long size)
[code]....
View 3 Replies
View Related