Software :: Searching Through Massive Amounts Of Data?
Feb 27, 2009
The company that I work for has massive amounts on our file server and that amount continues to grow. What we are looking for is a search appliance that will make it easier to search all documents on the file server and also search the content of those documents. I don't really like the idea of everyone using an app like X1 and searching the share drives that way on their individual PC. I would like a search appliance.
View 5 Replies
ADVERTISEMENT
Jul 16, 2010
Sometimes I need to copy a huge directory to another directory (local filesystem), and usually I will use the "cp" or "rsync" commands. These commands are good, but depending on the size of the data being copied, the copy is painfully slow. I realize we are limited because of the hardware we have with it's limitations, ie, I/O speed, and the filesystem (which is usually ext3). Are there any other utilities that maybe not well known, but can handle copying large amounts of data? (mostly in the TB range)
View 2 Replies
View Related
Sep 1, 2009
Are there any tools out there that let me select a bunch of data and burn it to multiple cd's or DVD's? I'm using k3b but have to manually select cd and dvd size amounts.
View 1 Replies
View Related
Feb 15, 2011
we've been trying to become a bit more serious about backup. It seems the better way to do MySQL backup is to use the binlog. However, that binlog is huge! We seem to produce something like 10Gb per month. I'd like to copy the backup to somewhere off the server as I don't feel like there is much to be gained by just copying it to somewhere else on the server. I recently made a full backup which after compression amounted to 2.5Gb and took me 6.5 hours to copy to my own computer ... So that solution doesn't seem practical for the binlog backup.Should we rent another server somewhere? Is it possible to find a server like that really cheap? Or is there some other solution? What are other people's MySQL backup practices?
View 8 Replies
View Related
May 29, 2015
I have been using Wheezy for two years, and everything was OK, so I decided to upgrade to Jessie. In fact i decided to perform a clean install of jessie, so I formatted the partition where Wheezy was installed, and the /boot partition, and I installed jessie in those partitions.
As I had done with Wheezy, I installed jessie in an encrypted LVM, and the installation was ok (well, almost everything was OK, because grub and plymouth were not working, but I will open another topic about it).
First thing I did after installing Jessie was editting sources.list in order to download a few programs (plymouth, firmware linux non free, libdvdcss2, gufw, menulibre). Moreover, I downloaded a few progrmas from the Debian install DVD (flashplugin, VLC, chromium, clamav). i did not perform a dist upgrade because i was not at home, and where I was I did not have a Wifi connection (so I was using my mobile connecion -USB ethernet with an android phone-. BTW I had just performed a hard reset I had not installed any apps after that, so that mobile phone was "clean").
After that, I created a desktop user account, and I rebooted the laptop . When i rebooted, I started to tweak my user account: I edited dconf, and the gnome shell theme; and I started my mobile connection to download three extensions for gnome shell (window list, simple dock and activities configurator. I had used those extensions with Wheezy, and I had never had any problems). Ten minutes after I started the mentioned mobile connection, I reveived a SMS as I had used over 300 MB. Gnome monitor showed that I have downloaded 300 MB, and the android native data usage app showed the same. I did not download any video or music neither watch any videos on youtube, dailymotion..., I did not visit any suspicious web
I had a look at the apt logs and I din not find anything significant (I was using a no sudo user account, so i was not able to perform a dist.upgrade), I had a look at the download, video, music, picture folders and i din not find anything. I tried to check the iceweasel cache folder, but there were so many subfolders I could not check everything.
View 4 Replies
View Related
Apr 30, 2011
I just installed Kubuntu 11.04.
I ran VLC and it brought my PC to a halt.
I attached a picture of memory usage from htop.
Why do you think VLC is using so much system resources?
View 3 Replies
View Related
Mar 6, 2010
I am trying to move a large amount of files (over 30k and 86GB) to another HDD but I get a Augment list too large error?? I tried rsync, cp, mv and still the same error
View 1 Replies
View Related
Aug 20, 2009
In my cpanel, it reports the RAM on my server as:
Physical used 17.31%
768MB Free (928MB Total)
yet when I use top I only have 88Mb free:
Code:
Mem: 950568k total, 862464k used, 88104k free, 91072k buffers
Swap: 522072k total, 136k used, 521936k free, 604752k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
[code]....
View 3 Replies
View Related
Oct 27, 2010
I've been using GIMP's 'save for the web' tool to reduce the file sizes of images.
I now have a directory with about 50 images. I'd like to avoid processing them all by hand.
I have a (very) basic knowledge of programming, and I'm comfortable with the commandline. I don't mind doing some homework on how to use new tools.
All I'm really concerned with here is reducing the file sizes of the images I have.
What possible pathways are there for me to prepare large amounts of images for the web?
View 4 Replies
View Related
Dec 20, 2009
i have a car stereo that reads a USB drive with all my music on it, however to sort through the music it uses a method of finding folders containing music, then displaying them all in a list. i find this interface annoying because in order to sort the music by artist i have to go and manually move it out of the album folders by hand, this takes a long time for 11+ GB of music so i was trying to use the linux CLI to quicken the process. use a command like this
Code:
mv /media/usb/music/*/*/* /media/usb/music/*/
but for some reason this moves all my music into the last folder alphabetically in my drive, the music is all pre-arranged like this /media/usb/music/artist/album/song
View 5 Replies
View Related
Nov 30, 2009
I have a small office network (about 30 machine) with linux gateway (6Mbps internet bandwidth). Every user get only 500Kbps bandwidth, and they use the internet very poor. The internet getting slow lately, and I noticed that there are huge amount of small packets (78 byte, 48 byte) coming to linux machines. My question is: How can I solve which machine(s) sending those small packets? Do you have any ideas with netstat command?
View 1 Replies
View Related
Dec 8, 2009
We're load testing some of our larger servers (16GB+ RAM), and when memory starts to run low they are kicking off the oomkiller instead of swapping. I've checked swapon -s (which says we're using 0 bytes out of 16GB of swap), I've checked swappiness (60), I've tried upping the swap to 32GB, all to no avail. If we pull some RAM, and configure the box with 8GB of physical ram and 16 (or more) GB of swap, sure enough it dips into it and is more stable than a 16GB box with 16 or 32GB of swap.
View 6 Replies
View Related
Mar 24, 2010
The issue I am currently facing is more of an annoyance / curiosity, and it may not even be a problem, but it sure feels like one. Background: I am becoming a computational chemist (grad school begins in the fall) and the code I run is all in fortran. I am currently compiling with gfortran. When I compile the code (on a box running ubuntu server), everything appears to compile fine, but the linking stage is taking five to ten minutes. I ran the time command while making it and got the following results.
Quote:
time make
gfortran -c -O3 -fomit-frame-pointer -finline-functions -ffast-math suijtab.f
Linking testCompile ...
done
[code]...
I just don't understand why it is taking 5 minutes of real time if it only takes 10 - 15 sec of system time?
View 4 Replies
View Related
Sep 1, 2011
I run minecraft with java and it uses massive CPU up to 60%. Java is dedicated almost 1GB of my 4GB RAM. I have a cheap AMD quad core proccessor (I think its equivalent to dell i3 or dell dual-core.)
View 5 Replies
View Related
Feb 28, 2010
Somebody, without saying anything to us, has deleted (not moved) a lot of wiki pages for laptop testing and setup guide. A lot of people were working hardly to document as best as possible each laptop computer to configure Ubuntu GNU/Linux.
Here follows a list with only a few of the lost pages:
[url]
[url]
[url]
[url]
View 9 Replies
View Related
May 19, 2010
my processor usage spikes and it fades epic slow then there is no lag when the screen saver comes up
the screen saver is skyrockets my computer's specs are in my signature
View 1 Replies
View Related
Aug 16, 2010
I've been having some problems with Lucid; all my applications seem to be hogging memory like no tomorrow. Within about 15 minutes from booting the system, processes like Google Chrome, Nautilus, Python, Pidgin all start to take seemingly too large amounts of memory.
Chrome is the worst one, easily shooting over 200-300MB of my 2GB's of RAM. I would have reported this as a bug in Chrome itself, but my other applications seem to share the problem to some extent. Also: My colleague has identical hardware and identical versions of Ubuntu / Chrome, while he has no memory problems whatsoever.Currently I am running Chrome, Geany, Pidgin, Thunderbird and FileZilla. For this and itself, Ubuntu now consumes 1.8GB of RAM (that's including 500MB cached).
View 7 Replies
View Related
Aug 7, 2010
The system is 13.1. Updated using slackpkg. All other software, with the exception of closed source Vbox, is installed via SB.orgDid a slackpkg update this morning for firefox 3.5.8 and there was also a glib upgrade (likely culprit). Now it seems that most of my software does not work. Did I do the upgrade wrong?
Code:
frank@hdwise:~$ firefox
/usr/lib/firefox-3.6.8/firefox-bin: symbol lookup error: /usr/lib/libgnome-2.so.0: undefined symbol:
[code]....
View 10 Replies
View Related
Aug 2, 2010
Though under Windows my Internet connection works fine and fluent under Debian linux it connects and disconnects repeatedly at short amounts of time. It didn't used to be like that but at a certain time i was forced to install pppoeconf to get my DSL internet connection going, after that even if i've uninstall pppoeconf and now i'm only using nm-applet to monitor my internet connection i still have this annoying problem.
View 1 Replies
View Related
Nov 8, 2010
I have read several of the thousands of posts regarding installing of fonts on Fedora. All come to the same conclusion of having to install each font individually.Is there a way to install a massive amount of fonts, ie: over 500 ttf's? And just for kicks in, I need to save it into the folder for all users to have access to, not under my profile.
View 11 Replies
View Related
Mar 7, 2010
There are quite a few threads in this forum on how to convert a DJVU file into a PDF (among which this thread the instruction of which I followed.) I converted a DJVU file by printing it to a PS file from the Document Viewer. Then I printed again the PS to file into a PDF from that same application. The second phase took more than 24 hours and resulted in a scanned PDF of less than 500 pages with a size of 315 MB! I have many books of more pages which come down to 1 MB only. What happened in the process to make it so damn slow and so inefficient? Is there a good alternative (not ps2pdf because it provides the same result)?
View 1 Replies
View Related
Jun 3, 2011
How to do it from internet, with similar paths and regular file names? using regular expression? and massive rename or change directory?
View 1 Replies
View Related
Dec 24, 2010
After installing Opensuse 11.3 i get massive graphical errors after reboot. Those errors directly start after the first resolution change after grub. So i used nomodeset from now on and installed the drivers from nvidia.
After this X11 only flasht up very shortly and i got this logs:
Code:
View 9 Replies
View Related
Jul 6, 2010
I've ripped all my cd's in to MP3 using ripit, however, it seems as if ripit added 2 seconds to my mp3's, so how do I shave two seconds off each of my mp3's in one massive folder?
View 1 Replies
View Related
Nov 26, 2010
having some massive sound quality issues... my sound sounds so incredibly horrible.. it sounds like my speakers are blown out, all crackly and ******. at a loss because i do have sound, it just sounds horrible.
View 2 Replies
View Related
Jan 24, 2011
I hadn't updated until yesturday night, today after the massive update and reboot, my x stopped working. What happens is, after the splash screen, both screens goes black and unresponsive. Tried killing X and alt + ctrl + f1 etc. I can probably fix this myself if I could access the terminal but, the other day I had to be stylish and remove everything from my grub boot besides Windows 7 and the default Ubuntu so I can't boot into root terminal. So, how would I gain access to a console before X starts?
View 8 Replies
View Related
Feb 17, 2010
I have a program that is very heavily hitting the file system, reading and writing randomly to a set of working files. The files total several gigabytes in size, but I can spare the RAM to keep them all mostly in memory. The machines this program runs on are typically Ubuntu Linux boxes.
Is there a way to configure the file system to have a very very large cache, and even to cache writes so they hit the disk later? I understand the issues with power loss or such, and am prepared to accept that. Crashing aside, in normal operation the writes should eventually reach the disk!Or is there a way to create a RAM disk that writes-through to real disk?
View 5 Replies
View Related
Mar 21, 2010
The laptop I regularly use is out of use after having tried to upgrade from ext3 to ext4, following the instructions that can be found at namely, Step 2: Upgrading to Ext4Well, the first step was done without problems, but when I reached "Mount your filesystem", the problems started:When typingsudo mount -t ext4 /dev/XXXX /mntwhat appears is:UNEXPECTED INCONSISTENCY; RUN fsck MANUALLYStupid me, I did nothin but jump to the next step:gksu gedit /mnt/etc/fstabWhere I only got a fully white screen with no written text at all, instead of the menu that should have appeared, as can be seen on that webpage.
However, after running Gparted again, it showed the extension as EXT4, not EXT3, so I thought the system had automatically upgraded. Then I proceeded to boot the system, and this is where the massive failure appearedAfter seeing the Ubuntu logo for a while, the system directs me to a blank screen with the message:Mount of filesystem failed.dc5123-ec01-4438-8bd6-cb85bb080f87A maintenance shell will now be started.CONTROL-D will terminate this shell and re-try.root@hihihi-laptop:#So I type CTRL-D:
mountall start/startingfsck from until-linux-ng 2.16
/dev/sda1: One or more block group descriptor checksums are invalid. FIXED
/dev/sda1: Group descriptor 0 checksum is invalid
[code]....
View 9 Replies
View Related
Jun 30, 2010
I've had a password on my 10.04 installation since I installed it (when it was released) and since last week it has disappeared. My computer seems to log in automatically. When I check the login screen settings, it is set to "show the screen for choosing who will log in".
View 5 Replies
View Related
Nov 16, 2010
I have Ubuntu 10.10 installed at work on my Dell Optiplex 755, which runs perfect. So I decided to install Ubuntu on my home desktop, which so far has been a mistake. Booting up takes almost 5 minutes, after I login it takes about 2 minutes to get to my desktop, typing in any application (even a shell) is horribly lagged, almost to the point where the PC is unusable. I also get extreme lag when executing programs, or just simply clicking a submit button. It will take 2-10 seconds to actually click or execute. The weird part is visual effects extra works perfect and smooth.
CPU: i7 920
Ram 6Gb's
HD1: Veloci Raptor 300Gb 10k RPM
HD2: WD 1.5Tb (Storage)
Video: MSI N250GTS 1Gb
SmartD shows my Raptor healthy, here are the read test speeds.
Min Read Rate 7.6Mb/s
Max Read Rate 128.2 Mb/s
Avg Read Rate: 105.5 Mb/s
Avg Access Time: 8.6ms
Note: The graph has quite a few drop downs (as you can see the minimum is very low). I've also ran badblocks on the drive which came back with 0 errors. Do I need a special driver for my hard drive? I'm attempting to watch a movie on [URL] and it works fine as long as I continually move my mouse, when I stop moving the mouse the video gets jumpy.
View 4 Replies
View Related
