Ubuntu :: Screen Data Scraping Into A Database?
Feb 18, 2011
I use ubuntu lucid 10.4. I would like to monitor stock prices in realtime with a web browser and generate spreadsheets. Is there a program to scrape the data from the screen and put it into an openoffice spreadsheet so that I can categorize the data appropriately? I would like to do this in realtime if possible. Perhaps a firefox addon or chrome extension. I have searched for this for a while and think I may have to write my own code, which would be okay, but I imagine that there is some open source code out there that would work or can be changed a little to suit my needs.
View 3 Replies
ADVERTISEMENT
Jul 3, 2011
so I wrote a simple script to scrape whatsmyip.org for my public IP address and then email it to me if it has changed. I set this up as a cron job ever 30mins.the problem is, it works for about 18 hours and then will have an issue where it says "line 33: [: too many arguments" Which line 33 happens to be my if statement. Now like I said it works fine for about 18 hours, i run the script with 2>&1 to a .out file so i can see what its doing...this is the output of the .out file, the "xxx" does actually show the ip.."PHP Code:
xxx.xxx.xxx.xxx
already in file
xxx.xxx.xxx.xxx
[code]....
View 14 Replies
View Related
May 29, 2011
Please tell me spesification if i want buid mail server on ubuntu 10.04 with good features :
1. For MTA ??
2. For Mail Filtering ??
3. For MDA ??
3. For Webmail ??
4. For Mailling List ??
View 3 Replies
View Related
May 29, 2010
I have a LAMP server. I know sql but I'm currently learning PHP.
Correct me if I'm wrong, but you're supposed to use PHP to insert/select data from the database, right? If so, SQL shouldn't need a port at all.
Apache uses port 80, but it's listening for connections. MySQL uses port 3306, but I've never logged into it from anywhere besides 127.0.0.1/localhost. I have mysqladmin installed and it demands that I give it an IP so I used localhost.
I'm not interested in doing anything remote, so can I block this port? When I say "block" I mean on the router, not locally through iptables. Obviously I have to foward port 80 because that's how people would connect to my server. But once they connect and send something, my server would interpret the data locally and send it back on port 80.
View 8 Replies
View Related
Feb 19, 2011
We purchased a new database system at work last October, ditching the old system because of a lack of support from the vendor. This is a retail Point of Sale and Backoffice database system. I am not sure what system the new one runs on, but the system we replaced was a Firebird data base. The reason I am posting is because we are now in need of the information contained in the old database which was not completely imported into the new system. Unfortunately, we parted company with the old vendor under extremely hostile conditions, and they are not a source of help for this problem, nor are the new vendors.
Basically the problem is this: The database in on a Windows XP system and I found a copy of SQL Manager Lite 2008 on the system, which after quite a bit of studying, I figured out how to extract the database into a removable file. I have this file (178MB) on a USB stick in a file called Backoffice.fbd. I can get into this database with MySQL. All I want to be able to do get into the database and create tab deliminated spreadsheet files for each of the database sections (Customers, Repairs, Sales History, stock files, etc.) Is it possible to do this with Ubuntu and MySQL and if so, can an expert suggest one or two things to get me started.
View 3 Replies
View Related
Jun 12, 2010
I want to download some specific information(age of the building, value, square feet, etc.) from a county auditor's website:[url]
For every property in the county and ultimately match it to a .dbf file I already have containing geographic information for each parcel.
The website only displays information for a single address at a time in response to a query form asking for either the parcel ID# or street address. I have a table of all the county's parcel ID#'s and valid addresses-about 350,000 or so of them.
I imagine there must be some way to automatically crawl the website inputting the information from my tables into the form and gathering specific information in some sort of format that could then be matched to corresponding information in my .dbf file.
View 1 Replies
View Related
Mar 4, 2010
I have try to sync my MySQL Server database and HLDS Data on lan,one is windows server 2008 and one is Ubuntu Linux 9.10 i have try to use the remote address(192.168.0.4:3306) but can't connect and say the error code is 10060 i have check the connect is normal and ok,the accont is can let any address to contro.
View 1 Replies
View Related
Jul 19, 2010
I have set up a MySQL database which has only three fields for user-name, password and index number. But there is a LOT of data (thousands of columns) which would take too long to enter manually. I'm sure MySQL must have a file format whereby a single large file containing different data types can be successfully automatically imported and the various data elements distributed to their intended fields. Does anyone know which characters MySQL uses as field seperators for this formatting of data, or is there a program already written which can take raw data and organize it into a SQL compatible file?
View 8 Replies
View Related
Apr 13, 2011
I have a need of a database, and the information to go into it. I know how I want to organise my data for optimal cross referencing, but what I know is merely what I know and I do not know if what I know is what is known...
Madness to say... looking at my database I see this is the best way to organise the data, but what I do not know is how the computer will respond to my methodical madness.
Is it quicker to have one table with many fields or many tables with few fields?
We've got information for 10 users with 17 idiosyncrasies.
So, would one table with 17 columns and 10 rows... or... a group of 50 tables with one column and 3 rows and one table with 4 columns and 5 rows...
Be more efficient concerning:
QUERIES...?
Searching?
Comparison?
Insertion?
Deletion?
Modification?
The queries and modifications will be daily, and constant. These numbers are really much larger but the math is easier for me to articulate on a smaller scale. I use Perl, php, and C with SQL for management.
View 14 Replies
View Related
Aug 26, 2010
I have a CSV called source1 with data like:
Date12:00:00 AM12:30:00 AM01:00:00 AM01:30:00 AM02:00:00 AM02:30:00 AM03:00:00 AM ...
07/31/1051.5252.4852.6452.863.3663.8445.6 ...
08/01/1049.4449.9261.2860.1652.6449.1247.52 ...
08/02/1049.1248.9648.9650.7260.1658.0851.04 ...
08/03/1060.3260.6459.260.9637.9233.2832.8 ...
[Code].....
Also, the CSV file is updates every few hours and I need to load any new data from the file to the database without creating duplicates of data that has already been loaded to the database.
View 6 Replies
View Related
Jun 22, 2015
This is a continuation of the post in [URL].... . What has happened is that I have a huge log of 600 MB + and do not know from when the data is in there.
Code: Select all┌─[shirish@debian] - [~/.config/quassel-irc.org] - [9714]
└─[$] ll -h quassel-storage.sqlite                                                 Â
-rw------- 1 shirish shirish 647M Jun 22 21:15 quassel-storage.sqlite
So I need to know if there is a way to get just the first five or ten entries listed as one can do using head, tail etc. I did try just using head but it was unsuccessful. The second thing would be deleting some days, weeks or months data from the existing sqlite log. I did see URL... but in my case I am deleting old entries in the sqlite log and also do not know if sqlite makes backups of the data or not ?
View 4 Replies
View Related
Mar 7, 2011
am using zabbix open source solution for systems monitoring. I am facing a problem and discussed it on zabbix forum. my post was as "My zabbix server is behaving abnormally, approximately daily from 9 to 12, the server stop accumulating logs. I observed that the server report is RUNNING but it did not accumulate log values and also the machine have no extra load. Its shown in the graph image attached.t the following reply,"database performance?are you monitoring database IO and available database threads? "So any one have any idea that how can I do this as I am using MySQL as backend database on RHEL 3.
View 1 Replies
View Related
Jan 5, 2011
I am writing a simple application that will run as user foo (i.e. Ubuntu user foo).However, the application will connect to my database as (database user foobar). IIRC, database users have nothing to do with Linux system users - but I just need to clarify that.So can an app launched to run as user 'foo', connect to a database as user 'foobar'?
View 1 Replies
View Related
Jul 8, 2011
I recently moved into a new place and when I hooked up my webserver, I wasn't able to bring up my page, even from localhost. With some digging, it seems that I can't access the database that housed my posts (wordpress installation). I looked for the datadir in MySQL and that directory shows the wordpress directory that should be holding the database and all the files are still there. 1) why the database no longer shows up 2) how to restore the database from the files?
View 2 Replies
View Related
Apr 4, 2011
Adding several 2TB HDDs somehow crashed SATA controller on motherboard.Fedora will no longer boot.
I connected HDD (with Fedora OS) to another computer.Fedora gets past GRUB, but hangs before loading the splash screen.Is something wrong with GRUB settings?Previously, I had dual boot at GRUB (Fedora and WindowsXP).
What is the easiest way to get Fedora to boot properly on another system?I need access to those files.
View 14 Replies
View Related
Jun 16, 2011
I've setup a CentOS apache web server, and loaded a site onto it. The site previously was functioning under windows xp (before I "stepped in") and wamp. What I did was copy the site in /var/www, reconfigure the httpd.conf (DocumentRoot /var/www/site, etc). Although when I try to open the webpage I get this error: Database Error: Unable to connect to the database:The MySQL adapter "mysql" is not available.
[code]...
View 10 Replies
View Related
Mar 25, 2010
I've a problem which occurs during the boot process.I'll turn on the machine and Slackware (13) will begin loading. I'll get the "BIOS data check successful message" and then the screen will go black. Nothing happens after that. This doesn't happen every time I boot and other than that, Slackware works fine.I'm guessing it's the video card. Uname and lspci output to follow.
uname -r: 2.6.29.6-smp
lspci -vv:
00:00.0 Host bridge: Intel Corporation 82845G/GL[Brookdale-G]/GE/PE DRAM
[code]...
View 4 Replies
View Related
Jan 17, 2011
I've been using Excel from OpenOffice with moderate success for the last year. Now I've just tried to open a new spreadsheet and I'm suddenly getting a highly condensed view. The cells are about 1mm X 4mm. If I expand them by selecting and dragging, then try to copy data from another spreadsheet onto the new one, the data shrinks to the small scale -- unreadable.
View 4 Replies
View Related
Jun 3, 2010
Ubuntu 9.10 64bit. Acer Laptop
Using ubuntu 9.10 cd to access laptops hard drive.
foun readme file in /media/disk/mark
opened with openoffice. THIS DIRECTORY HAS BEEN UNMOUNTED TO PROTECT YOUR DATA.
From the graphical desktop, click on: "Access Your Private Data"
or
From the command line, run: $ ecryptfs-mount-private
So I tried a couple of ways...
View 9 Replies
View Related
Jul 27, 2011
Right now I have a 320GB system drive and 3TB data drive. I want to add two more 3TB drives and do a software RAID5 3x3TB. Is that possible without losing the data that is already on the data drive?Just want to make sure before I bought the 2 two drives. Not looking for instructions on how to do it,but if you want to include some that would be great too Just making sure it will work.
View 1 Replies
View Related
Feb 19, 2011
We purchased a new database system at work last October, ditching the old system because of a lack of support from the vendor. This is a retail Point of Sale and Backoffice database system. I am not sure what system the new one runs on, but the system we replaced was a Firebird data base. The reason I am posting is because we are now in need of the information contained in the old database which was not completely imported into the new system.
Basically the problem is this: The database in on a Windows XP system and I found a copy of SQL Manager Lite 2008 on the system, which after quite a bit of studying, I figured out how to extract the database into a removable file. I have this file (178MB) on a USB stick in a file called Backoffice.fbd.
My studying suggests to me that I can get into this database with MySQL. I have never used this and have no clue how to do this. All I want to be able to do get into the database and create tab deliminated spreadsheet files for each of the database sections (Customers, Repairs, Sales History, stock files, etc.) Is it possible to do this with Ubuntu and MySQL and if so, can expert suggest one or two things to get me started. While a guided tutorial would be nice because I am not an expert, I am willing to learn on my own if someone could point me in the right direction.
View 3 Replies
View Related
Oct 6, 2010
Where apart from ~/.kde4/share/apps/kmail does kmail keep account data, such as login data/names of pop/smtp servers? I wanted to install an entirely clean version of kmail, so I uninstalled/re-installed kmail via YAST. I manually deleted ~/.kde4/share/apps/kmail. After installing kmail I found that the account data of my Yahoo account were still listed in the kmail configuration panel. Shall I delete the Yahoo account within kmail prior to uninstalling the kmail package?
View 4 Replies
View Related
Jul 14, 2010
A few months ago I have setup a server with three hard disks. The partition mapping the disks as follows:
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x7ca36fee
[code]....
Now I have the following problem the LVM file system don't mount properly.If I open the mount point I see only a few files of the LVM disk. If I want to unmount the disk I get the following error:
umount /data/
umount: /data/: not mounted
If I want to mount the volume I get the following error:
mount -a
mount: /dev/mapper/gegevens-Data already mounted or /data busy
View 3 Replies
View Related
Jan 23, 2010
I'm trying to connect to the VPN of my employer; after fix various minor issues I reach this point in which the DNS entries and the default gateway of the VPN are overwritten with the values of the eth0 device that appears by default. Therefore the vpn is not useful.
View 1 Replies
View Related
Mar 5, 2010
I have an application where I am sending data via serial port from PC1 (Java App) and reading that data in PC2 (C++ App). The problem that I am facing is that my PC2 (C++ App) is not able to read complete data sent by PC1 i.e. from my PC1 I am sending 190 bytes but PC2 is able to read close to 140 bytes though I am trying to read in a loop.Below is code snippet of my C++ AppOpen the connection to serial port
Code:
serialfd = open( serialPortName.c_str(), O_RDWR | O_NOCTTY | O_NDELAY);
if (serialfd == -1)
[code]...
View 5 Replies
View Related
Jun 11, 2009
I am using read() in c++ to get data from a serial port. However, if no data is available on the serial port the function blocks until dta arrives.Example code:
//------------------------------------------------------------
char m_readBuffer[255] = {0};
char* p_curChar = m_readBuffer;
[code]...
View 1 Replies
View Related
May 9, 2010
I have some data files that should be distributed with my program. Using dist_pkgdata_DATA in Makefile.am, I get these files installed to /usr/local/data/share/package-name. The problem is that data is read-only, and my program needs to modify it. Playing with dist_sharedstate_DATA, dist_localstate_DATA, dist-data_DATA varibles, I got different installation directories, like /usr/local/com, usr/local/var, but data is always read-only.
How can I distribute modifiable data files with my package? I need some common directory for all users, or maybe local data in a user directory.
View 1 Replies
View Related
May 12, 2010
I can't seem to get Ubuntu to run on my old Dell Dimension 2300. It boots to a purple screen then it goes to a black screen with a load of writing nothing else happens after this screen. [URL] Could someone advise how I can get it to work?
View 1 Replies
View Related
Mar 4, 2011
I am trying to generic way to convert the string datatype to other primitive data type. To achieve, i used Template . But i getting error and couldn't resolve the issue and error reported is also clueless.
Code
====
#include <vector>
#include <iostream>
#include <string>
[Code].....
View 7 Replies
View Related
Aug 8, 2010
About 7 years ago I setup a Redhat system with 8 serial terminals and 8 serial printers, this is still running but I need to get another system running so the original can be taken down for maintenance. The printers are ancient Newbury data dot matrix wide carriage with 7 data bits receive 8 data bits transmit Xon/Xoff flow control. Now I remember setting the transmit and receive data bits as above but after 7 years can't remember where.
View 2 Replies
View Related
