Programming :: Methods For Extracting Data Strings From Output Files?

Aug 23, 2010

I am trying to develop a method of reading files generated by other programs. I am trying to find the most versatile approach. I have been trying bash, and have been making good progress with sed, however I was wondering if there was a "standard" approach to this sort of thing. The main features I would like to implement concern reading finding strings based on various forms of context and storing them to variables and/or arrays. Here are the most general tasks:

a) Read the first word(or floating point) that comes after a given string (solved in another thread)

b) Read the nth line after a given string

c) Read all text between two given strings

d) Save the output of task a), task b) or task c) (above) into an array if the "given string(s)" is/are not unique.

e)Read text between two non-unique strings i.e. text between the nth occurrence of string1 and the mth occurrence of string2

As far as I can tell, those five scripts should be able to parse just about any text pattern. I am by no means fluent in these languages. But I could use a starting point. My main concern is speed. I intend to use these scripts in a program that reads and writes hundreds of input and output files--each with a different value of some parameter(s).

The files will most likely be no more than a few dozen lines, but I can think of some applications that could generate a few hundred lines. I have the input file generator down pretty well. Parsing the output is quite a bit trickier. And, of course, the option for parallelization will be very desirable for many practical applications.

View 14 Replies


ADVERTISEMENT

Programming :: Extracting URLs From Strings?

Jun 28, 2009

I have a PHP script written that is checking a string to see if it contains a link in it (i.e. a URL). I have the following if statement, that uses 3 possible regular expressions to determine if there is a link or not.

Code:
// check if we found a link
// links are denoted by strings that:
// - contain http://
// - contain www.*.*

[Code]....

I'm not convinced yet that writing a shell script to do this is the best course of action. If someone is capable of doing this with a Perl or a Python script that's fine too. If you want to make it super high performance and write it in assembly

View 1 Replies View Related

Programming :: C++: Extracting Data From A NESTED STL Map?

Feb 8, 2011

Code for insertion of data:

Code:

#include <map>
#include <iostream>
int main ()

[code]....

I don't know how to fetch the data from the nested map here.

View 5 Replies View Related

Debian Programming :: Emacs And ECB Missing Python Methods In Methods Window

May 12, 2014

I work with python and I use emacs as my IDE tool. I have been running Debian Squeeze (6.0.9) for some time now with emacs 23.2.1 and ecb 2.32. I am able to access my python methods in the ecb-methods window with no problems. However I recently upgraded my desktop to Debian Wheezy (7.5) running emacs 23.4.1 and ecb 2.40 but I have lost access to the methods in the ecb-methods window. The window is just empty while the others (directories, sources and history) are all populated. I have a second laptop which I decided to upgrade to Debian Jessie, however Jessie recommends emacs 23.4.1 which is running with ecb 2.40 also. The result is the same as on Wheezy.

I have used the ecb menus and googled for a solution or even just a mention that such a problem exists but have come up with nothing. Either I have a unique situation here or am doing something really dumb.

I would like to upgrade to Wheezy or Jessie but I need access to methods in the ecb methods window. How to keep my upgrade and see the methods in the methods window of the ecb system ....

View 0 Replies View Related

General :: Searching And Replacing Strings In File With Strings In Other Files

Jul 29, 2010

I want to search and replace strings in a file with strings in other files/i need to do it with big strings(string1 is big) and i want to use a txt file for this.But this code not working :

View 14 Replies View Related

Programming :: Output Data To An Ascii Table?

Oct 18, 2010

I've got lines of data in the following format:
space1=number of times error has occured
space2=IP address
space3=Error

I've set this out nicely with printf and made it email me, the problem is - it's not entirely clear what each column/space is and the IP and occurances can sometimes seem confusing. Is there any (easy) way to output this into an ascii like table? There will always be 5 occurances, and the format will always be the same

View 1 Replies View Related

Programming :: Retrieving And Organizing Data From Command Output?

Mar 31, 2011

I was messing around with Bash scripting just now and was wondering if there was a way to organize the output of a command into an array. Like the Bash equivalent of the PHP explode() function.

View 3 Replies View Related

OpenSUSE Install :: Copy Data From The Straight Disk To The Array, Using Several Methods: With Dolphin, With Cpio?

Jan 9, 2010

I'm not sure wether this is the correct forum for this, but this is the best place I can see at the moment, so I'll give it a try. Please redirect me if I'm mistaken.Running Suse 11.2, I have a RAID-5 device mounted, and a straigt disk. I want to copy data from the straigt disk to the array, using several methods: with Dolphin, with cpio. Copying runs for some time, sometimes one or some files are copied indeed, but after a short time (sometimes half a minute, sometimes 10 minutes or more) I get a

Message from syslogd@linux-wrth at Jan 9 22:44:03 ...
kernel:[ 381.602651] Oops: 0000 [#1] PREEMPT SMP
Message from syslogd@linux-wrth at Jan 9 22:44:03 ...

[code]....

View 2 Replies View Related

Ubuntu :: AWK And Extracting Data From Hyperlinks

Mar 29, 2011

I have come across a webpage where there are several (thousands of hyperlinks). Of those hyperlinks, many are named "CDS" and in each of these CDS hyperlink- there are certain lines which i need to chop off!! I kind of have this feeling that AWK would be the one to use.

View 9 Replies View Related

Red Hat / Fedora :: Extracting Data From A Xml File Using C?

Feb 11, 2010

I am trying to extract data from the xml file using c coding. here is the sample code. this code gets compiled with errorfree. but the call back functions start and end are not invoked. coding:

#include<stdio.h>
#include<string.h>
#include<expat.h>

[code]...

View 1 Replies View Related

Software :: Extracting Cache Data From FF 4.0?

May 27, 2011

well probably most of you have switched to FF 4.0

my major problem with it however , is the way the cache is stored on the hdd , now everything is sort of "sorted" into an endless number of folders.

i'm wondering why did they do it that way ? how are we supposed to browse the cache folders from now?

btw i've always resorted to the cache folder for extracting all kinds of things the browser downloads..

View 7 Replies View Related

Debian :: Mounting .iso Image Versus Extracting Data From .img File?

Feb 28, 2010

I use Lenny, and was trying to mount a .iso image, supposedly a cd imagem.

[code]....

This is what I get from dmesg | tail:

debian:/home/zac/cscd# dmesg | tail
[ 1811.505199] floppy0: disk absent or changed during operation
[ 1811.505207] end_request: I/O error, dev fd0, sector 0

[code]....

I did a little research on the web and it seems that this file is not really a cd image, but simply data in a .img file. What do you think of that?

debian:/home/zac/cscd# file cscd3.iso
cscd3.iso: data

Some people recommend to extract the data via the dd command, but it didn't seem very safe for me to do that!

[URL]

is it possible to extract the data into a directory (instead of a device) using dd? This file is supposed to be a software. I wanted to run it on wine by keeping it mounted on a mount point in my file system. Does it make any sense to try to do this if the file simply isn't a cd image?

View 6 Replies View Related

Programming :: Invoke Methods Via Ssh On Servers From CSV?

Jan 14, 2011

I have to execute certain commands (like shutdown Tomcat) on several servers so I'm using a loop and ssh. I put the server's IP on a CSV file which I parse, execute the commands for each line and send the output to a file. The problem is that after processing one line the program stops execution. I wonder if someone could lend me a hand with this, I'm new in bash scripting and I ran out of ideas.

The CSV (servers.csv) file looks like this:

Code:

192.168.254.5:Server 1
192.168.254.6:Server 2
...

And the script looks like this:

Code:

#!/bin/bash
while IFS=: read ip name
do
sshpass -p "pass" ssh -o "StrictHostKeyChecking no" root@"$ip" 'sh <CATALINA_BASE>/bin/shutdown.sh' >> output.log
done < servers.csv

View 1 Replies View Related

Programming :: Pipe Output To 2 Files But Not STDOUT?

Mar 24, 2011

I want to have the output of a program go to 2 different files but not going to standard out. Is there a way to do this in bash? I know that in Z shell its really easy. omething like: Code: echo "test" >> file1 >> file2 Would work. But in Bash it doesn't seem that easy. I know that tee will send the output to 2 files but it also sends it to STDOUT.Something like:Code: echo "test" | tee -a file1 file2 Would put the word "test" in file1, file2, and STDOUT. Is there a way to just send the output to file1 and file2?

View 2 Replies View Related

Programming :: Formated Output Of All The Files Under A Particular Directory?

Jul 7, 2010

I wanted formated output of all the files under a particular directory. I am trying to use find.Something like find -P ./ -type f -name '*.cpp' -printf "%p "I want all the files with specific extension like .c .cpp .h to be printed out separated by space. One more thing i want is absolute path names instead of relative.

View 5 Replies View Related

Programming :: Redirect Output To Files Using Shell?

May 21, 2011

I have a huge database of students, I would like extract these data and write to individual file for each students.

I am running a loop in shell program (.sh file), the output of each run in the loop need to redirected to a file with variable name.

I tried the following line, but it did not work, where BodyMsg is the data and Rollno is the students roll number.

echo $BodyMsg > $RolNo".html"

View 5 Replies View Related

Programming :: Bash Scripting - Output As Multiple Files?

Jan 26, 2011

I have wrote a 1 line command that parses a file, locates the IP Address in the file and then trims the output the way I want it, and then sorts numerically and by uniqueness and then >> appends to output.txt

I can get all the IP's into 1 file "output.txt", but what I am really looking for is some type of way to create a text file, for each IP it finds labeled xxx.xxx.xxx.xxx.txt and also put that ip address into that file..

xxx.xxx.xxx.xxx = the ip address it finds

View 14 Replies View Related

Programming :: Redirect The Output To Multiple Files Without Displaying It To The Screen?

Oct 26, 2010

To redirect standard output to multiple files:

Code:

echo Test | tee file1 file2

My problem is that the word "Test" still displays to the screen? I want same effect as:

Code:

echo Test > file1

but with multiple file redirection.

View 3 Replies View Related

Programming :: Need To Copy The Obtained Output Files Inside Corresponding Folders?

Nov 2, 2010

I have encountered a problem:I have "while" loop; at each run a set of outputs is produced but then I need to shift them into a corresponding folder ; otherwise next run the new outputs will be over-written. Furthermore, I need to pipe what I have on the screen inside a file. I have put my code in the following:

Code:
# !/bin/bash
jf="GeoQuery.jar"

[code]...

View 3 Replies View Related

Programming :: Reading Data From Multiple Files?

Mar 17, 2011

I have a folder which includes bunch of folders each having data files in them. [ Folder A has F1, F2 F3 ..... F1000 folders in it, and F1, F2, F3 ... each has about 10 different files named FILE 1, FILE2, FILE3 .... in them.

I am interested in File 1 of each Folder, because that contains the data I need in it. More specifically, that File1 s have a line "ANSWER=..." in them, and i need to get that value of the ANSWER from each file. So doing it by hand is so hard, so I need to write a script that will scan all folders and give me a list of values of eache ANSWERs.

View 4 Replies View Related

Programming :: Convert The XML Output Files To SQL In Order To Create Or Update An SQL Database?

Dec 7, 2010

I've a program which manages my pdf and references. I wish to put some of the information on my website but that program (Mendeley) does export only in XML (or bibtex). I'd like to simply convert the XML output files to SQL in order to create or update an SQL database.I'm not an expert in either XML or SQL (use only PHPMyadmin). Does someone get help me to figure out?

View 2 Replies View Related

Programming :: Have Files With Rows Of Data And Need To Check If The Next Row Of The Same Type?

Aug 16, 2010

I have a problem - I have files with rows of data and I need to check if the next row (of the same type) has the NEXT date in it so I need to extract a date in YYYYMMDD format from a row (easy enough) then add one day to it and compare it to the the next date I encounter on a subsequent row.

So if I have
DATE|20100530
VAL|kahskjhd

[code]...

View 6 Replies View Related

Programming :: Using Sed To Change Strings?

Sep 3, 2010

I am having a lot of problems trying to change one string by another using sed: the sentence is like this:

sed -i 's/KERNEL=="tty[A-Z]*", NAME="%k", GROUP="uucp", MODE="0660"/KERNEL=="tty[A-Z]*", NAME="%k", GROUP="uucp", MODE="0666"/g' 50-udev.rules
it is just to fing the line with:
KERNEL=="tty[A-Z]*", NAME="%k", GROUP="uucp", MODE="0660"

[code]....

View 8 Replies View Related

Programming :: C Truncating Strings

Dec 11, 2009

I wrote this small program that will truncate a string that's entered in by the user.An example of its usage:if the user enters in a string say "abcdefghijklmnopqrstuvwxyz" the program will only take the first 9 characters and truncate the rest so that the user can be prompted for a second string and not be worried about remaining characters left in the stream.Now this program works O.K. but I would like to find something in C that has this functionality build into it...Does anyone know of any function that will accomplish this.

View 1 Replies View Related

Programming :: Replace Txt Between 2 Strings?

Aug 4, 2010

I am trying to replace a section of a file between the first instances of the strings {}, with the contents of another file. Example of the format of the file I'm trying to modify

Code:

Servername=something.com
hosts {
macaddress1
macaddress2

[code].....

Then captured all the "macaddress#"s to a variable and used sed to swap

sed "s/$CURRENTDATA/$NEWDATA/" filename

However I get 1 of 2 errors,

Using a small number of macs in "$NEWDATA"

sed: command garbled: s/ macaddresshere

Or when using a large number of macs in the $NEWDATA variable get

bash: /usr/bin/sed: Arg list too long.

how to replace a large block of txt in one file with another large block from another?

View 6 Replies View Related

Debian Programming :: Sh Command Strings

Feb 8, 2014

I use udhcp with some of my minimal installs. I've messed around with the code a bit when it wasn't working correctly - a few years ago. I will find time - I hope soonish - to figure out how to do a few other things with it.

For now though, I'm using this string to grab my ip after startup

Code: Select allsu -c 'ifconfig eth0&&ifdown -a&&ifup lo&&ifup eth0'&&/sbin/ifconfig eth0

I realize I could substitute ifup -a but I'm more interested in figuring out how to make ifup wait for the ip to become available if it is not available yet.

Never mind that one, just typing out the question answered it for me, when I find it in the scripting man ' ; : " & =

Or if there are any other suggestions for better construction of the string.

View 2 Replies View Related

Programming :: Best Command To Use To Parse Strings?

Mar 24, 2010

what is the best command to use to parse strings?I have a variable $str and need to parse this string.Can you provide an example of the command used to get a substring of $str based on the index values of start and end

View 3 Replies View Related

Programming :: Cannot Match Strings That Appear To Be Identical

Feb 25, 2011

i have an sql table with 2 columns i run a script that randomly selects a word from the table in column 1.
the word is displayed on the screen and I guess what it means i concatenate the randomly selected word and the answer the script looks for a match in mysql if it finds a match it says "Good job!" if there is no match it will say "not correct". However when i get it right it says not correct even though when i echo the variables they look exactly the same. the script below:

#!/bin/bash
var=$(mysql translator -u root --password=*-N<<EOF
SELECT word FROM tagalog ORDER BY RAND() LIMIT 1
EOF
)

[Code]....

View 4 Replies View Related

Programming :: Get The Strings That Matched Sub-expressions?

Jun 8, 2010

Using the regex.h functions, how do you:

1) get the matched strings?

2) get the strings that matched sub-expressions?

View 4 Replies View Related

Programming :: 'strings' Binary From Binutils - How To Use It

Jul 14, 2010

it is about 'strings' binary from binutils. I recently wanted to test what it does and how it works. The best I did was this:

Code:

#!/bin/sh
PROGRAM="
int main()
{

[code]....

Ok, from the output I don't get it.Can someone show me in a simple script, what is 'strings' good for ?

View 10 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved