Programming :: Bash Script To Count Number Of Lines With A Specific Property7?
Aug 11, 2010
I would like to parse an input file in which there are two columns per each row. We want to see how many lines are duplicated where we define duplicate to be having the same second field and different first field. For instance if the input file looks like the following:
I recently found myself in possession of a large file (a few million lines in length) of short strings and would like to count the number of lines that are unique to the file. I thought this would be an easy process, but while working on the problem, I encountered the following. Can anyone explain this weird result to me?
I need to create a script to count the number of lines from a text file . The output must be put on another text file (no_lines.txt) and in this file i need to generate from the script this output :"File $FILE has $NO_LINES lines ".
i need to count the number of files and put the output into a variable. i used wc -l filename but i couldnt find an option to put the output to variable. example if the number o line is 5, i need the output of echo $x is 5.
I need to search a bunch of files in a specific folder for a specific number and add all the numbers together to a total sum. I use Rsync everyday, everytime I run rsync i get a logfile (rsync output) witch contains the textstring "Total bytes sent: xxxxxx".
The "xxxxx" can vary in lenght. I need to extract the "xxxxxx" from each file and add the numbers together to a total size over a week or a month. Is this possible? And I wish to only use bash. One way of doing stuff at a time my friends .
And I'm trying to count the number of slashes in each line. I figured (with my limited knowledge of bash) that the best thing to use would be sed. So I ran this to print "not /": sed '!s////g' file # and eventually adding " | wc -m" to it. and I got the same result as if I ran cat, no modification at all:
I am trying to get the count of number of CD-ROMs attached with my Linux system using a bash shell script. I have decided to use the following method for it:
What I want to do is from a file having block like
<event> 8 3 0.2685416E-02 2 -1 0 21 -1 0
[code]...
The first line after the "<event>" is its process-id, so I would like to have at the end a summary of how many "event" block I have for each type, ie how many
6 1 0.2685416E-02
or how many
7 2 0.2685416E-02
etc etc
I do not know in advance how many different-kind of block I will have, so it has to be a bit smart to scan the file, and make an new "summary" info for each unique type I was using something like
I would like to know how do I print the line # in a script. My requirement is, I have a script which is about ~5000 lines long. If there are any errors happen I just exit. And I would like to add the line # of the script where the error happened.
Unfortunately, the second grep is greedy swallowing everything up to the last </ul> close tag. (The desired result is 2.) Speed is an issue as I will be searching through 350,000 files.
I'm working on a bash script that will go through a directory, find the sub-directories that have been created since the last time the script ran, count the results, and output that integer (will most likely be '1' or less per each instance run) to a file. Give the circumstances, my previous (and very limited) experience with bash is not sufficient for me to pull this off. since it probably has bearing, is that my mail server stores files that it flags as viruses in a folder. It creates a sub-directory for each virus that it quarantines .I want to count those subdirectories and graph them with MRTG. Hence the script. I'm going to post what I've got so far and the purpose of it, because I'm told I have a very odd and efficient way of doing scripting.
[Code]...
But then it dawned on me that it wouldn't work because I would have to not count the directories that have already been counted and count the ones that have not been counted. Given that the purpose of this is to generate a graph about every 5 minutes, using find won't work because, to my knowledge, that will only find things based on whole day values, I need it almost down to the minute.
To save on the writing of WAY to many files with very little in them, I want to put it all in one file and read a specific few lines. There will be six variables to be read at a time. Format is as such:
//Set 1 string name 5 12
[code]....
From name to 5th number is a set. The name will be of different lengths for each set. This will be a big file of probably 40+ sets. My problem lies in reading one and only one set be it set 5 or set 34. It needs to be done in C++.
I'm trying to come up with ideas for a simple way to strip a specific "entry" from a text file.I know tools like sed and perl can remove specific lines from a file but I haven't been able to come up with an elegant way to do my group of lines.In my file, the first "Location" line and the "SVNPath" line should be unique every time... but are they enough to strip out the whole set of the group plus the trailing one line of white space separating each group? Add to this, my file will grow as new entries are added (always appended to the end) but new entries will have the same formatting.
I'm trying to extract specific lines from a flat file. I need lines that fall within a range of coordinates. The -F can be either ! or = If the line is in this set range I need all of the data on that line. ranges lat 36 to 39 and longitude -74 to -84
I am trying to create a socket to listen for a bootp response so I am using a PF_PACKET socket so that I get the response based on my mac. My problem is that I don't want to hear all traffic (as I do now) so would like to use a specific port number and bind to it.
I have a file witch I need to list 10 line by 10 lines with something like press enter to go on in between. Well, the problem is that i have absolutely no idea on how to implement this.
Using awk I pull the first field of a random line from my datafile.myvar1=`awk -F" " 'NR=='$randline' {printf "%s", $1}' myfileThis works fine. The problem is there will be empty lines at the end of the file. Rather than using awkto filter out blank lines I would like to figure this out first.So I test $myvar1 for a blank string after setting $randline to one that I know is blank:test -z "$myvar1" && echo "true" || echo "false"But, this returns "false"? So the string is not zero length. Why? It's a tab-separated file. Is awk storing the tab with the $1 field or something.This is where I get headache. I try to echo my variable to see what it looks like.
echo "$myvar1" outputs: nothing echo "My variable is [$myvar1]" outputs: [y variable is [
Why is the closing bracket at the beginning? What character could be stored in $myvar1 that would do such a thing and how did it get there?
I would like to know how I can get the ouput from the following dmidecode command in example 1 to look like example 2 without having to grep -v all the unwanted lines.Is there a way in awk or sed?Example 1
Code: Processor Information Socket Designation: Socket 1 CPU 1
I'm trying to search through some pdf files and I'm doing so by converting them to text files using pdftotext which is fine but I'm trying to get the number of occurrences in a paragraph of different words and it's adding a new line character at what it thinks is the right hand margin. I'm trying to remove all these singe new line characters but keep the doubles and I can't seem to work it out. i.e.
This is some text that has been broken. Another paragraph. becomes This is some text that has been broken. Another paragraph
#!/usr/bin/perl use DBI; my ($db, $user, $pw) = ('dbname', '****', '***********'); my $dbh = DBI->connect("DBI:mysql:$db",$user,$pw) or die "Cannot connect to $db: $DBI::errstr
[code].....
The error message is
[Wed Feb 24 13:03:27 2010] myscript.cgi: DBD::mysql::st execute failed: Column count doesn't match value count at row 1 at myscript.cgi. [Wed Feb 24 13:03:27 2010] myscript.cgi: DBI::db=HASH(0x8a30c60)->errstr
So I need to write a bash script that can read lines and column 3 from a file. It needs to start on line 16 and read every 20th line starting from there. But the value that it reads needs to be checked, should it be too great I need it to shut the program down.I'm pretty new to bash, but my ultimate goal is being able to safely run a program on a GPU for an extended period of time with out worrying about it overheating. I have a command that outputs information from the GPU every second, and I can save this to a file. So all I really need is something to read and check that file, I played around a bit with the awk command and can't get it to work within my for loop with dynamic variable.