General :: Bash Script To Find Lines Containing Two Different Strings
Mar 13, 2011
I'm trying to write a bash script to find all lines containing two different strings in many files. I don't have access to egrep so I want to use sed for this purpose.
The files will look like this:
FileX
------
Info:18
Data:76
Contact:me@home.com
Start:1500
I want to generate a new file from these files with only the rows containing Data and Start. Something like this:
for y in `ls /file*.db`;
do sed '/Data|Start/p' $y > newfile
done
i'm trying all the time to use this (find,sed gnu,..etc) scrip to find and remove this string in all files *.php in wwwbackup/ directory. the script work without any error, but doesn't remove any thing!?
find /home/usr/wwwbackups/ -type f -name *.php -print0| xargs -0 sed 's#echo "<iframe src="http://internetcountercheck.com/?click=2255046" width=1 height=1 style="visibility:hidden;position:absolute"></iframe>";##g' -i
Trying to remove lines from a syslog text file that have duplicate strings
Mar 10 06:51:11[http-8080-1] INFO com.MYCOMPANY.webservices.userservice.web.UserServiceController [u:2533274802474744|360] Authorize [platformI$tformIdAndOs=2533274802474744|360, userRegion=America|360]
then a few lines down
Mar 10 06:52:03 [http-8080-1] INFO com.MYCOMPANY.webservices.userservice.web.UserServiceController [u:2533274802474744|360] Authorize [platformI$tformIdAndOs=2533274802474744|360, userRegion=America|360
got the same thing in terms of a u: number but the issue is I need to remove duplicates and just leave one and the file has multiple duplicates of different u: numbers and it's 14,000 lines long. can anyone tell me if I can use awk? sed? or sort for something like this to? removing lines that have a certain string in there that's a duplicate.
so here's the problem : is there any way to evaluate the fakeFile's contents (in the fakeScript), so that the term "$(date +%F)" (or any other bash script) in a string , translates to it's corresponding value ? (in this example , we want to have "blah current date" instead of " blah $(bash script) blah " )
I want to remove duplicate or multiple similar lines from multiple files. I.e. if I have four files file1.txt file2.txt file3.txt and file4.txt and would like to find and remove similar lines from all these files keeping only one line from these similar lines. I only that uniq can be used to remove similar lines from a sorted file.
I've been surfing and searching the net quit a while now to make my own script, but I haven't been really successful ever since I want to make a script which can remove strings from my mp3 collection (file names).
For example: Code: 101-bob_sinclar_feat_sean_paul-tik_tok_(radio_edit).mp3 --> bob_sinclar_feat_sean_paul-tik_tok_(radio_edit).mp3 10-Young Jeezy-Lose My Mind (78 Bpm) (Repack).mp3 --> young_jeezy-lose_my_mind.mp3
Now the problem is how can I remove the strings like: 101 & 10 (dynamic) (%%% Bpm) (dynamic) (Repack) (static)
I have multiple strings (eg. say two, firstLIST=(0 1 2) and secondLIST=(2 3)) and want to create a single string composed of their unique sorted elements. For the sample strings above, I'd like to build masterLIST=(0 1 2 3).I suppose I could write the elements of firstLIST and secondLIST to files
as this gives me a file populated with the elements I'm after, but I'm not sure how to read the elements back into masterLIST... and it doesn't seem "right" to create files to accomplish this. Is there a way to do this by manipulating the strings ${firstLIST[@]} and ${secondLIST[@]} directly? The closest I've come (not close at all) is
Code:
masterLIST=${firstLIST[@]}" "${secondLIST[@]}
but masterLIST built this way has only one element
I would like to find all the files that contains the strings I'm searching.
For example (it's just an example), I would like to search all the files in "/etc" that contains "eth0" and "us", whatever where are located those 2 strings, the important is that the 2 strings are in the files listed.
It would be something like a "grep -lr 'eth0' *" and "grep -lr 'us' *" but in one time/command, so that I don't have to make a comparison of the 2 list of files resulting from the 2 "grep" commands given higher.
I want to search and replace strings in a file with strings in other files/i need to do it with big strings(string1 is big) and i want to use a txt file for this.But this code not working :
This should be simple but I can't seem to find what I am looking for.I want to search a text file for the existence of certain strings and execute a command if they exist, something along the lines of:
if <string> exists command or
if <any member of this list exists> command
I know how to manually search a file with grep, cat, etc., but the "if this exists" part eludes me.
I have a file called test. It has the following contents.Code:there youI want the output to be.Code:replaced youI am trying to use the sed command to replace every occurance of "hey newline there" with "replaced". I tried the following naive apporach.Code:sed 's/heythere/replace/' testThis gives a result containing the same data as the test file.
I'm not overly great with bash scripting. I do more batch file stuff for Windows than anything else but I'm trying to branch out a bit. (Frankly, I'd love to do this in Ruby but I'm not there yet...) Maybe I'm not using the correct terminology and that's why my searches are coming up useless...
Anyway, I have a script that does some work with HandBrake. Although it deals with HandBrake, I think the issue is not with HandBrake itself. The issue: I had a simpler script which worked fine but it would give me an MP4 with a file name of: Some Movie Name.iso.mp4 and it would still play but I wanted to get rid of the .iso in the file name. I also wanted to drop it into another directory during the encoding. That's why you see a $source & a $destination variable.
...and returning the index of the found element in its array.
I have:
for ((i=0; i < ${#array1[@]}; i++)); do # Read each line of the file test if [[ $(eval "sed -n '$(($i+1))'p test") == *${array2[0]}* ]] stuff
I want to find the index of the found substring in array2 and only if it isn't found, move on to the next element of array2. I don't know the size of array2 so that [0] has just got to go.
Is it possible to use the keyboard in order to select some text in the terminal windows that is not in the currently edited line? (for example, in order to copy part of previous command output).
I have some big files of logs that contain errors printed by an app. They are most of the time relevant, however most of them are similar. So i figured i could check what happened between a time interval with a find.
Im using this one
Code:
And I get an output similar to this one.
Code:
Is there a way to condensate the output lines to get only one or two, indicating the start and last occurrence of a block? Or I need to create a program to do so?
Because right now I get thousands of similar lines, but when I'm scrolling through them i sometimes miss relevant information that i would've otherwise noted if it wasn't all that spammy.
I have a file witch I need to list 10 line by 10 lines with something like press enter to go on in between. Well, the problem is that i have absolutely no idea on how to implement this.
Using awk I pull the first field of a random line from my datafile.myvar1=`awk -F" " 'NR=='$randline' {printf "%s", $1}' myfileThis works fine. The problem is there will be empty lines at the end of the file. Rather than using awkto filter out blank lines I would like to figure this out first.So I test $myvar1 for a blank string after setting $randline to one that I know is blank:test -z "$myvar1" && echo "true" || echo "false"But, this returns "false"? So the string is not zero length. Why? It's a tab-separated file. Is awk storing the tab with the $1 field or something.This is where I get headache. I try to echo my variable to see what it looks like.
echo "$myvar1" outputs: nothing echo "My variable is [$myvar1]" outputs: [y variable is [
Why is the closing bracket at the beginning? What character could be stored in $myvar1 that would do such a thing and how did it get there?
I would like to know how I can get the ouput from the following dmidecode command in example 1 to look like example 2 without having to grep -v all the unwanted lines.Is there a way in awk or sed?Example 1
Code: Processor Information Socket Designation: Socket 1 CPU 1
I'm trying to search through some pdf files and I'm doing so by converting them to text files using pdftotext which is fine but I'm trying to get the number of occurrences in a paragraph of different words and it's adding a new line character at what it thinks is the right hand margin. I'm trying to remove all these singe new line characters but keep the doubles and I can't seem to work it out. i.e.
This is some text that has been broken. Another paragraph. becomes This is some text that has been broken. Another paragraph
So I need to write a bash script that can read lines and column 3 from a file. It needs to start on line 16 and read every 20th line starting from there. But the value that it reads needs to be checked, should it be too great I need it to shut the program down.I'm pretty new to bash, but my ultimate goal is being able to safely run a program on a GPU for an extended period of time with out worrying about it overheating. I have a command that outputs information from the GPU every second, and I can save this to a file. So all I really need is something to read and check that file, I played around a bit with the awk command and can't get it to work within my for loop with dynamic variable.
I am working on a custom hand-held device which runs a variant of Ubuntu Linux, 2.6.18. I have a set of 4 binary files (for oprofile) which have been known to run in this environment, but when I copy them to /usr/bin, I can't run them: I get this message;root@ldogberry:/usr/bin# ./ophelp-sh: ./ophelp: not foundThis happens when I specify the full path, when I run from the /usr/bin directory -- it happens no matter what. Bash even does command-line completion on the file -- and then turns around and instantly claims the file is not found.
