Programming :: Perl - Delete Line From Text File With Duplicate Match At Beginning Of Line
Apr 1, 2009
Was wondering if any perl guru's could help me with a quick log file adjustment. I have a text file that looks like so (tabs and newlines are revealed so you can see what separates the data):
There are maybe 100 lines of text in this file at any given time. I need to delete all duplicate lines only looking at the first bit of text prior to the first tab. It doesn't matter which one gets deleted as long as there are no two lines that begin with that same text at the beginning before the first tab. So in this example, either the fist line "1234" or the last line "1234" would need to be deleted. I already have code in my script that opens the files - I just need the code to read the text into an array and the part that would find matches based on the above criteria, and make the deletions.
If it would be easier, I can even do a system call and use SED (v4.1.5) and/or AWK (3.1.5) instead.
View 7 Replies
ADVERTISEMENT
Jan 13, 2010
I have to delete a certain line of text from the a textfile via ubuntu's shell scripting.I have done research, and it seems that most people advocate the usage of sed /d option. sed makes does not edit the text file. Hence, most options I discovered involved the use of a temporary variable/textfile and then overwriting the old file with the temporary new file. Is there anyway whereby I can bypass the use of temporary storage containers? I hope there is any magical combination of commands to edit the file directly.
View 3 Replies
View Related
Mar 21, 2011
This solution works but is slow with large files. I am looking for a faster solution.
The 2 files contain filenames, one of them has associated data I want to append to the other file's matching filenames
file1:
file2:
I append file2 by matching the unique_filenames and appending them with the tag data and some formatting
appended file2:
Here is the SLOW code
while read inputline.
View 9 Replies
View Related
Nov 5, 2010
Is there any way to add a character to the end of a line when a certain match text is matched. I have a text file where any line that contacts text XYZ a z is placed at the end.
View 1 Replies
View Related
Oct 18, 2010
I have two txt files containing x and y coordinates: xcoord.txt & ycoord.txt. I need to open them; read them line by line to get each coordinate; then each time I need to update Xs and Ys parameters inside another file called "dc.in" with the grabbed values.
Finally each time I need to run two exe files ( dc_2002 and st_vac) and produce corresponding output for each Xs and Ys ( dc.in is an input file for this exe files)
I have written the following code but it does not work:
View 14 Replies
View Related
Jul 7, 2011
bash 3.1.17(2) I'm trying do write a shell script which must operate on each line of an ASCII text file. So, all the code must be inside a loop, and inside the loop, the first thing should be to read the next line from the file. I have the bash read command. But it reads from stdin. Any way to make read from a file?
View 6 Replies
View Related
Feb 28, 2011
At the moment I have a flat file which is being used by a few people. I want a script to remotely change the file, so I can start logging who is doing what.At this point here is one requirement I am trying to develop. We have text blocks who pretty much look like.I hope this is somewhat clear. I try to find $param for the right $workflow and change that. Can you help me to find $$var3 and change that?
View 1 Replies
View Related
Feb 28, 2011
I have a lot a folders, each named by a number, and in each of these folders I have a specific file (stddev.dat) containing a single line (of numbers) I need to have a single file with each line being one of the stddev.dat (no matter if it is sorted or not), and also I need to add at the begining of each line the number of the folder it comes from.
I 'm no bash expert, and the "add at the begining of the line" is a bit of problem to me". Here is what I've come up with so far, just to put everything in one file, (and also if you know a better/more elegant way to do the same thing I've done, I'm listening)
[Code]...
View 6 Replies
View Related
Aug 2, 2011
I just learn perl script.May i know how to simplify the code below especially in the red color part? i saw some examples in internet, they use "next" command.
View 20 Replies
View Related
Apr 12, 2010
I am trying to create a shell script, on taking a input file as parameter, which need to do 3 things
1) create a copy of existing file.
2) add a new line to the copied file.
3) strip off all the absolute paths inside the copied file
The first 2 points are straight forward. but i am finding it difficult to acheive the 3rd point. myself not very good with awk and sed. but gave it a shot in vain. For example, the input script consists of below,
PROGRAM=`/usr/bin/basename $0`
HOST=`/usr/bin/uname -n`
echo Start $PROGRAM `/usr/bin/date` |
/usr/bin/tee -a $LOG
The output of the script should look like,
PROGRAM=`basename $0`
HOST=`uname -n`
echo Start $PROGRAM `date` |
tee -a $LOG
View 14 Replies
View Related
Feb 13, 2011
I want to access a file, and check the length of every line.After, i want to check and replace all lines with length over 10 characters, with a message.Does anyone have a clue on that?
View 1 Replies
View Related
Apr 14, 2010
Is there a way to remove duplicate files from a specific folder through SSH? I've uploaded a lot of flash games on my server and I can see in the Webmin's file manager that I have many duplicates. Their names are different, of course.
View 2 Replies
View Related
Sep 12, 2010
I currently have 3 files with floating point data that I wish to have in a single file with the format:
Code:
F1 F1 F3 Output
a1 b1 c1 a1 b1 c1
a2 b2 c2 a2 b2 c2
a3 b3 c3 a3 b3 c3
a4 b4 c4 a4 b4 c4
View 3 Replies
View Related
Jul 6, 2010
I'm a bit new to Python programming and hoped that someone might be able to help with a problem I'm having. What I essentially want to do is to combine two text files line for line. I know how to do this in a bash script so to give you a better idea here's the code for that:
Code:
This is basically for adding on values to the end of a CSV file that uses ';' as the delimiter. So say file1 said:
And file2 said:
Then running this command would create merged_file1_and_file2 which would be:
The code I'm using at the moment is:
Code:
As I'm sure any experienced python programmer will see, this prints out the first line of the file "csvraw" and then all of the lines of "stamps" and then the remainder of "csvraw".
What I'd like to do is something like: (pseudo code, I know it's not python ;-))
Code:
Is this possible? I've tried googling and my Python Pocket Reference hasn't been much help. I've looked at pickling but that doesn't seem appropriate.
View 1 Replies
View Related
Feb 7, 2011
I have a file "test.txt" with following data
#1
aaa
#2
bbb
#3
aaa
#4
ddd
I wanted it to be displayed as
#1
aaa
#2
bbb
#4
ddd
I used awk "'!x[$0]++' test.txt > file.new"
,but it deleted #1 also.I tried using uniq command but i didn't work.
Can anyone Please let me know is there any way to do this using shell script.
View 2 Replies
View Related
May 3, 2010
a sed command to add a text before line number in text file? I have text file with 500 lines, and i want to add 3 more lines with text after line 300, OR before line 302, isn't no problem.
View 16 Replies
View Related
Oct 13, 2010
I want to delete lines which contain the word "Fehler" but for some reason my programm isn't working:
Code: BEGIN { FS="|"
OFS="|"
SUBSEP="|"
[code]...
View 6 Replies
View Related
Apr 9, 2011
I would like to delete a single line from a file that contains many lines passing through the same values as the two parameters. Again, I would like to delete a single line and not all those that contain parameters. How can I make bash?
View 14 Replies
View Related
Aug 3, 2010
I need to output of a script to a file with tags.
I'm trying sed and was able to put a tag on the end of the line
Code:
Output is
Quote:
What I need to have is
Quote:
I'm thinking of I redirect my first sed command to a file and then use sed again to put <group> at the beginning of the line. My problem is how to put <group> to all beginning of the string.
View 2 Replies
View Related
Oct 16, 2010
i have a file containing contents as
1 2 3
2 2 3
11 2 3
now i want to delete the line which first column content is 1. so, if i pass the parametere 1 to function delete_row. is should remove the first row as
2 2 3
11 2 3
and then write the contents in file.
View 2 Replies
View Related
Jan 26, 2010
Quote:Originally Posted by topcatI would like to know how i can write a shell script to delete a line if a particular pattern exists?E.g. I have a text file with multiple lines. Say 1000s. in the following pattern.
username@email.com:149.0.3.4:1
username1@email.com:149.0.3.4:1
username1@email.net:149.0.3.4:1
username1@email.edu:149.0.3.4:1
If the patternusername@email.com exists then the line "username@email.com:149.0.3.4:1 should be deleted from the file.I have a very similar question but I need to delete one line in a file which matches one very precise instance of a string only. Let's assume I have a file composed of thousands of lines and let's call the file chap-secrets. Let's take the following sample entries:
Code:
#USERNAME SERVER PASSWORD IP
pp pptpd blahblah *
[code]....
View 7 Replies
View Related
Jul 26, 2010
I'm trying to add text to a file for a specific group of users, I'll need to do examples as I can't think of an easy way of explaining, my file is like this:
Code:
users{
user1
user2
[code]....
At the present my code lists all the available groups, how would I add a user to a specified group? (e.g add "members user3") to the end of group 1 for example. So the code ends up like this
Code:
members user2,user3
View 14 Replies
View Related
Apr 16, 2010
I try to do a function which removes '#' from beginning of line. But if there is not '#' then add it on beginning of line. I try something like this, can you correct me?
Code:
awk '
!/^#/{ sub(/^[^#]*/,"#$0","'$line'")
}
/^#/{ sub(/#/,"","'$line'")
}'
View 14 Replies
View Related
Oct 17, 2010
I am having some difficulties with an expect script. What I want is that as soon as expect sees a newline followed by a dollarsign ($), that interact is done. So I would expect something like this to work:
Code:
"
$" { interact }
However it doesn't.
This is debugging output:
Code:
expect: continuing expect
expect: does " " (spawn_id exp6) match glob pattern "#"? no
[Code]...
View 3 Replies
View Related
Nov 6, 2010
Say I have a text file like:
Code:
1
3
4
How would I use ksh to put the number '2' into the second line of that file?Okay it's not bash, it's ksh because this computer is OpenBSD
View 14 Replies
View Related
Dec 31, 2010
I have a text file called namelist.wps. In this file there is a line that reads:
Code:
start_date = '2010-12-26_12:00:00', '2010-12-26_12:00:00',
I have to automatically update the year, month, and day of month for this line without changing the rest of the file. Here is the script that I have:
[Code]....
View 5 Replies
View Related
Jun 21, 2010
Does any one know what syntax i could use to allow me to replace all instances at the beginning of a line with ones.
Before :
Code:
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical device number 0
Logical device name : RAID1Mirror
RAID level : 1
Status of logical device : Optimal
After
Code:
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical device number 0
111Logical device name : RAID1Mirror
1111RAID level : 1
11111Status of logical device : Optimal
View 12 Replies
View Related
Aug 5, 2010
I am trying to create a shell script where a user can specify a file with a list of logins and the script will create a batch file with specific information in specific columns.
Example:
loginfile.txt has
User1
User2
User3
I need the output to be as below:
1 User1 Date 12/31/9999
2 User2 Date 12/31/9999
3 User3 Date 12/31/9999
Where the columns are separated as below.
1[tab][tab]User1[tab]date +%D[tab]12/31/9999
2[tab][tab]User2[tab]date +%D[tab]12/31/9999
3[tab][tab]User3[tab]date +%D[tab]12/31/9999
I can use the nl utility to get the numbers easily enough, but I need two tab separations between the number and the user list. Is there a sed command that will insert at the beginning of each line? If so I can just run nl after I get some tabs up in the front.
View 2 Replies
View Related
Dec 30, 2010
I have a text file called namelist.wps. In this file there is a line that reads:Code: start_date = '2010-12-26_12:00:00', '2010-12-26_12:00:00', I have to automatically update the year, month, and day of month. I set values for the year, month, and day of month using the following code in a c-shell script:Code: set y1 = `date +%Y`set m1 = `date +%m`set d1 = `date +%d` After I do this, how do I update year, month, and day of month, without changing any of the other lines in the namelist.wps file?
View 2 Replies
View Related
Apr 15, 2009
I am thinking of appending something to each line in a text file with Java. I prefer not write a new file with content appended from the old one.That 'something' would probably be Time Stamp when the file is created (which is same for each line).I am not sure Java provide some easy way for it or not
View 3 Replies
View Related
