Programming :: Perl - Delete Line From Text File With Duplicate Match At Beginning Of Line
Apr 1, 2009
Was wondering if any perl guru's could help me with a quick log file adjustment. I have a text file that looks like so (tabs and newlines are revealed so you can see what separates the data):
There are maybe 100 lines of text in this file at any given time. I need to delete all duplicate lines only looking at the first bit of text prior to the first tab. It doesn't matter which one gets deleted as long as there are no two lines that begin with that same text at the beginning before the first tab. So in this example, either the fist line "1234" or the last line "1234" would need to be deleted. I already have code in my script that opens the files - I just need the code to read the text into an array and the part that would find matches based on the above criteria, and make the deletions.
If it would be easier, I can even do a system call and use SED (v4.1.5) and/or AWK (3.1.5) instead.
I have to delete a certain line of text from the a textfile via ubuntu's shell scripting.I have done research, and it seems that most people advocate the usage of sed /d option. sed makes does not edit the text file. Hence, most options I discovered involved the use of a temporary variable/textfile and then overwriting the old file with the temporary new file. Is there anyway whereby I can bypass the use of temporary storage containers? I hope there is any magical combination of commands to edit the file directly.
Is there any way to add a character to the end of a line when a certain match text is matched. I have a text file where any line that contacts text XYZ a z is placed at the end.
I have two txt files containing x and y coordinates: xcoord.txt & ycoord.txt. I need to open them; read them line by line to get each coordinate; then each time I need to update Xs and Ys parameters inside another file called "dc.in" with the grabbed values.
Finally each time I need to run two exe files ( dc_2002 and st_vac) and produce corresponding output for each Xs and Ys ( dc.in is an input file for this exe files)
I have written the following code but it does not work:
bash 3.1.17(2) I'm trying do write a shell script which must operate on each line of an ASCII text file. So, all the code must be inside a loop, and inside the loop, the first thing should be to read the next line from the file. I have the bash read command. But it reads from stdin. Any way to make read from a file?
At the moment I have a flat file which is being used by a few people. I want a script to remotely change the file, so I can start logging who is doing what.At this point here is one requirement I am trying to develop. We have text blocks who pretty much look like.I hope this is somewhat clear. I try to find $param for the right $workflow and change that. Can you help me to find $$var3 and change that?
I have a lot a folders, each named by a number, and in each of these folders I have a specific file (stddev.dat) containing a single line (of numbers) I need to have a single file with each line being one of the stddev.dat (no matter if it is sorted or not), and also I need to add at the begining of each line the number of the folder it comes from.
I 'm no bash expert, and the "add at the begining of the line" is a bit of problem to me". Here is what I've come up with so far, just to put everything in one file, (and also if you know a better/more elegant way to do the same thing I've done, I'm listening)
I just learn perl script.May i know how to simplify the code below especially in the red color part? i saw some examples in internet, they use "next" command.
I am trying to create a shell script, on taking a input file as parameter, which need to do 3 things
1) create a copy of existing file.
2) add a new line to the copied file.
3) strip off all the absolute paths inside the copied file
The first 2 points are straight forward. but i am finding it difficult to acheive the 3rd point. myself not very good with awk and sed. but gave it a shot in vain. For example, the input script consists of below,
I want to access a file, and check the length of every line.After, i want to check and replace all lines with length over 10 characters, with a message.Does anyone have a clue on that?
Is there a way to remove duplicate files from a specific folder through SSH? I've uploaded a lot of flash games on my server and I can see in the Webmin's file manager that I have many duplicates. Their names are different, of course.
I'm a bit new to Python programming and hoped that someone might be able to help with a problem I'm having. What I essentially want to do is to combine two text files line for line. I know how to do this in a bash script so to give you a better idea here's the code for that:
Code:
This is basically for adding on values to the end of a CSV file that uses ';' as the delimiter. So say file1 said:
And file2 said:
Then running this command would create merged_file1_and_file2 which would be:
The code I'm using at the moment is:
Code:
As I'm sure any experienced python programmer will see, this prints out the first line of the file "csvraw" and then all of the lines of "stamps" and then the remainder of "csvraw".
What I'd like to do is something like: (pseudo code, I know it's not python ;-))
Code:
Is this possible? I've tried googling and my Python Pocket Reference hasn't been much help. I've looked at pickling but that doesn't seem appropriate.
I used awk "'!x[$0]++' test.txt > file.new" ,but it deleted #1 also.I tried using uniq command but i didn't work. Can anyone Please let me know is there any way to do this using shell script.
a sed command to add a text before line number in text file? I have text file with 500 lines, and i want to add 3 more lines with text after line 300, OR before line 302, isn't no problem.
I would like to delete a single line from a file that contains many lines passing through the same values as the two parameters. Again, I would like to delete a single line and not all those that contain parameters. How can I make bash?
I'm trying sed and was able to put a tag on the end of the line
Code:
Output is
Quote:
What I need to have is
Quote:
I'm thinking of I redirect my first sed command to a file and then use sed again to put <group> at the beginning of the line. My problem is how to put <group> to all beginning of the string.
now i want to delete the line which first column content is 1. so, if i pass the parametere 1 to function delete_row. is should remove the first row as
Quote:Originally Posted by topcatI would like to know how i can write a shell script to delete a line if a particular pattern exists?E.g. I have a text file with multiple lines. Say 1000s. in the following pattern.
If the patternusername@email.com exists then the line "username@email.com:149.0.3.4:1 should be deleted from the file.I have a very similar question but I need to delete one line in a file which matches one very precise instance of a string only. Let's assume I have a file composed of thousands of lines and let's call the file chap-secrets. Let's take the following sample entries:
Code: #USERNAME SERVER PASSWORD IP pp pptpd blahblah *
I'm trying to add text to a file for a specific group of users, I'll need to do examples as I can't think of an easy way of explaining, my file is like this:
Code:
users{ user1 user2
[code]....
At the present my code lists all the available groups, how would I add a user to a specified group? (e.g add "members user3") to the end of group 1 for example. So the code ends up like this
I try to do a function which removes '#' from beginning of line. But if there is not '#' then add it on beginning of line. I try something like this, can you correct me?
I am having some difficulties with an expect script. What I want is that as soon as expect sees a newline followed by a dollarsign ($), that interact is done. So I would expect something like this to work:
Code: " $" { interact } However it doesn't. This is debugging output: Code: expect: continuing expect expect: does " " (spawn_id exp6) match glob pattern "#"? no
Say I have a text file like: Code: 1 3 4 How would I use ksh to put the number '2' into the second line of that file?Okay it's not bash, it's ksh because this computer is OpenBSD
I have a text file called namelist.wps. In this file there is a line that reads:
Code: start_date = '2010-12-26_12:00:00', '2010-12-26_12:00:00', I have to automatically update the year, month, and day of month for this line without changing the rest of the file. Here is the script that I have:
Does any one know what syntax i could use to allow me to replace all instances at the beginning of a line with ones.
Before :
Code: ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical device number 0 Logical device name : RAID1Mirror RAID level : 1 Status of logical device : Optimal
After
Code: ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical device number 0 111Logical device name : RAID1Mirror 1111RAID level : 1 11111Status of logical device : Optimal
I am trying to create a shell script where a user can specify a file with a list of logins and the script will create a batch file with specific information in specific columns.
Example:
loginfile.txt has
User1 User2 User3
I need the output to be as below:
1 User1 Date 12/31/9999 2 User2 Date 12/31/9999 3 User3 Date 12/31/9999
I can use the nl utility to get the numbers easily enough, but I need two tab separations between the number and the user list. Is there a sed command that will insert at the beginning of each line? If so I can just run nl after I get some tabs up in the front.
I have a text file called namelist.wps. In this file there is a line that reads:Code: start_date = '2010-12-26_12:00:00', '2010-12-26_12:00:00', I have to automatically update the year, month, and day of month. I set values for the year, month, and day of month using the following code in a c-shell script:Code: set y1 = `date +%Y`set m1 = `date +%m`set d1 = `date +%d` After I do this, how do I update year, month, and day of month, without changing any of the other lines in the namelist.wps file?
I am thinking of appending something to each line in a text file with Java. I prefer not write a new file with content appended from the old one.That 'something' would probably be Time Stamp when the file is created (which is same for each line).I am not sure Java provide some easy way for it or not
