Programming :: Split / Break Up Large Text File
Feb 18, 2011
I have a large text file with three columns. I'm trying to write a PERL script that splits the file up based on the value of the 3rd column. So every time the third column reads 0, a new file is created and all the data up until the next 0 is found is written to that new file. This should happen over and over until the initial file has been entirely split up.
ex data:
000
2024
2243
2143
96962871
97972878
000
2034
3034
3352
So with the data above, the file would be split into two files:
data_1.txt would contain
000
2024
2243
2143
96962871
97972878
and data_2.txt would contain
230
2034
3034
3352
View 10 Replies
ADVERTISEMENT
Sep 18, 2010
I have a file (called twitterstatus.tmp) that looks like this:
Code:
<status>
<id>24854489768</id>
<text>Are we gonna ride the sun home?</text>
<id>55266987</id>
[code].....
How could I feed this into an array, with each element containing everything between the <status> </status> tags?
View 9 Replies
View Related
Jan 11, 2010
I use csplit to split up a large file and I get file xx01, xx02 and so on. I use a for loop to loop through the files.
Code:
for f in "xx**"
do
echo test
echo $f
[Code].....
View 3 Replies
View Related
Apr 16, 2011
I am removing some old graphics from my server and one of the gallery programs have created two enormous directories that I cannot even open with FTP.
I tried to tar each directory and the first came out to about 37gb and the second keeps failing (its bigger one would assume).
How can I archive and split these into smaller files?
View 13 Replies
View Related
Dec 15, 2010
I have a clump of text that needs to be broke up:gdbm Sat 07 Feb 2009 03:28:18 AM EST libattr Sat 07 Feb 2009 03:28:18 AM EST db4 Sat 07 Feb 2009 03:28:19 AM EST mktemp Sat 07 Feb 2009 03:28:19 AM EST keyutils Sat 07 Feb 2009 03:28:20 AM EST pcre Sat 07 Feb 2009 03:28:21 AM EST setserial Sat 07 Feb 2009 03:28:24 AM EST zlib Sat 07 Feb 2009 03:28:24 AM EST gawk Sat 07 Feb 2009 03:28:25 AM EST readline Sat 07 Feb 2009 03:28:26 AM EST rhpl Sat 07 Feb 2009 03:28:28 AM EST cracklib-dicts Sat 07 Feb 2009 03:28:37 AM EST setools Sat 07 Feb 2009 03:28:37 AM EST hal Sat 07 Feb 2009 03:28:38 AM EST which Sat 07 Feb 2009 03:28:39 AM EST Is there a way to get everything after the EDT in the text to be moved to a new line?
View 13 Replies
View Related
May 17, 2011
How does one split a large linux file and transfer to windows external drive ?
View 2 Replies
View Related
Sep 28, 2009
I have on my windows machine several hundred files that are a format of .nc .ncs for a CNC machine. I need to convert them to txt which is something as easy as opening in notepad and then saving as .txt but there are so many that this kind of action would take way too long.
The reason I am writing the linuxquestions is because I would feel more comfortable in loading a live CD and using some sort of terminal command to do this than I would to download one of the many "freeware" type programs I have found for windows (even more so since I have had a root kit before and had to start all the way over to get rid of it).
I need to know:
1. Is this possible to do with the terminal without super advanced knowledge.
2. Can one please point me in the right direction; something to read or an example
View 2 Replies
View Related
Nov 26, 2010
I'm trying to manipulate a large text file full of records (metadata - one complete record per line). I need to delete every line on which certain words appear - there are five different words, all pretty simple all-caps strings with occasional whitespace. I tried using grep -v, which worked a treat, but only string-by-string. Ideally I'd like to run this as grep -v -f, where the file targeted by the -f contains the strings I need to match in order to delete the lines they're in.
i.e. grep -v -f filecontainingSTRINGS.txt targetfile.txt > outputfile.txt
When I try this, however, I don't get any matches - or more specifically, no changes are made in the output file. It works fine if there's only one string in filecontainingSTRINGS, but it doesn't work if there's more than one (I'm using newline as the delimiter). (Also my machine doesn't recognise /usr/xpg4/bin/grep - no idea what that's all about!)
View 5 Replies
View Related
Feb 27, 2011
I have a single file that contain multi-text something likes this:
Quote:
No. Time Source Destination Protocol Info
185 27712.068199 192.168.18.23 192.168.18.191 SMTP S: 250 2.1.5 Ok
No. Time Source Destination Protocol Info
186 27715.068293 192.168.0.50 192.168.5.2 TCP suncacao-jmxmp > 44693 [ACK] Seq=1 Ack=1 Win=64807 Len=1380
[Code].....
View 14 Replies
View Related
Jan 28, 2009
I have a text file called file1.txt containing many lines eg.
line1
line2
line3
line4
line5
line6
Then i have another text file called file2.txt contains
3
5
6
Is there a command to remove the lines in file1.txt based on the keywords in file2.txt? note: It should remove line3,line5,line6 based on 3,5,6
View 10 Replies
View Related
Jun 25, 2010
I have a file in which contains one line with a lot floating points.In the very first place and some times in the downstream, there are a few integers, surrounded by blank spaces.1 1.02-4 1.03-5 544 1.04-1 65 2.98-1 5.78-10 3.45-2 etc etc.I aim to split the file in more files each of them containing an integer and the following floatings until the next integer.
View 1 Replies
View Related
Aug 26, 2010
If there's another way to improve a script that I have created, since I'm not an expert! It works, and it made what I wanted, but it took a while to do it... maybe it can be improved. Here's the background. I have one file, with 244000 lines, let's call it X. I needed to split it in 1000 files, each one of 244 lines. I also needed the files to have the .arp extension. So here is what I did:
Code:
for i in {1..1000}
do
sed -n '1,244p;244q' X > $i.arp
sed -i '1,244d;' X # in this way I deleted the copied lines each time
done
View 12 Replies
View Related
May 14, 2011
whether it is possible to split open a file and edit only one of the open windows.
View 2 Replies
View Related
Jan 19, 2009
I need to insert 3-4 lines of text to the beginning of a text file. The file is a largish MYSQL dump, the result of a backup shell script. This shell script should insert the required text.I've wrestled with sed, but lost.
View 2 Replies
View Related
Jan 13, 2010
I have to delete a certain line of text from the a textfile via ubuntu's shell scripting.I have done research, and it seems that most people advocate the usage of sed /d option. sed makes does not edit the text file. Hence, most options I discovered involved the use of a temporary variable/textfile and then overwriting the old file with the temporary new file. Is there anyway whereby I can bypass the use of temporary storage containers? I hope there is any magical combination of commands to edit the file directly.
View 3 Replies
View Related
Jan 8, 2011
I want to display something in my text view widget in glade using c code. that's all right.
now I need to attach a save button beneath the text view.so that on click the text view content should save as a txt file..
View 8 Replies
View Related
Feb 9, 2011
I want to display the contents of a particular log file (simple text file, I mean in Linux). But there is a problem: The contents need to be organized in a fixed format. Have a look at this log file:
sampleLog.txt
Code:
User Name: XYZ
Reported Problems Description: Blah! Blah! Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah!
[code]....
So, while displaying the contents of above file on a web page, I want to format the field names found in the log file: User Name:, Reported Problems Description:, and Remarks:. These fields may contain a variable length of text and no specific line number is assumed for them to appear on.
The desired output should look like this:
User Name: XYZ
Reported Problems Description: Blah! Blah! Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah! Blah!Blah!
[code]....
Well, what I am trying to do may sound wierd to some of you. The filed "Reported Problems Description:" can possible contain text which embeds colon (.
View 15 Replies
View Related
May 3, 2010
a sed command to add a text before line number in text file? I have text file with 500 lines, and i want to add 3 more lines with text after line 300, OR before line 302, isn't no problem.
View 16 Replies
View Related
May 20, 2011
Im in a unique situation where a box has a 400G file, on FreeBSD, without thesplit' utility available
View 11 Replies
View Related
Apr 19, 2011
I have a .txt-file with ~50.000 lines of numbers, generated by a mathematics program. From this file, I need line ~ 1.100 to line ~16.000 (these lines are always the same btw, this may make the solution easier, dunno) to be copy/pasted to another file, where the lines ~500 to ~15.000 (also, every time the same) should be overwritten by the aforementioned lines...I haven't found or come up with anything that works yet, mostly I find solutions to copy everything from one file to another but I can't find something to specifically overwrite a part of a file with part of another.
View 4 Replies
View Related
Oct 16, 2010
I'm looking for a c++ IDE and/or Text editor with split screen abilities like what you see below with notepad ++. Having the ability to split your screen is very helpful when comparing code.
[URL]
I have tried geany which chokes VERY badly when trying to compile multiple files. I have tried kate which doesn't have a compiler.
View 6 Replies
View Related
Nov 11, 2010
What I plan to do is, create a duplicate file of a text file, and then append some text into the new text file.
View 1 Replies
View Related
Mar 11, 2011
I have been working on this since 3 days but wasn't able to achieve what I want
I have a big text that has the following format:
Current max fieldLen for table1 (a):
Fld# Width MaxWidth ERR NAME
---- ------ -------------- ------ ---------
2: 80 38 *** field-name
3: 4 2 field-version
4: 40 7 field-value
[Code]....
View 7 Replies
View Related
Apr 28, 2011
I have a text field that is just list of servers and I need to add the word hostname in front of them... It must be brain fart but I can't think of how to do this. Basically I need this:
server1
server2
server3
To this:
hostname server1
hostname server2
hostname server3
(And I just mean simply the word "hostname")
View 2 Replies
View Related
Feb 10, 2011
I need a sed and renaming the text in file. we have this one:
Quote:
Originally Posted by nickname
nickname presents: $subject
Size: size
[code]....
View 2 Replies
View Related
Mar 24, 2011
I want to know that how the break points and watch points work in the GDB. I know the GDB works on symbol table. But would like to know more details of it. How watch points differ from break points.
View 3 Replies
View Related
Feb 25, 2011
I have some variables that are loaded from mysql - in mysql, the line breaks are not indicated by ' or '<br>' or anything -they are just line breaks. the variable is loaded into javascript and then inserted into a <pre> element. the line breaks are not showing up.
pre1 = pre1.replace(/>/g,">"); - this works well, naturally. but how do i get those line breaks to show up as line breaks in a browser?
I've tried many variations of something like:
View 1 Replies
View Related
Jun 30, 2010
I need to extract som text from a text file. The text is a test log with system info at the top and results further down. What I need is to add different tags with formatting before and after each line. I have prepared a template with html formatting, but the number of lines in the test log may be different from case to case, so I need to be able to add formatting tags by need. Can this be done using bash script, sed, awk, head, tail... ?
View 4 Replies
View Related
Dec 21, 2010
I have a file with 5000 lines. it is a list of books authors, series and titles. all lines start with the author names, than there is a dash (-) than the series name, a dash again and the title of the book.
The problem I encounter is that sometime there is a series, sometime not, and as I try to enter this list in a database, I wanted to create a cvs file to import into mysql.
ex:
The best would be to be able to add in the second line, a "space dash space" just after the author name, but how to make sure it does not do it to the first line as well.
If I could separate all line with 2 dash, (grep ?) then I would be able to do a simple replace, and change the single dash into two.
View 2 Replies
View Related
Jun 3, 2011
Code:
int main ()
{
[code]...
View 9 Replies
View Related
