Professional Documents
Culture Documents
Home
Shell Scripts
More
Different methods
Next Blog
Create Blog
Q&A
Contact
Sign In
About
Email address...
Submit
In this article of sed tutorial series, we are going to see how to delete or remove a particular line or a
particular pattern from a file using the sed command.
Let us consider a file with the sample contents as below:
$ cat file
Cygwin
Unix
Linux
Solaris
AIX
1. Delete the 1st line or the header line:
$ sed '1d' file
Unix
Linux
Solaris
AIX
Join us
pdfcrowd.com
Facebook Fans
2.9K likes
pdfcrowd.com
6. Delete the first line AND the last line of a file, i.e, the header and trailer line of a file.
Article Archive
$ sed '1d;$d' file
Unix
Linux
Solaris
2014 (2)
2013 (19)
2012 (81)
December 2012 (6)
Multiple conditions are separated using the ';' operator. Similarly, say to delete 2nd and 4th line,
you can use: '2d;3d'.
7. Delete all lines beginning with a particular character, 'L' in this case:
8. Delete all lines ending with a particular character, 'x' in this case:
'x$' indicates lines ending with 'x'. AIX did not get deleted because the X is capital.
pdfcrowd.com
content
awk - 10 examples to group data in a
CSV or text f...
2011 (29)
2010 (36)
'^$' indicates lines containing nothing and hence the empty lines get deleted. However, this wont
delete lines containing only some blank spaces.
11. Delete all lines which are empty or which contains just some blank spaces:
Developer works
About Linux
Linux Screw
Linux Journal
UbuntuLife
LFY
'*' indicates 0 or more occurrences of the previous character. '^ *$' indicates a line containing
zero or more spaces. Hence, this will delete all lines which are either empty or lines with only some
blank spaces.
12. Delete all lines which are entirely in capital letters:
$ sed '/^[A-Z]*$/d' file
Cygwin
pdfcrowd.com
Unix
Linux
Solaris
[A-Z] indicates any character matching the alphabets in capital.
13. Delete the lines containing the pattern 'Unix'.
$ sed '/Unix/d' file
Cygwin
Linux
Solaris
AIX
The pattern is specified within a pair of slashes.
14. Delete the lines NOT containing the pattern 'Unix':
$ sed '/Unix/!d' file
Unix
15. Delete the lines containing the pattern 'Unix' OR 'Linux':
$ sed '/Unix\|Linux/d' file
Cygwin
Solaris
AIX
The OR condition is specified using the | operator. In order not to get the pipe(|) interpreted as a
literal, it is escaped using a backslash.
16. Delete the lines starting from the 1st line till encountering the pattern 'Linux':
$ sed '1,/Linux/d' file
Solaris
AIX
pdfcrowd.com
Earlier, we saw how to delete a range of lines. Range can be in many combinations: Line ranges,
pattern ranges, line and pattern, pattern and line.
17. Delete the lines starting from the pattern 'Linux' till the last line:
$ sed '/Linux/,$d' file
Cygwin
Unix
18. Delete the last line ONLY if it contains the pattern 'AIX':
$ sed '${/AIX/d;}' file
Cygwin
Unix
Linux
Solaris
$ is for the last line. To delete a particular line only if it contains the pattern AIX, put the line
number in place of the $. This is how we can implement the 'if' condition in sed.
19. Delete the last line ONLY if it contains either the pattern 'AIX' or 'HPUX':
$ sed '${/AIX\|HPUX/d;}' file
Cygwin
Unix
Linux
Solaris
20. Delete the lines containing the pattern 'Solaris' only if it is present in the lines from 1 to
4.
$ sed '1,4{/Solaris/d;}' file
Cygwin
Unix
Linux
AIX
This will only delete the lines containing the pattern Solaris only if it is in the 1st four lines,
pdfcrowd.com
nowhere else.
21. Delete the line containing the pattern 'Unix' and also the next line:
$ sed '/Unix/{N;d;}' file
Cygwin
Solaris
AIX
N command reads the next line in the pattern space. d deletes the entire pattern space which
contains the current and the next line.
22. Delete only the next line containing the pattern 'Unix', not the very line:
$ sed '/Unix/{N;s/\n.*//;}' file
Cygwin
Unix
Solaris
AIX
Using the substitution command s, we delete from the newline character till the end, which
effective deletes the next line after the line containing the pattern Unix.
23. Delete the line containing the pattern 'Linux', also the line before the pattern:
$ sed -n '/Linux/{s/.*//;x;d;};x;p;${x;p;}' file | sed '/^$/d'
Cygwin
Solaris
AIX
A little tricky ones. In order to delete the line prior to the pattern,we store every line in a buffer
called as hold space. Whenever the pattern matches, we delete the content present in both, the
pattern space which contains the current line, the hold space which contains the previous line.
Let me explain this command: 'x;p;' ; This gets executed for every line. x exchanges the content of
pattern space with hold space. p prints the pattern space. As a result, every time, the current line
goes to hold space, and the previous line comes to pattern space and gets printed. When the
pattern /Linux/ matches, we empty(s/.*//) the pattern space, and exchange(x) with the hold space(as
a result of which the hold space becomes empty) and delete(d) the pattern space which contains the
pdfcrowd.com
previous line. And hence, the current and the previous line gets deleted on encountering the pattern
Linux. The ${x;p;} is to print the last line which will remain in the hold space if left.
The second part of sed is to remove the empty lines created by the first sed command.
24. Delete only the line prior to the line containing the pattern 'Linux', not the very line:
$ sed -n '/Linux/{x;d;};1h;1!{x;p;};${x;p;}' file
Cygwin
Linux
Solaris
AIX
This is almost same as the last one with few changes. On encountering the pattern /Linux/, we
exchange(x) and delete(d). As a result of exchange, the current line remains in hold space, and the
previous line which came into pattern space got deleted.
1h;1!{x;p;} - 1h is to move the current line to hold space only if it first line. Exchange and print for
all the other lines. This could easily have been simply: x;p . The drawback is it gives an empty line at
the beginning because during the first exchange between the pattern space and hold space, a new
line comes to pattern space since hold space is empty.
25. Delete the line containing the pattern 'Linux', the line before, the line after:
$ sed -n '/Linux/{N;s/.*//;x;d;};x;p;${x;p;}' file | sed '/^$/d'
Cygwin
AIX
With the explanations of the last 2 commands, this should be fairly simple to understand.
Tweet
Like 9
111K
70 comments:
Ravichander August 27, 2012 at 8:31 PM
Nice collection of commands !! Keep posting such Good articles !! :)
Reply
pdfcrowd.com
pdfcrowd.com
Guru Prasad
Thanks Anies
Reply
Use the head(to get top 10) and tail(to get last 10) commands for this.
Reply
pdfcrowd.com
10 columns in which 2,5,7,8 columns are coming as null/blank for some records.Those should
be removed from the file.I am using awk command but how will be the performance if millions
of records are there in a file.
Reply
Replies
Guru Prasad
The command you used is correct one. However, only if your file content is
beginning with ',Pub,...', use the ^, else drop it.
Reply
pdfcrowd.com
pdfcrowd.com
Reply
Replies
Guru Prasad
x=$(wc -l <file)
x=$(expr $x - 10)
sed -n "1,$x p" file
Reply
pdfcrowd.com
Reply
pdfcrowd.com
Older sed has these problems. Use GNU sed. Try inserting a new line character
after the long line and try once.
Reply
pdfcrowd.com
Hi Guru,
This was so helpful. I have a quick question.
I need to match a string and delete the previous lines (not the matched line) from a CSV file.
My file would look like:
6,ZYX
0006/11262013,SLK,SLK,ABN,10:00:45.302765,19:00:00.000000,19:00:00.000000,19:00:00
.000000,0,0,0,0,0,0,0,0,0,0,0,0,N
7,ZYX
0007/11262013,SLK,SLK,ADI,10:00:45.302774,19:00:00.000000,19:00:00.000000,19:00:00.
000000,0,0,0,0,0,0,0,0,0,0,0,0,N
8,ZYX
0008/11262013,SLK,SLK,ADI,10:00:45.302780,19:00:00.000000,19:00:00.000000,19:00:00.
000000,0,0,0,0,0,0,0,0,0,0,0,0,N
920,ABY
4531/11262013,#DM02,SLK,KEY,10:28:36.221893,10:28:36.234889,19:00:00.000000,19:00
:00.000000,0,0,0,0,0,0,0,0,0,0,0,0,N
921,AAA
0110/11262013,#DM02,SLK,MDZ,10:28:36.233397,10:28:36.311391,19:00:00.000000,19:00
:00.000000,0,0,0,0,0,0,0,0,0,0,0,0,N
Reply
Replies
Guru Prasad
pdfcrowd.com
Guru Prasad
pdfcrowd.com
Reply
Replies
Guru Prasad
Please post this in Q&A(tab before contact). Also give an example input file and
output expected.
Reply
pdfcrowd.com
A3 3 4 5
And file B has entries like
A1
A2
How do I delete all lines from file A whereever ids match to file B?
Reply
Replies
Guru Prasad
pdfcrowd.com
pdfcrowd.com
Reply
pdfcrowd.com
pdfcrowd.com
- The "&" is used as a sed delimiter to make it more readable, since your pattern includes
backslashes.
- The double backslashes "\\" are necessary since \ is used to escape special characters.
Hope this helps!
R
Reply
Replies
Anurag Tiwary April 20, 2015 at 10:57 AM
Thanks for the help. I'll try this and get back.
Reply
pdfcrowd.com
-------data15 data25
data16 data26
data17 data27
data18 data28
Header
Page: 3
column1 column2
-------data19 data29
data20 data30
Expected output b.txt
Header
Page: 1
column1 column2
-------data11 data21
data12 data22
data13 data23
data14 data24
data15 data25
data16 data26
data17 data27
data18 data28
data19 data29
data20 data30
I tried with sed '/Header/,/---/'d a.txt >b.txt
but in that process I am losing first occurance of header as well.
I wish to start above command from line 5, however I could not build one.
Tried sed '5,/Header/,/---/'d a.txt >b.txt, but its seems wrong syntax.
any inputs, help?
Reply
Replies
pdfcrowd.com
Guru Prasad
pdfcrowd.com
output must be
L1137Q1 shantanu 040013.15 L Prashant .00
L1337Q1 shantanu 040013.15 L Prashant .00
Thank you in advance.
Reply
Replies
Guru Prasad
pdfcrowd.com
ID JDK
Version 5.1.1.12
ID ND
OUTPUT
--------------------------Version 5.1
Version 5.1.1.12
ID BASE
Version 5.1
Version 5.1.1.12
ID ND
Reply
Replies
Guru Prasad
Refer to example 23
Reply
pdfcrowd.com
pdfcrowd.com
$ y=$(wc -l<file)
$ let x=y-3+1
$ sed "$x,$y d" file
Reply
pdfcrowd.com
dd start ee
ff end gg
The output should be
bbb ccc
aaa cc
aa bb cc
dd
gg
Thanks!
Reply
Comment as:
Publish
Select profile...
Preview
pdfcrowd.com
Newer Post
Home
Older Post
Copyright 2013 The UNIX School. All rights reserved. Awesome Inc. template. Powered by Blogger.
pdfcrowd.com