首页 > 其他 > 详细

文本处理三剑客之sed

时间:2019-12-20 00:03:11      阅读:106      评论:0      收藏:0      [点我收藏+]

 

 

 

 


 

 

 

 

 

文本处理三剑客之sed知识概述

 

三剑客sed命令概述介绍
三剑客sed命令执行原理命令基本语法
三剑客sed命令实操练习增删改查

 

 

 

 

 

一sed命令概述

 

 

sed - stream editor for filtering and transforming text

1)可以查找过滤筛选出指定的信息
2)擅长编辑文件内容信息
3)擅长对文件行进行操作

 

 

 

命令语法结构:sed     [参数]      ‘ 条件   指令‘     文件信息

定义条件:根据行号   根据内容    根据正则信息

定义指令:满足条件的信息处理动作(添加删除修改输出)

 

 

 

二sed命令执行原理

 

 

sed命令执行过程核心原则:找什么信息,做什么事情

找什么信息==定义查找条件

做什么事情==操作执行命令(替换删除增加输出)

 

 

 

 

 

 

三sed命令的查询操作

 

实际上是输入操作

 

(一)根据行号筛选单行信息

[root@centos71 ~]# sed  -n   3p‘   /etc/selinux/config
# SELINUX= can take one of these three values:

 

 

 

 

 

(二)根据行号筛选多行信息

连续的多行信息

[root@centos71 ~]# sed  -n   2,4p‘   /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.

 

 

 

 

 

不连续的多行信息

[root@centos71 ~]# cat  -n /etc/selinux/config
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=enforcing
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 ~]#  sed  -n   1p;3p;5p  /etc/selinux/config

# SELINUX= can take one of these three values:
#     permissive - SELinux prints warnings instead of enforcing.

 

 

 

 

 

 

 

(三)根据字符筛选单行信息   

 

/ /这种结构和搜索替换是类似的,搜索替换包含了搜索

搜索替换sed   ‘s///g‘

[root@centos71 test]# sed   -n  /dis/p   selinux.txt 
#     disabled - No SELinux policy is loaded.
SELINUX=disabled

 

 

 

 

 

 

(四)根据字符筛选多行信息

 

连续的多行信息

筛选从dis开头,target结尾的行

[root@centos71 test]# cat  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

[root@centos71 test]# sed   -n  /dis/,/target/p   selinux.txt 
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
[root@centos71 test]# 

 

 

 

 

 

不连续的多行信息

[root@centos71 ~]# sed  -n  /dis/p;/en/p‘  /etc/selinux/config  
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled

 

 

 

 

 

 

区间范围查找,搜索的起始字符一定要的,结束字符不一定要

有头无尾可以(壁虎),有尾无头不可以,直接死了

是贪婪匹配,每行进行过滤

 

 

 

 

 

 

 

四sed命令删除操作

 

 

(一)根据行号进行单行删除

 

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   6d  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

(二)根据行号进行多行删除

 

连续多行

 

注意操作是在内存的模式空间操作,对磁盘的block操作才真正操作

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   1,11d  selinux.txt
SELINUXTYPE=targeted

 

 

 

 

 

 

不连续多行

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   7d;12d  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.

 

 

 

 

 

 

 

五sed命令添加操作

 

(一)根据行号单行添加信息

 

巧记:a表示add+after

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   1a  hahahah   selinux.txt 

hahahah
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

(二)根据行号多行添加信息

 

连续多行

[root@centos71 ~]# cat  -n  /etc/issue  
     1    \S
     2    Kernel \r on an \m
     3    
[root@centos71 ~]# sed  1,2aHAPPY‘  /etc/issue
\S
HAPPY
Kernel \r on an \m
HAPPY

 

 

 

 

 

 

不连续多行

 

要加-e

[root@centos71 ~]# cat -n  /etc/issue
     1    \S
     2    Kernel \r on an \m
     3    
[root@centos71 ~]# sed  -e  1aHAPPY‘   -e  2aHAPPY‘    -e  3aHAPPY‘  /etc/issue
\S
HAPPY
Kernel \r on an \m
HAPPY

HAPPY

 

 

 

 

 

 

 

 

在指定行后面添加多行信息

 

[root@centos71 test]# sed   1a  hahaha\nxixixi   selinux.txt 

hahaha
xixixi
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

在指定行前面插入信息

 

i插入,联系vim的i就是插入

[root@centos71 test]# sed   1i  hahaha\nxixixi   selinux.txt 
hahaha
xixixi

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

企业常见配置文件添加信息方法

 

在文件的最后一行添加内容

$表示最后一行;在最后一行后面附加新的信息

 

[root@centos71 ~]# cat -n  /etc/issue
     1    \S
     2    Kernel \r on an \m
     3    
[root@centos71 ~]# sed   $ahahaxixi‘  /etc/issue
\S
Kernel \r on an \m

hahaxixi

 

 

 

 

 

 


基础命令是最低级别的,不能识别正则符号\n\t,会认为是字符

grep只能识别基础正则

sed只能识别基础正则

awk统计分析命令,是高级命令,可以识别基础正则和扩展正则

这种情况类似于高级语言和机器语言

 

 

 

 

 

 

六sed命令修改操作

 

(一)根据单行信息搜索替换

[root@centos71 test]# sed  7s#disabled#enforcing#g  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

(二)利用正则将信息匹配出来再做替换

 

 

[root@centos71 ~]# cat  -n  /etc/selinux/config
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 ~]# sed 2,6s/#/abc/g‘   /etc/selinux/config

abc This file controls the state of SELinux on the system.
abc SELINUX= can take one of these three values:
abc     enforcing - SELinux security policy is enforced.
abc     permissive - SELinux prints warnings instead of enforcing.
abc     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

 

 

(三)利用后向引用前项做替换——修改网卡地址

 

 

[root@centos71 ~]# sed -r /IPADDR/s#(.*=).*#\1192.168.111.200#g‘ /etc/sysconfig/network-scripts/ifcfg-eth0 
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
UUID=9d9e2656-f3ac-4f75-8722-3136d239985d
DEVICE=eth0
ONBOOT=yes
IPADDR=192.168.111.200
PREFIX=24
GATEWAY=10.0.0.254
IPV6_PRIVACY=no
DNS1=223.5.5.5

 

 

 

 

 

 

 

 

练习——sed批量创建用户信息test01 .. test10并设置随机密码(6位)

 
 

 

(一)创建用户

[root@centos71 ~]# seq  -w  10
01
02
03
04
05
06
07
08
09
10

 

 

 

 

 

[root@centos71 test]# seq -w   10  |  sed  -r   s#(.*)# useradd  test\1#g
 useradd  test01
 useradd  test02
 useradd  test03
 useradd  test04
 useradd  test05
 useradd  test06
 useradd  test07
 useradd  test08
 useradd  test09
 useradd  test10

 

 

 

 

 

(二)设置随机密码

[root@centos71 ~]# # seq -w 10|sed -r s#(.*)#useradd test\1;echo  password  | passwd --stdin test\1#g

 

 

 

 

 

 

(三)保存随机密码

 注意因为设置的是随机密码,所以要使用变量

[root@centos71 ~]# seq  -w 10 |  sed -r  s#(.*)#useradd test\1;passwd=$(tr -cd ‘a-zA-Z0-9 </dev/urandom|head -c 6);
echo $passwd | passwd --stdin test\1; echo test\1 $passwd >>/tmp/user_passwd#g

 

 

 

 

 

 

 

 

练习——实现批量修改文件名称

 

 


将文件名称扩展名改为jpg

问题我的操作可以吗?

[root@centos71 test]# touch   happy{01..10}.txt
[root@centos71 test]# ls  happy*
happy01.txt  happy03.txt  happy05.txt  happy07.txt  happy09.txt  happy.txt
happy02.txt  happy04.txt  happy06.txt  happy08.txt  happy10.txt
[root@centos71 test]# ls  happy*  |  sed   -r  s#(.*\.)txt#\1jpg#g 
happy01.jpg
happy02.jpg
happy03.jpg
happy04.jpg
happy05.jpg
happy06.jpg
happy07.jpg
happy08.jpg
happy09.jpg
happy10.jpg
happy.jpg

 

 

 

 

 

[root@centos71 test]# sed   -r  s#(.*\.)txt#\1jpg#g  happy.txt
happy01.jpg
happy02.jpg
happy03.jpg
happy04.jpg
happy05.jpg
happy06.jpg
happy07.jpg
happy08.jpg
happy09.jpg
happy10.jpg

 

 

 

 

 

法二

[root@centos71 test]# ls  happy*
happy01.txt  happy03.txt  happy05.txt  happy07.txt  happy09.txt  happy.txt
happy02.txt  happy04.txt  happy06.txt  happy08.txt  happy10.txt
[root@centos71 test]# ls  happy* | sed   -r  s#(.*)txt#mv \1txt \1jpg#gmv happy01.txt happy01.jpg
mv happy02.txt happy02.jpg
mv happy03.txt happy03.jpg
mv happy04.txt happy04.jpg
mv happy05.txt happy05.jpg
mv happy06.txt happy06.jpg
mv happy07.txt happy07.jpg
mv happy08.txt happy08.jpg
mv happy09.txt happy09.jpg
mv happy10.txt happy10.jpg
[root@centos71 test]# ls  happy* | sed   -r  s#(.*)txt#mv \1txt \1jpg#g‘    | bash
[root@centos71 test]# ls  happy*
happy01.jpg  happy03.jpg  happy05.jpg  happy07.jpg  happy09.jpg  happy.txt
happy02.jpg  happy04.jpg  happy06.jpg  happy08.jpg  happy10.jpg

 

 

 

 

 

 

 

 

练习——利用sed命令取出IP地址

 

(一)取出有IP地址所在行

[root@centos71 ~]#   ip a s eth0|sed -n 3p
    inet 10.0.0.200/24 brd 10.0.0.255 scope global noprefixroute eth0

 

 

 

 

 

(二)取出IP地址

将前面/后面减掉,简言之掐头去尾留中间

[root@centos71 ~]# ip a  s  eth0  |    sed   -n  3p‘ |   sed   -r  s#.* (.*)\/.*#\1#g10.0.0.200

 

 

 

 

巧记:

双引号比单引号更多,作用更大,比如可以调用变量

 

 

 

 

 

 

七sed命令使用忠告

 

 

(一)sed操作时候要做备份

 

真正删除

 -i[SUFFIX], --in-place[=SUFFIX]
                 edit files in place (makes backup if SUFFIX supplied)

-i---将模式空间操作影响到磁盘中也就是将模式空间信息--覆盖--磁盘文件中

在编辑文件时,直接多文件进行备份sed   -i.bak

注意sed命令在使用时用到多个参数,要将-i参数写在所有参数后面

下面对文件进行搜索替换的同时也进行了备份

[root@centos71 test]# cat  -n /etc/selinux/config 
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15
[root@centos71 test]#  sed  -ri.bak    7s#disabled#enforcing#‘   /etc/selinux/config
[root@centos71 test]# ls  /etc/selinux/config*
/etc/selinux/config  /etc/selinux/config.bak
[root@centos71 test]# ls  /etc/selinux/config* -l
-rw-r--r-- 1 root root 543 Dec 19 19:43 /etc/selinux/config
-rw-r--r-- 1 root root 542 Dec 17 16:59 /etc/selinux/config.bak
[root@centos71 test]# diff   /etc/selinux/config*
7c7
< SELINUX=enforcing
---
> SELINUX=disabled

 

 

 

 

 

 

(二)sed在使用参数时不要一起使用参数-i、-n

 

如果做了第1步,那么第2步做了还可以恢复,但是对企业还是会有损失

所以小心驶得万年船

文本处理三剑客之sed

原文:https://www.cnblogs.com/wang618/p/12069956.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!