ãã®æçš¿ã¯ãPOSIXåç䞻矩è 以å€ã«ã¯äœã®äŸ¡å€ããªã倿 èšäºã ããŸããGNU AWKãzshãåžžçšããŠãã人ã«ã䟡å€ããªãã以äžã«è©²åœããæ¹ã¯ãé»ã£ãŠãã®èšäºãéãããã
ã·ã§ã«ã¹ã¯ãªããã¯ãã€ããªããŒã¿ãèªã¿æžãã§ããªã
POSIXåç䞻矩è ã§ããç§ã¯ãBase64ãšã³ã³ãŒã/ãã³ãŒããããããã«POSIXæšæºã§ã¯ãªãbase64ã³ãã³ãã«äŸåããã®ãå«ã ã£ããããã§ãPOSIXæšæºã³ãã³ãã ãã§äœã£ãŠããŸããšæã£ãã®ã ããããã«ã¯å€§ããªå£ãç«ã¡ã¯ã ãã£ãã
ã·ã§ã«ã¹ã¯ãªããã¯ãã€ããªããŒã¿ãå å·¥ã§ããªãã®ã§ãããå šç¶æ±ããªãã®ã§ã¯ãªãããŸãšãã«æ±ããªããšèšãã¹ããããããªããâŠâŠã
Base64ã³ãŒããã¯ãäœããããã°ããã€ããªããŒã¿ã®èªã¿æžããã§ããªããã°ãªããªãããšãããæ¬¡ã®ã³ãã³ããææã¡ã®ç°å¢ã§å®è¡ããŠã¿ãã°ãŸãšãã«æ±ããªãããšããããããã
$ printf 'This is \000 a pen.\n' | awk '{print $0;}'
This is # nullçµç«¯æ±ããããŠãa pen.ãç¡ãã£ãããšã«âŠOTL
$
$ str=$(printf 'abc\000def')
$ echo "$str"
abcdef # ããããŸããã£ãã?
$ echo ${#str}
6 # ãããnullãåãé€ãããŠ6æåã«ãªã£ãŠãâŠorz
$
ã€ãŸããAWKãã·ã§ã«ãã倿°ã«null(0x00)ãæ ŒçŽã§ããªããšããããšã ããã ãããããŸã§ãäœåãèšããŠããããã«GNUçAWKãšzshã¯ã§ããã®ã ãâŠâŠã
äž»ãªã³ãã³ãã®ãã€ããªããŒã¿å¯Ÿå¿ç¶æ³ïŒç¬èªèª¿æ»ïŒ
ããã§çŸç¶ãæŽçãããããå šç¶æ±ããªããã§ã¯ãªããããŸãšãã«ããšèšãçŽããããã³ãã³ãã«ãã£ãŠæ±ãããã®ãããã
| ãã€ããªããŒã¿åãæ±ãäžå¯ | ãã€ããªããŒã¿åãæ±ãå¯ |
|---|---|
| awk bash cut echo grep sed sh sort xargs | awk(GNUç) cat dd head od printf tail tr wc |
å®è£ ã«ãã£ãŠãéãã ããããã ããããããªæãã ãšæãããã®äž»ãªåå ã¯ãããŒã¿ãè¡ãšããåäœã«åŠçããèšèšã§ãããã©ããã«ãããããäž»èŠãªã³ãã³ããé察å¿ãããªãããããã ãã ãŒãä»èšèªã«ä¹ãæãããã
ãããPOSIXåç䞻矩è ã¯ãããªããšã§è«ŠããŠã¯ãããªãïŒé察å¿ã³ãã³ãã¯ããã察å¿ã³ãã³ãããããããªããïŒ
ãã€ããªããŒã¿å¯Ÿå¿ã³ãã³ãã§åããŠãããã¹ãããŒã¿ã§åŠçãããã€ããªããŒã¿ã§åãåºããŠããã°ããïŒ
ãšããããã§ãã®æ¹éã«åºã¥ããPOSIXåç䞻矩ã貫ããªãããã€ããªããŒã¿ããã°ãããŠããŠãæãããããããŠèªäœã®catãbase64ã³ãã³ãããã£ãŠããã®ããŠããŠãå£ã ãã§ã¯ãªãããšã瀺ãã
Part1. ãã€ããªããŒã¿ãåãåãã«ã¯
ãããŸãã¯ããŒã¿ã®åãåãæ¹ã ãã§ãããã¯ç°¡åãodã³ãã³ãã§ãã€ããªããŒã¿ããã³ãããŠãããšã¯éœåããå å·¥ããã°ããã
odã³ãã³ãã«æž¡ãã¹ããªãã·ã§ã³
ãã³ãããšããšæ®éã¯ã¢ãã¬ã¹ãä»ããŠãããããããçŽç²ãªããŒã¿ã¹ããªãŒã ã«ããããªãã¢ãã¬ã¹ãé衚瀺ã«ãããªãã·ã§ã³-A nãã€ããŠãããšããã ããã
ãŸãããã¡ã€ã«ãæšæºå
¥åºåã®ãã€ããªããŒã¿ã®æå°åäœã¯ãã€ãã§ãããããã³ãã¯1ãã€ãåäœã«ãªãããã«ãã¹ãã ãããŸãç¡é§ã«ããã¹ããã¡ã€ã«ãµã€ãºãå¢ãããŠããã©ãŒãã³ã¹ãäžããªãããã«ã¯16é²è¡šçŸã§åºåããæ¹ãããã ããããã®å Žå-t x1ãšãããªãã·ã§ã³ãã€ããã
æåŸã«ã-vãšãããªãã·ã§ã³ãä»ããŠããããšãããããªããšåäžã®å€ãããçšåºŠé£ç¶ããå Žåã«è¡šç€ºãã¯ãããããšãããããã ã
ãŸãšãããšããã€ããªããŒã¿ãã·ã§ã«ã¹ã¯ãªããã®æåã«ã¯æ¬¡ã®1è¡ãæžããšããã
$ od -A n -t x1 -v <ã³ã³ã«ãã¡ã€ã«å>
trã³ãã³ãã§éœåã®ããããã¹ãããŒã¿åã«
odã³ãã³ãã®åºåã¯ã¹ããŒã¹ãæ¹è¡ãªã©ã§äººéåãã«ã¬ã€ã¢ãŠããããŠããããæ©æ¢°åŠçã«ã¯ç¡é§ãªã®ã§æ¬¡ã«ããããšã¯ç¡é§ãªæååãåãäœæ¥ã ãããã«ã¯trã³ãã³ãã䜿ããšããã
å¿ èŠãªæåã¯"0"ïœ"9"ãš"a"ïœ"f"ã®16çš®é¡ã§ããã®ã§ããã®ä»ã¯åãé€ããããããã°ãåŸç¶ã®ã³ãã³ãã§ã¯2æåãã€åŠçããŠããã°æžãããã«ãªãã
ãã ãæ°ãã€ããªããã°ãªããªãã®ã¯ãæ¹è¡ã³ãŒããç¡éã«é€å»ããŠã¯ãããªããšããããšãsedãAWKãgrepãªã©ã®ããã¹ãåŠçã³ãã³ãã®åŠçã¯è¡åäœãåºæ¬ã§ããããããšã«ãã1è¡ïŒæ¹è¡ã³ãŒããåºãŠããããã¡ã€ã«çµç«¯ã«éããããããŸã§ïŒäžžããšèªã¿èŸŒãŸãªããšäœæ¥ãããªãã®ã ãã ããtrã§æ¹è¡ã³ãŒããŸã§åãé€ããŠããŸããšã次ã®ããã¹ãåŠçç³»ã³ãã³ãã¯å šéšèªã¿çµãããŸã§åŸç¶ã®ã³ãã³ãã«ããŒã¿ãæž¡ããããã€ããè©°ãŸã£ãŠããŸããUNIXããã䞊ååŠçãå¹ããªããªããããã¡ã¢ãªããã«å°ãããã
ãšããããã§ãodã®æ¬¡ã«ç¹ããtrã³ãã³ãã¯ã次ã®ããã«ããã®ãããã ããã
tr -Cd '0123456789abcdef\n'
ãšããã§ãªãæåã®ç¯å²ã'0-9a-f'ãšæžããªãã®ããšãããšãããã¯System Vç³»ã®trãšBSDç³»ã®trã§æžåŒãéãã坿¬æ§ãæãªãããŠããŸãããã ãPOSIXã§ã¯åŸè
ã®ä»æ§ãèŠå®ãããŠããããPOSIXåç䞻矩ã¯å¯æ¬æ§ç¢ºä¿ã®ããã«è²«ããŠããã®ã§ãPOSIXå€ã®ä»æ§ã ãããšãã£ãŠç°¡åã«åãæšãŠã¯ããªãã®ã ã
ããšã¯ç ®ããªãçŒããªã
ãããŸã§ããã°ããäœãå°ãããšã¯ç¡ãã®ã§ãsedãªãawkãªã奜ããªã³ãã³ãã䜿ã£ãŠç ®ããªãçŒããªãæãã®ãŸãŸãããã2æåïŒ1ãã€ãïŒãã€æ¹è¡ããŠãããããšãããªãtrã®æ¬¡ã«äžã®ãŸãšãã«ç€ºããŠãããããªsedãæžããšããã ãããã¡ãªã¿ã«LFãšãã倿°ãä»ããŠsedã綺éºã«æžããã¯ããã¯ïŒsedã³ãã³ãã§æååãæ¹è¡ã«çœ®æããããããã¹ããŒãã«çœ®æããïŒãå©çšããŠããã
LF=$(printf '\\\n_'); LF=${LF%_}
od -A n -t x1 -v <ã³ã³ã«ãã¡ã€ã«å> |
tr -Cd '0123456789abcdef\n' |
sed "s/../&$LF/g"
Part2. ãã€ããªããŒã¿ãæžãåºãã«ã¯
次ã¯åºåãããã¯çµæ§é¢åã ããprintfçšã®ãã©ãŒãããã§åºåæååãçæããprintfã«çæããããšããããæ¹ããããããã§ããprintfã¯AWKã®printf颿°ã§ã¯ãªãã³ãã³ãã®printfã ãïŒGNUçã¯å¥ã ããAWKã®printfã¯nullãåºåã§ããªãïŒ
AWKã§äžéšæåããšã¹ã±ãŒãããŠæ±ã
AWKã®å Žåãnull(0x00)ãèªåã§åºåããããšãã§ããªããããprintfã³ãã³ããè§£éã§ããåœ¢ã®æååãšããŠåºåããªããã°ãªããªãã
倧倿°ã®æåã¯ãã®ãŸãŸåºåã§ããããäžéšã¯printfã§æ¹å€ãåããŠããŸãããäžéšã¯printfã«äŒããåã«æ¹å€ãåããŠããŸãã®ã§ããããã£ãæåã¯å šãŠãšã¹ã±ãŒãããŠæž¡ããªããã°ãªããªããäžèšã®è¡šã«ãã®äžèЧã瀺ãã
| æå | AWKå ã§ã®è¡šçŸ | æååé· | ãšã¹ã±ãŒãããçç± |
|---|---|---|---|
% |
"%%" |
2 | printfã®ã¡ã¿æåã ãã |
\ |
"\\\\\\\\" |
4 | printfãã·ã§ã«ã®ã¡ã¿æåã ãã |
| null | "\\\\000" |
5 | ããã¹ããšããŠè¡šçŸã§ããªããã |
| LF | "\\\\n" |
3 | ã·ã§ã«ã«åºåãæåæ±ããããŠããŸããã |
| CR | "\\\\r" |
3 | ã·ã§ã«ã«åºåãæåæ±ããããŠããŸããã |
| ã¿ã | "\\\\t" |
3 | ã·ã§ã«ã«åºåãæåæ±ããããŠããŸããã |
| åçŽã¿ã | "\\\\v" |
3 | ã·ã§ã«ã«åºåãæåæ±ããããŠããŸããã |
| æ¹ããŒãž | "\\\\f" |
3 | ã·ã§ã«ã«åºåãæåæ±ããããŠããŸããã |
| åè§ç©ºçœ | "\\\\040" |
5 | ã·ã§ã«ã«åºåãæåæ±ããããŠããŸããã |
" |
"\\\"" |
2 | ã·ã§ã«ã«åŒæ°ã®å²ã¿æåæ±ããããŠããŸããã |
' |
"\\'"'"'" |
2 | ã·ã§ã«ã«åŒæ°ã®å²ã¿æåæ±ããããŠããŸããã |
- |
"\\\\055" |
5 | ãã®æåãå é ã«æ¥ããšprintfã«ãªãã·ã§ã³æ±ããããŠããŸããã |
0~9
|
"\\\\060"~"\\\\072"
|
5 | çŽåã«\0ooïŒooã¯2æ¡ã®8鲿°ïŒãšããæååããããšäžéšç°å¢ïŒMacOS XçïŒã§èª€å€æãããŠããŸããã |
å°ããã®ãšã¹ã±ãŒããåæ ãããµã³ãã«ã³ãŒãããã®æçš¿ã®æåŸã«äŸç€ºããã
xargsçµç±ã§æž¡ãããã®å·¥å€«
printfã³ãã³ãã¯ãæ®å¿µãªãããã©ãŒãããæååãæšæºå ¥åããåãåãããšãã§ããªãããããªããšåŒæ°ãšããŠæž¡ããããåŸãªããšããããã§xargsã³ãã³ãã®åºçªãªã®ã ããxargsã³ãã³ãã䜿ããšãªã£ããããã€ãå¶çŽãããã
åŒæ°ã®æååé·ãARG_MAXãè¶ ããããªã
äžã€ç®ã®å¶çŽã¯ããã§ããã1è¡ã®ã³ãã³ããšããŠèš±å®¹ãããæååé·(ARG_MAX)ãè¶ ããããã§ããã°ãè¶ ããªãããã«æååãåå²ããprintfãè€æ°åèµ·åããªããã°ãªããªãã
ããã§å çšã®AWKã§åºåããéã«ããšã¹ã±ãŒãããŠããç¶æ ã§ã®æååé·ãèšæ¶ããŠããããã®çޝç©å€ïŒåã³"printf "ãšããã³ãã³ãã®æååé·ã®åèšïŒãARG_MAXãè¶ ããŠããŸãåã«äžæŠæ¹è¡ã³ãŒããåºåããªããã°ãªããªããäžèšã®è¡šã«æååé·ãèšããã®ã¯ããããçç±ã§ããã
å°ãARG_MAXã¯getconfãšããPOSIXã®ç¯å²ã®ã³ãã³ãã§ç°¡åã«èª¿ã¹ãããã
ã ã!!! ã©ãããããããLinuxã§ãFreeBSDã§ããARG_MAXãã4000ãã€ããããå°ããå€ã«ããªããšxargsããšã©ãŒãè¿ãã æ¬¡ã®ã³ãã³ããå®è¡ããŠã¿ãã°ãããããã
$ getconf ARG_MAX
262144 # âFreeBSD 10.1ã ãš262144ãã€ãããã
# ããã§ARG_MAXãã4698ãã€ãå°ãªã"0"ãæµã蟌ããš
$ printf '%0257446d' 0 | xargs echo >/dev/null
xargs: insufficient space for argument # ARG_MAXããååå°ããã®ã«æããã
$
仿¹ãç¡ãã®ã§ç§ã¯ARG_MAXå€ã®ååïŒãããŸã§ããªããŠããããšã¯æããïŒã ããå©çšããããã«ããã
æšæºå ¥åããæž¡ã£ãŠããæååã®æåŸã«åºåãæåãå¿ ãã€ãã
ããã¯ãã©ã®ç°å¢ã§ã䜿ããã·ã§ã«ã¹ã¯ãªãããæžãããã®ã¡ã¢ ãã®äžã§ãæžããããäžéšã®xargså®è£ ã¯æž¡ãããæååã®æ«ç«¯ã«ãã¹ããŒã¹ãæ¹è¡ã®åºåãæåãèŠæ±ãããã®ããããïŒãããç¡ããšãæåŸã®æååã¯ç¡èŠãããïŒ
ããã¯å ·äœçã«èšããšAIXã®xargså®è£ ãªã®ã ããããããç°å¢ãžã®é æ ®ãPOSIXåç䞻矩è ãªãæ ã£ãŠã¯ãªããªãã
ã§ãããã·ã§ã«ã¹ã¯ãªããçcatã³ãã³ã
以äžã®ããŠããŠãé§äœ¿ããPOSIXã®ç¯å²ã§å®è£ ããã·ã§ã«ã¹ã¯ãªããããã¡ããæšæºå ¥åãããã€ããªãã¡ã€ã«ãèªã¿èŸŒãã§äžæŠããã¹ã圢åŒã«å€æããåŸãå ã®ãã€ããªåœ¢åŒã«æ»ãã ãã®ã³ãŒããçµæã ãèŠãã°catã¿ãããªãã®ã ã
# ! /bin/sh
# === ãã€ããªâããã¹ã倿ããŒã ===================================
LF=$(printf '\\\n_'); LF=${LF%_} # 0) æºå
od -A n -t x1 -v | # 1) 16é²ãã³ã(ã¢ãã¬ã¹ç¡ã)
tr -Cd '0123456789abcdefABCDEF\n' | # 2) 空çœã¯ãã«(ã§ãæ¹è¡ã³ãŒãã¯æ®ãã¹ã)
sed "s/../&$LF/g" | # 3) 1ãã€ã1è¡ã«(ããªããŠããããã©)
grep -v '^$' | # sedã§åºæ¥ã空è¡ããã«
#
# === ããŒã¿å å·¥ããŒã ===============================================
#
# (ããã§ç
®ããªãçŒããªããããããã)
#
# === ããã¹ãâãã€ããªå€æããŒã ===================================
awk -v ARGMAX=$(getconf ARG_MAX) ' #
BEGIN{ #
# 0) --- å®çŸ© ------------------------------------------ #
ORS = ""; # åŒæ°ã® #
LF = sprintf("\n"); # æå€§èš±å®¹æååé·ã¯ #
maxlen = int(ARGMAX/2) - length("printf "); # âARG_MAXã®çŽååãšãã #
arglen = 0; #
# 1) --- ãã€ããªå€æçšã®ããã·ã¥ããŒãã«ãäœã -------- #
# 1-1) å
šãŠã®æåãçŽ çŽã«å€æã§ãããã®ãšããŠäžæŠäœã #
for (i=1; i<256; i++) { #
hex = sprintf("%02x",i); #
fmt[hex] = sprintf("%c",i); #
fmtl[hex] = 1; #
} #
# 1-2) çŽ çŽã«ã¯å€æã§ããªãæåããšã¹ã±ãŒã衚çŸã§äžæžããã #
fmt["25"]="%%" ; fmtl["25"]=2; # "%" #
fmt["5c"]="\\\\\\\\"; fmtl["5c"]=4; # (back slash) #
fmt["00"]="\\\\000" ; fmtl["00"]=5; # (null) #
fmt["0a"]="\\\\n" ; fmtl["0a"]=3; # (Line Feed) #
fmt["0d"]="\\\\r" ; fmtl["0d"]=3; # (Carriage Return) #
fmt["09"]="\\\\t" ; fmtl["09"]=3; # (tab) #
fmt["0b"]="\\\\v" ; fmtl["0b"]=3; # (Vertical Tab) #
fmt["0c"]="\\\\f" ; fmtl["0c"]=3; # (Form Feed) #
fmt["20"]="\\\\040" ; fmtl["20"]=5; # (space) #
fmt["22"]="\\\"" ; fmtl["22"]=2; # (double quot) #
fmt["27"]="\\'"'"'" ; fmtl["27"]=2; # (single quot) #
fmt["2d"]="\\\\055" ; fmtl["2d"]=5; # "-" #
for (i=48; i<58; i++) { # "0"~"9" #
fmt[sprintf("%02x",i)]=sprintf("\\\\%03o",i); #
fmtl[sprintf("%02x",i)]=5; #
} #
} #
{ #
# 3) --- åºåæååãprintfãã©ãŒããã圢åŒã§çæ ------ #
linelen = length($0); #
for (i=1; i<=linelen; i+=2) { #
if (arglen+4>maxlen) {print LF; arglen=0;} #
hex = substr($0, i, 2); #
print fmt[hex]; #
arglen += fmtl[hex]; #
} #
} #
END { #
# 4) --- äžéšã®xargså®è£
ãžã®é
æ
®ã§ãæåŸã«æ¹è¡ãä»ãã - #
if (NR>0) {print LF;} #
} #
' |
xargs -n 1 printf
ããåäœã§ã¯å šã圹ã«ç«ããªãããã·ã§ã«ã¹ã¯ãªããã§ãã€ããªããŒã¿ãç ®ããªãçŒããªãããããã°ãã®ã³ãŒããããŒã¹ã«äœããšããããã
ãšããããã§ãã®ããŠããŠãå¿çšããbase64ã¯ããŒã³ã³ãã³ããäœã£ããâãã¡ã
ããã©ãŒãã³ã¹ã¯ãªãªãžãã«ã«æ¯ã¹ããšããªãå£ããã®ã®ãã³ã³ãã€ã«ç¡ãã«ãã³ããŒããã ãã§ãã©ã®ç°å¢ã§ããé·æé䜿ããå®è£ ãã§ããã®ã ã£ãã
ãããŸã