2020年10月9日金曜日

awk tr sed mondate CLI download

1. download cli from this url 

1.1 change downloaded file name to CLI3.csv

1.2 ~/R/R2/index/cli_download.sed  run at the terminal.


sed -n '/USA/p' CLI3.csv |awk -F, '{print $6"-01,"$7}'  |sed 's/\"//g' |awk 'BEGIN{print "DATE,DATA"}{print $0}' > usa.csv

# extract OECD entries and exclude OECDE

sed -n '/OECD[^E]/p' CLI3.csv |awk -F, '{print $6"-01,"$7}'  |sed 's/\"//g' |awk 'BEGIN{print "DATE,DATA"}{print $0}' > oecd.csv

# 

sed -n '/CHN/p' CLI3.csv |awk -F, '{print $6"-01,"$7}'  |sed 's/\"//g' |awk 'BEGIN{print "DATE,DATA"}{print $0}' > chn.csv

# 

sed -n '/EA19/p' CLI3.csv |awk -F, '{print $6"-01,"$7}'  |sed 's/\"//g' |awk 'BEGIN{print "DATE,DATA"}{print $0}' > ea19.csv

#

sed -n '/JPN/p' CLI3.csv |awk -F, '{print $6"-01,"$7}'  |sed 's/\"//g' |awk 'BEGIN{print "DATE,DATA"}{print $0}' > jpn.csv



1.3 run cli_load.r in R.


2.when cli is downloaded form this url 


2.1. do as below in terminal change FILE_NAME accordingly.

grep "OECD - Total" <FILE_NAME>.csv  | grep LOLITOAA |gawk -F, 'BEGIN{print "c("}{print $(NF-2)","}END{print ")  " }' | tr -d "\n" | sed 's/,)/)/'


output is like 


c(100.1448,100.0088,99.88226,99.7745,99.68903,99.62392,99.56622,99.5041,99.44289,99.39536,99.37106,99.38042,99.41794,99.46317,99.49193,99.47968,99.41767,97.69222,93.17708,94.81512,97.03869,98.08919,98.53291,98.79954) 


> length(c(100.1448,100.0088,99.88226,99.7745,99.68903,99.62392,99.56622,99.5041,99.44289,99.39536,99.37106,99.38042,99.41794,99.46317,99.49193,99.47968,99.41767,97.69222,93.17708,94.81512,97.03869,98.08919,98.53291,98.79954) )

[1] 24


# start and end dates are fixed. caution!

seq( as.Date(mondate(Sys.Date())-24)-day(mondate(Sys.Date()))+1,as.Date(mondate(Sys.Date())-1),by='months' )


>seq( as.Date(mondate(Sys.Date())-24)-day(mondate(Sys.Date()))+1,as.Date(mondate(Sys.Date())-1),by='months' )

 [1] "2018-10-01" "2018-11-01" "2018-12-01" "2019-01-01" "2019-02-01" "2019-03-01" "2019-04-01" "2019-05-01" "2019-06-01"

[10] "2019-07-01" "2019-08-01" "2019-09-01" "2019-10-01" "2019-11-01" "2019-12-01" "2020-01-01" "2020-02-01" "2020-03-01"

[19] "2020-04-01" "2020-05-01" "2020-06-01" "2020-07-01" "2020-08-01" "2020-09-01"

 

2.2. do as below in R

as.xts(c(100.1448,100.0088,99.88226,99.7745,99.68903,99.62392,99.56622,99.5041,99.44289,99.39536,99.37106,99.38042,99.41794,99.46317,99.49193,99.47968,99.41767,97.69222,93.17708,94.81512,97.03869,98.08919,98.53291,98.79954), seq( as.Date(mondate(Sys.Date())-24)-day(mondate(Sys.Date()))+1,as.Date(mondate(Sys.Date())-1),by='months' ))


2.3. alternatively do as below 

grep "OECD - Total" <FILE_NAME>.csv  | grep LOLITOAA |gawk -F, 'BEGIN{print "as.xts(c("}{print $(NF-2)","}END{print "), seq( as.Date(mondate(Sys.Date())-24)-day(mondate(Sys.Date()))+1,as.Date(mondate(Sys.Date())-1),by=\"months\" ))  " }' | tr -d "\n" | sed 's/,)/)/'



0 件のコメント: