calculate total file size.

Sum up file size for specific type of files.

 $ find ./<directory_name>  -print | grep -E  "mp4$|wmv$|avi$|jpg$|png$|mp3$|mkv" | awk '{print "ls  -l \""$0"\""}' > <tmporary_file_name>

The content of size_t_file is as below

ls -l "./<directory_name>/<subdirectory_name>/001.jpg"
ls -l "./<directory_name>/<subdirectory_name>/002.jpg"

Then

$ sh <tmporary_file_name> | awk '{total = total + $5}END{print "total is "total}'

This will count all designated type's file size and calculate the summation.

$find ./<directory_name>  -print | grep -vE  "mp4$|wmv$|avi$|jpg$|png$|mp3$|mkv$|JPG$|MP4$|AVI$"

The command above will omit all output which ends with motion and still picture file type. Use option "-v" for this purpose.

Tracking US GDP.

This is a memo to track US GDP forecast and actuals.


> last(GDPC96,n=12)
             GDPC96
2013-10-01 15793.93
2014-01-01 15747.05
2014-04-01 15900.78
2014-07-01 16094.45
2014-10-01 16186.74
2015-01-01 16268.98
2015-04-01 16374.18
2015-07-01 16454.88
2015-10-01 16490.68
2016-01-01 16524.99
2016-04-01 16570.17

GDPnow predicts 2.0% annual growth for 3rdquater. Then 3Q GDP is as below.

> 16570.17 * 1.005
[1] 16653.02

When Q3 GDP data is released, Q2 is also revised upward. Q3 growth was 2.9%.

> last(GDPC96,n=4)
             GDPC96

2015-10-01 16490.68
2016-01-01 16524.99
2016-04-01 16583.15
2016-07-01 16702.12

The Great Game

1)前半と後半にわけると前半のほうが圧倒的に面白い。
2)というわけで登場人物も前半のほうが魅力的である。しかし、彼らが何に見せられて中央アジアを探訪したのか、読了してもわからないままである。危険(殺されたり、投獄されたり、奴隷として売られたり)とみあうだけの報酬が保証されていたとは思えない。。。。。。書中には簡単には触れられているけど異教徒の自分にはあまり納得が行かない。
3)第一次アフガン戦争の当たりまでは英国がアフガン土着部族に対してテクノロジー的な優位性を持っていたわけではなかった。海はないから海軍は呼べないし、輸送インフラは皆無に等しいから火砲も自由に使えない。双方先込め銃同士で戦うわけだけど、イギリスの戦列歩兵の銃は部族のそれと比較して銃身が短いので射程も短かかった。とかとか。
4)後半になると優越性がハッキりするので、面白くなくなるといえる。
5)鉄道。マッキンダーは鉄道の出現をもって世界島の成立とランドパワーの優越を主張するわけだが本書の隠れたテーマは鉄道であるといっていい。それは大陸の東の端までつながるシベリア鉄道も含む。日露戦争がGreat Gameを終わらせたのもある意味当然。
6)ロシア側の登場人物はかなりが悲劇的な最後を迎えている。だから恐ろしい、にも関わらず恐ろしいとすべきか。
7)おなじく、素朴で好戦的なことにびっくりする。

function for S&P 500 forecast.

my_sp5 is the most simple one. Given GDP, Payroll and Housing under construction, the function returns the theoretical value for S&P 500.

my_sp5 <- function(x,y,z)
{
  # x <- GDP  :like 17556
  # y <- PAYEMS :like 150000
  # z <- UNDCONTSA :like 1037
  # 2.371e+04+y*-2.615e-01+z*-5.720e+00+x*-1.707e+00+y*z*1.590e-04+y*x*1.926e-05+z*x*-5.615e-04+y*z*x*-3.970e-09
  #
  # start_date <- "1992-01-01"
  # end_date <-  "2016-04-01"
  # please see > (summary(lm(to.quarterly(SP5[paste(start_date,end_date,sep='::')])[,4] ~
  #                to.quarterly(PAYEMS[paste(start_date,end_date,sep='::')])[,4] *
  #                 to.quarterly(UNDCONTSA[paste(start_date,end_date,sep='::')])[,4] *
  #                  GDPC96[paste(start_date,end_date,sep='::')])))
  2.805e+04+y*-2.959e-01+z*-1.095e+01+x*-2.02e+00+y*z*2.007e-04+y*x*2.174e-05+z*x*-1.949e-04+y*z*x*-6.888e-09
}

This will be given the length of terms of forecast as the parameter. If 12 is given, it will return values for next 12 quarters. GDP, payroll and housing data are derived from VAR model v_GPC_q_1992_2016.

my_sp5_fc <- function(x)
{
  # x <- span of forecast by # of quarter. 4 = 1 yr, 8= 2yrs.
   print(my_sp5(predict(VAR(v_GPC_q_1992_2016,lag.max=10),n.ahead=x)$fcst$GDPC96[,1],predict(VAR(v_GPC_q_1992_2016,lag.max=10),n.ahead=x)$fcst$PAYEMS[,1],predict(VAR(v_GPC_q_1992_2016,lag.max=10),n.ahead=x)$fcst$UNDCONTSA[,1]))
   # seq(as.Date(Sys.Date()),as.Date(last(seq(mondate(Sys.Date()), by=1, length.out=x*3))),
   # plot(seq(as.Date(Sys.Date()),as.Date(last(seq(mondate(Sys.Date()), by=1, length.out=x*3))), by="quarters"),my_sp5(predict(VAR(v_GPC_q_1992_2016,lag.max=10),n.ahead=x)$fcst$GDPC96[,1],predict(VAR(v_GPC_q_1992_2016,lag.max=10),n.ahead=x)$fcst$PAYEMS[,1],predict(VAR(v_GPC_q_1992_2016,lag.max=10),n.ahead=x)$fcst$UNDCONTSA[,1]),ylab="",xlab="s&p500")
 }

This function is extended to accept another parameter for VAR data structure. The limit lies where the structure must have GDPC96,PAYEMS and UNDCONSTA as members and, other than those 3 will be discarded.

my_sp5_fc <- function(x,var_p)
{
  # x <- span of forecast by # of quarter. 4 = 1 yr, 8= 2yrs.
  # VAR structre like v_GPC_q_2001_2016
  print(my_sp5(predict(VAR(var_p,lag.max=10),n.ahead=x)$fcst$GDPC96[,1],predict(VAR(var_p,lag.max=10),n.ahead=x)$fcst$PAYEMS[,1],predict(VAR(var_p,lag.max=10),n.ahead=x)$fcst$UNDCONTSA[,1]))
  # seq(as.Date(Sys.Date()),as.Date(last(seq(mondate(Sys.Date()), by=1, length.out=x*3))),
  plot(seq(as.Date(Sys.Date()),as.Date(last(seq(mondate(Sys.Date()), by=1, length.out=x*3))), by="quarters"),my_sp5(predict(VAR(var_p,lag.max=10),n.ahead=x)$fcst$GDPC96[,1],predict(VAR(var_p,lag.max=10),n.ahead=x)$fcst$PAYEMS[,1],predict(VAR(var_p,lag.max=10),n.ahead=x)$fcst$UNDCONTSA[,1]),ylab="",xlab="s&p500")
}

Please see below as a sample output for next 36 quarters.

 my_sp5_fc(36,v_GPC_q_2001_2016)
 [1] 2169.916 2230.324 2306.517 2374.391 2442.389 2482.522 2494.811 2514.468 2513.888 2503.617 2500.096
[12] 2514.607 2536.827 2573.543 2628.566 2683.178 2727.886 2772.437 2806.057 2829.112 2850.500 2874.021
[23] 2898.234 2928.297 2968.731 3010.476 3053.568 3097.477 3138.014 3174.560 3208.802 3241.897 3273.784

[34] 3308.672 3349.468 3396.346

S&P 500 model from 1992-01-01 to 2016-04-01

Red line shows actual, black one for the forecast. The graph starts from 1992-01-01 and ends at 2016-04-01 on a quarterly basis.


start_date <- "1992-01-01"
end_date <-  "2016-04-01"
(summary(lm(to.quarterly(SP5[paste(start_date,end_date,sep='::')])[,4] ~
              to.quarterly(PAYEMS[paste(start_date,end_date,sep='::')])[,4] *
                   to.quarterly(UNDCONTSA[paste(start_date,end_date,sep='::')])[,4] *
                    GDPC96[paste(start_date,end_date,sep='::')])))

Below shows how to draw the graph.

plot(to.quarterly(SP5[paste(start_date,end_date,sep='::')])[,4],col=2,ylim=c(0,2500))

par(new=T)

plot(seq(as.Date(start_date),as.Date(end_date),by="quarter"),predict(lm(to.quarterly(SP5[paste(start_date,end_date,sep='::')])[,4] ~
       to.quarterly(PAYEMS[paste(start_date,end_date,sep='::')])[,4] *    

           to.quarterly(UNDCONTSA[paste(start_date,end_date,sep='::')])[,4] *
GDPC96[paste(start_date,end_date,sep='::')])),ylim=c(0,2500),ylab="",type='l')

New GDP VAR model based on quarterly data.

New S&P 500 forecast is based upon quarterly data. Therefore new quarterly data based VAR model for GDP is required.

v_GPC_q_1992_2016 <- merge(GDPC96["1992::2016-04-01"],to.quarterly(UNDCONTSA["1992::2016-04-01"])[,4])
v_GPC_q_1992_2016 <- merge(v_GPC_q_1992_2016,to.quarterly(PAYEMS["1992::2016-04-01"])[,4])
names(v_GPC_q_1992_2016)[2] <- "UNDCONTSA"
names(v_GPC_q_1992_2016)[3] <- "PAYEMS"
VARselect(v_GPC_q_1992_2016)
$selection
AIC(n)  HQ(n)  SC(n) FPE(n) 
    10      2      2     10 

$criteria
                  1            2            3            4            5            6            7            8            9
AIC(n) 2.789256e+01 2.551374e+01 2.550815e+01 2.549235e+01 2.552063e+01 2.551927e+01 2.558270e+01 2.561892e+01 2.537707e+01
HQ(n)  2.802866e+01 2.575191e+01 2.584840e+01 2.593467e+01 2.606502e+01 2.616573e+01 2.633124e+01 2.646954e+01 2.632976e+01
SC(n)  2.823038e+01 2.610492e+01 2.635270e+01 2.659026e+01 2.687190e+01 2.712390e+01 2.744070e+01 2.773029e+01 2.774180e+01
FPE(n) 1.299168e+12 1.204798e+11 1.200424e+11 1.185840e+11 1.226860e+11 1.235582e+11 1.332081e+11 1.403119e+11 1.124435e+11
                 10
AIC(n) 2.525669e+01
HQ(n)  2.631145e+01
SC(n)  2.787478e+01
FPE(n) 1.023095e+11

Then "lag.max" parameter for the new model is 10.

predict(VAR(v_GPC_q_1992_2016,lag.max=10))
my_sp5(predict(VAR(v_GPC_q_1992_2016,lag.max=10))$fcst$GDPC96[,1],predict(VAR(v_GPC_q_1992_2016,lag.max=10))$fcst$PAYEMS[,1],predict(VAR(v_GPC_q_1992_2016,lag.max=10))$fcst$UNDCONTSA[,1])

private function "my_sp5" will be explained in the  future post .

MySQL-R connection

This is the memo, which might be replaced by more comprehensive ones.

Install R package.

DBI  

RMySQL 

Homebrew package for OS

mariadb-connector-c

After installing mariadb-connector-c, it might not be able to install homebrew mysql package. The process ends abnormally with the message to unlink mariadb-connector-c.

samples codes.

con <- dbConnect(MySQL(), host="10.251.66.58", port=3306, dbname="DB_NAME", user="USER_ID", password="PASSWORD")
supportdataset <- dbGetQuery(con,"SELECT * FROM csupportdata")

replace DB_NAME,USER_ID and PASSWORD with appropriate ones.

Install android SDK and read AndroidManifest.xml

install java for brew environment.

$  brew cask install java

install android-sdk.

$ brew install android-sdk

install acutal sdk. this command will kick GUI.

$ android

add " export ANDROID_HOME=/usr/local/opt/android-sdkadd " into .bashrc.

$ aapt l -a KES10_10.5.113.1539_Release.apk 

appt failed with the message  "appears you do not have 'build-tools-23.0.1' installed." 

$ android list sdk -a | grep Build

   3- Android SDK Build-tools, revision 24.0.2

   4- Android SDK Build-tools, revision 24.0.1

   5- Android SDK Build-tools, revision 24

   6- Android SDK Build-tools, revision 23.0.3

   7- Android SDK Build-tools, revision 23.0.2

   8- Android SDK Build-tools, revision 23.0.1

$ android update sdk -a -u -t 8

will install "build-tools-23.0.1". "android update sdk -a -u -t 3" will install version 24.0.2 instead.

$ aapt d xmltree KES10_10.5.113.1539_Release.apk AndroidManifest.xml > manifest_ascii.xml

will convert  AndroidManifest.xml into human-readable "manifest_ascii.xml".