Monday, October 15, 2018

Scripting Tmux

Tmux has the super power to run your terminal in splitted windows (tmux panes).

It provides sub-commands to enable scripting. For instance, I would like to launch 4 commands with 4 different panes.

# cmd1 | cmd2
# cmd3 | cmd4

tmux new-window 'cmd1'
tmux split-window -h 'cmd2'
tmux select-pane -L
tmux split-window -v 'cmd3'
tmux select-pane -R
tmux split-window -v 'cmd4'

This can also be useful when orchestrating SSH sessions across different hosts, or creating multiple SSH tunnels at a time (don't ask me why I'm doing this).

Happy hacking Tmux!

Saturday, September 22, 2018

vlc chromecast shortcut with ranger file manager

I own a Sony TV which has a nice Chromecast built-in feature. So far I've been using VLC's Chromecast renderer for playing the videos.

Today I took a look at ranger file manager custom commands and wrote a shortcut for it. Just to save my life to avoid a few clickings every time I watch anime.

class tv(Command):
    def execute(self):
        tv = "vlc --sout="#chromecast{ip=192.168.xx.xx}" --demux-filter=demux_chromecast"
        filepath =
        command = '{} "{}"'.format(tv, filepath)

Wednesday, February 14, 2018

Moving Google Drive data via Google Compute Engine

Recently I've been using Google Compute Engine to move some personal data (several TB) from one Google Drive account to another. This post is my experience for moving more than 10TB data.

The rough monthly cost I paid to Google: $75/mo.

Compute Engine Standard Intel N1 1 VCPU running in Americas for 744 hours: $35/mo
Compute Engine Storage PD Capacity: 1024GB Gibibyte-months: $40/mo

Yes, I inserted a 1TB drive for temporary storage so I can move stuff in bulk.

The cool thing is that Google doesn't charge for the egress traffic data from/to Google Drive, which actually saves a lot of money. Saying you got 10TB data to move, according to Google's Internet egress rates, that would be $0.12 * 10 * 1024 = $1228.8 for only the traffic.

I used the Go-written tool called skicka to move my stuff, that I've been using for downloading/uploading files to my Google Drive for a long time.

By default, skicka saves its metadata at ~/.skicka.metadata.cache  and ~/.skicka.tokencache.json, but it's also easy to use a different set of metadata when you have multiple accounts:

skicka -tokencache ~/skic/account1.tokencache.json -metadata-cache-file ~/skic/account1.metadata.cache download /<dir name> <local dir name>
skicka upload <local dir name> <dir name>

I used the commands above to move the files in different pre-organized directories, each has around 500~800GB stuff.

The reason why I'm glad to pay more money for the N1 instance was because of the egress traffic throughput caps. Basically each vCPU has a 2 Gbps egress cap, the more vCPUs you have, the faster the speed would be (up to 16 Gbps). Shared vCPU instances (such as f1-micro or g1-small) are counted as 0.5 vCPU so the egress cap is 1Gbps. This really matters when you have over 10TB data to move around.


My ISP plan has 250Mbps Down / 25Mbps Up. Using Google Compute Engine as the tmp storage just saved lots of time and also avoided the freaking data usage cap enforced by my ISP.

It actually took me several months to move all my data from one Google Drive account to the other (because I was so busy). I really wish I could invest one weekend to write some automation scripts to grab stuff from/to Google Drive but unfortunately I didn't. The total cost ended up to $75*N months but in which I applied some portion of my $300 credits to it, so it turned out not really a bad deal.

Sunday, March 12, 2017

Notes for iproute2, network namespace, and iptables NAT

Some random notes for stuff. Have fun!

# the ip command line tool (iproute2)

# setup an $INTERFACE and bind to $IP/$MASK
# ip link set $INTERFACE up
# ip addr add  $IP/$MASK dev $INTERFACE
# ip route add $IP/$MASK dev $INTERFACE

# delete the above settings
# ip route del $IP/$MASK dev $INTERFACE
# ip addr del $IP/$MASK dev $INTERFACE
# ip link set $INTERFACE down

# static routing
# ip route add default via $GATEWAY dev $INTERFACE
# ip route add $IP dev $INTERFACE

# search devices name
$ ls /sys/class/net/
$ ip link

# flush everything!
# ip addr flush dev $INTERFACE
# ip route flush dev $INTERFACE
# ip link set $INTERFACE down

# evil stuff

# promiscuous mode
# ip link set dev $INTERFACE promisc on

# MAC address spoofing
# ip link set dev $INTERFACE down
# ip link set dev $INTERFACE address XX:XX:XX:XX:XX:XX
# ip link set dev $INTERFACE up

# network namespace (need root permission)

# add and delete the network namespace
# ip netns add $NS_NAME
# ip netns del $NS_NAME

# ip -n shortcut
# ip -n $NS_NAME [ OPTIONS ] OBJECT { COMMAND | help }
# ip netns exec $NS_NAME ip [ OPTIONS ] OBJECT { COMMAND | help }

# run an app under $NS_NAME (remember to downgrade permission)
# ip netns exec $NS_NAME sudo -u $REGULAR_USER $application

# in scripts ...
NS_EXEC="ip netns exec $NS_NAME"

# virtual ethernet interface (create vpn0 outside and peer vpn1 under $NS_NAME)
# ip link add vpn0 type veth peer name vpn1
# ip link set vpn0 up
# ip link set vpn1 netns $NS_NAME up

# finding apps under $NS_NAME, and kill them all!
# ip netns pids $NS_NAME
# ip netns pids $NS_NAME | xargs -n1 -I{} ps -q {} -o pid=,comm=
# ip netns pids $NS_NAME | xargs kill

# sysctl settings to enable forwarding
#   save to /etc/sysctl.conf or /etc/sysctl.d/30-ipforward.conf
# sysctl net.ipv4.ip_forward=1
# sysctl net.ipv6.conf.default.forwarding=1
# sysctl net.ipv6.conf.all.forwarding=1

# iptables stuff

# allow a $PORT from $IP/$MASK for tcp/udp
#   drop: -j DROP
# iptables -A INPUT -p tcp --dport $PORT -s $IP/$MASK -j ACCEPT
# iptables -A INPUT -p udp --dport $PORT -s $IP/$MASK -j ACCEPT

# IP masquerade for NAT
#   -o output interface
#   -s source $IP/$MASK

# DNAT on all protocols from $IP1 to $IP2
#  -d (original) destination IP
#  --to-destination destination IP behind NAT
# iptables -t nat -A PREROUTING -d $IP1 -j DNAT --to-destination $IP2

# specify only tcp/$PORT
#   -p protocol, usually tcp, udp, icmp, icmpv6, or all
#   --to-destination could be a range [ipaddr[-ipaddr]][:port[-port]], remain the same port if not specified
# iptables -t nat -A PREROUTING -p tcp -d $IP1 --dport $PORT -j DNAT --to-destination $IP2

# use MATCH extension, see iptables-extensions(8)
#   Example:
# iptables -t nat -A PREROUTING -i $INTERFACE -p tcp -m tcp --dport $PORT1 -j DNAT --to-destination $IP:$PORT2

# save and restore
# iptables-save
# iptables-restore

# dump current rules
#   -n no DNS lookup
#   -v verbose
#   -L list
#   -t table (default: filter)
#   --line-numbers might be helpful

# iptables -nvL
# iptables -nvL -t nat

# Reference:

ip(8), ip-address(8), ip-link(8), ip-netns(8), ip-route(8)

Friday, March 10, 2017

Running a service when your ISP doesn't like you

We are running out of IPv4.

It is more common that ISPs won't give you a public IP nowadays. Especially if you live in an apartment, they only give you a DHCP port usually. This is a bit inconvenient (or annoying?) for people who want to run services, which requires a public IP.

Here's a common solution:

1. Rent a VPN on DigitalOcean ($10/mo)
2. Connect my desktop with my VPN
3. Run the server on my desktop
4. Forward traffic from my VPN to my desktop

The issue here is that OpenVPN will take over the entire interface (all the traffic will route through the tun0 device). My solution to that is jailing OpenVPN under a Linux network namespace.

Recently I modified other's script to run different kinds of VPNs under a LInux network namespace:

With the help of this script, I can just simply use iptables to perform a port forwarding on my VPS's public interface to the private IP of my desktop.

The overview of the design would be like this:

| VPS public IP (eth0 / x.x.x.x)
| openVPN (tun0 /
| ---------------------------------------------------------------
| sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 1234 -j DNAT --to
| sysctl net.ipv4.ip_forward=1
| my ISP ( under NAT ... orz)
| Desktop interface (enp0s0 / 192.168.a.b)
| Linux network NS (vpn0 /
| ip netns exec  ip route add default via dev vpn1
| iptables -t nat -A POSTROUTING -o en+ -s -j MASQUERADE
| sysctl net.ipv4.ip_forward=1
| Linux network NS (vpn1 /
| ---------------------------------------------------------------
| openVPN (tun0 /
| ---------------------------------------------------------------
| server running on port 1234
| ---------------------------------------------------------------

Sunday, January 15, 2017

upgrade to awesome 4.0

awesomeWM v4.0 release came out last Christmas, I spent some time to migrate my rc.lua to v4.0. Tones of features are changed.

Recommended readings:

v4.0 NEWS:
Porting tips From 3.5 to 4.0:

Here are the main changes to my personal rc.lua:

- awful.screen added new method to iterate all screens
  - only go over all screens once via the main awful.screen.connect_for_each_screen() loop
  - all for s = 1, screen.count() loops are removed

- hotkey popups : [Mod4+s] gives you the hotkeys help of awesome and vim
  - awful.key takes a table for the descriptions
  - some vim keys are predefined in awesome-git/lib/awful/hotkeys_popup/keys/vim.lua

- New wallpaper setup: the helper function set_wallpaper will check if beautiful.wallpaper is a function, if so, run it, otherwise, setup with gears.

-awful.util.pread is removed, use io.popen instead
  - Caution: this is not recommended in awesome, but I only use them for tooltips and timer updates.
  - there's a new awful.spawn.easy_async for other purposes, check awesome-git/lib/awful/spawn.lua as a reference.

- use declarative widget syntax for adding widgets to the wibox
  - imperative style is still supported

awful.widget.textclock is renamed to wibox.widget.textclock

- titlebars are enabled for all normal clients and dialogs by default in v4.0
  - I hide the titlebar by default but add [Mod4+Shift+t] to toggle it

mytextbox:set_text("foo") is changed to mytextbox.set_text = "foo"

- awful.util.spawn is renamed to awful.spawn

- use double quotes ("") for most of the strings

My rc.lua:

Monday, January 2, 2017

bye, 2016


去年是承先啟後,今年大概是大破大立吧。(錢都燒光了意味 #有錢就好了)

今年最大的一件事情應該就是搬家了,離開了住了 23 年的熱帶太平洋小島,到冰天雪地的 Westworld 來,剛換過來除了經濟狀況有點吃力之外,感謝同事朋友的幫忙,日子過的還算順利。

說到搬家,特別感謝 Ray 學長,半買半送了大量的二手傢俱給我,連車都一起讓我開走了 可惜買不起湖邊大別墅 ,下飛機第一個晚上直接洗洗睡才是 relocation 的王道阿。工作仍然一樣的繁忙,手上又是擺滿了具有挑戰性的高難度專案,客戶又是知名大公司,壓力實在不小,每天也都是兢兢業業的忙著。

年初等 Visa 的期間實在是煩悶到瘋掉, 2016 做的最瘋狂的事情大概是把身上存款燒光,半年內來了 10 趟日本旅遊成為東京窮遊代表。東京沖繩大阪東京沖繩東京北海道東京福岡東京。東京去了好多趟還是一樣每次都發現新東西,和新朋友吃飯,去了新的地方 (到底要花幾趟才能把關東地區玩透呢?) ,沖繩除了沙灘海洋之外,幾度拜訪友人才是旅行的目的。多數時間都是自己一個人來日本拜訪日本朋友,不過也帶了老媽和好友黑羊在日本轉轉。抱著人生只活了一趟 (YOLO) 的態度,買了樂桃會員,深夜工作心情煩悶就手滑張機票到東京吃吃,不知不覺日文會話能力也提高了不少。就是年少輕狂才能這樣,半年內毫無顧慮的燒了五十萬存款這麼瘋狂的事情,應該是腦子燒了才是。

總之趁著年輕時好好玩吧,目前的人生規劃,能這樣瘋狂放蕩的時間也就這樣一次 "年少你我也曾如此風光 往往在深夜中醒來笑到天亮"。


要說到今年特別的成就大概是只花兩個週末學會開車吧,然後第三週就考到駕照了 (18 天)。某個週五晚上同事 K 問我要不要學,就直接從深夜停車場學轉向,隔天下大雪,到另一個比較大的停車場開始學 溜冰 操控油門,第二個週末就直接開上路 (高速公路了)。回想起來週日晚上在兩個交流道之間來回個十幾趟簡直是神經病 XDDDDD

人在異鄉特別感到友誼的可貴,一個感恩節週末竟然接連兩個高中好友來訪,倒屣相迎是一定的。特別是因為到中國唸書而 人間蒸發 的學弟突然 show up 在這個鄉下小鎮,更讓本肥宅垂死病中驚坐起,笑問客從何處來。

2016 年蠻感謝黑羊的,常常被我無故拖出來喝咖啡吐苦水,不過可能接下來要看到大姊姊腰肉的機會應該不多惹,也喝不到大姊姊泡的咖啡和好吃的乳酪蛋糕惹 QAQ

感情的事情也不必多談,除了幾些小插曲之外,基本上沒啥進展,2017 年可能也沒什麼好說了 XD



Wednesday, October 19, 2016

New idea sharing site

Recently, my friend +Ronmi Ren and +Jeremy Chang created a new site for idea sharing (would be in Chinese mostly, sorry) and invited me as an article writer. I would be kind of starting sharing my thoughts from work experience articles there and keep more detailed technique stuff or personal thoughts in this blog.

This is one of my recent post there, feedbacks are welcome. :)


The name of this new site if from a inner joke on PTT, 公道價八萬一, we hope that every developer can make more money away from the current low paid level.

Saturday, January 2, 2016

bye, 2015


今年該怎麼說呢? 算是承先啟後的一年吧。

年初一月底順利的把交大的必修學分都解決,大四上大魔王的兩科數學:微分方程和機率,也都順利結束掉。修了日文三,武藤老師還是一樣的親切,雖然打分上變得嚴格一些,日文終於進步到基本的動詞型態變化,愈來愈能聽懂一些簡單的句型了,不再只是ですます的單純敘述。學期末系計中那邊的工作也正式的跟團長說要放掉了,一來想離開學校,二來也多留時間給自己唸書工作,再者想學的東西也差不多學完了,該是撤退的時候了。唯一比較殘念的大概是原本預計大四碰的 CCNA 隨著後來補學分忙沒什麼問之外就放棄了,有點可惜,不然有些管設備的經驗應該不錯。走了以後回頭看,現在像是靠當年我上課教出來的學弟們撐著,至於以後的事情,大概也跟我無關吧,都退休了。應該也不會再回去。

新學期的大四下,原本想要趁著大四上把學分修完後就離開鬼城,問了系上規定後確得知我沒有達成提早畢業條件(雖然學分已經滿了),而必須待在學校。對於我這種成績算中下的人來說,提早畢業的門檻為 80 分實在是有點嚴苛,最後終於找出了算是妥協的作法:申請最低修業。依據學校的規定,最低修業要九學分,也就是三門課。挖了許多行政規章後找到一條例外條款,特殊原因的修業申請可以把這個門檻降到一門課,和系辦小姐商討後,決定已要在公司實習為理由送出申請。那天學校行政效率意外的高,還記得是個禮拜五早上把申請單從系辦蓋章後給學務處,下午前往台北的客運上就接到了確認,於是就把手上原本選課系統點好的課程通通退掉了。

一開始是想要繼續修武藤老師的日文四,可惜加簽失敗,後來經過 joyqul 學妹的勸說下(?) 挑戰了上條老師的 "中級日文會話" ,這是個兩學期的課,我四上時沒有修過 "初級日文會話" 直接跳來修中級,壓力著實不小。所以大四下的修業就成了只有一門日文課,於是我把宿舍東西打包後退了宿回台中了,過著半年台中 - 新竹通勤的日子。

以修課之後來說,我蠻感謝上條老師這門會話課的充實訓練,尤其是逆質問(ぎゃくしつもん)的部份,和人對話除了問問題之外也要有能力反問和追問下去,這是非常重要的能力。印象最深刻的大概是老師整學期不停的推坑同學參加各式各樣的比賽,而我這組是被推去動畫配音比賽 Orz. 找了四個學妹和一位電機系的同學組隊參加,本來還想找另一個電機所的學姊的,結果比賽僅限定大學部,殘念。從百度上找到的謎之字幕,拿給老師修正後,用好久沒用的 Aegisub 在從 torrent network 找的影片檔(感謝 YYDM)加上上了字幕,練習用的版本就誕生了。除了每週去上課外,又額外約了好幾個晚上練習,整整兩個多月間爆出了一筆交通費囧,而且莫名我被老師指派為組長 QQ. 老師對發音非常要求,也聽的很仔細,學妹日文底子比較弱就被老師糾正了好幾次,壓力似乎很大。比賽當天倒是開心的一日遊,捷運上合流後一路到了會場,無聊的開幕式跟學妹在打電動www 我們是下午第一場選手,不過也就呼壟過去了。倒是其他組的選手都很厲害,除了日文明顯比我們好很多之外,音效和道具的準備也十分充足。冠軍的台大毋庸置疑,他們的螢火蟲之墓關西腔講的十分道地,非常厲害!上條老師的會話課讓我的日文能力有幾乎說是爆炸性的進步,短文寫作、聽力也進步許多,其中還感謝幾位 CTF 認識的日本推友幫忙。

其他日子就在家工作,繼續寫公司的案子,領著棉薄的打工仔薪水。空閒時間就看看動畫看看書。雖然只有一個日文課但其實我覺得份量不算少,除了練習日文之外準備比賽也花了很多時間。 在家工作這段時間,念了些英文日文,雖然數量少,也看了一些書,唸書速度還是很慢QQ 六月初和學長決定暑假再去美國一趟後,跟老師還有系上橋了一下時間,在六月二十二把畢業證書搞到手了。還是先拿到畢業證書才去考日文期末口試的,雖然大爆炸了XD

六月算是正式職業生涯的開始,終於順利的從 intern 轉成正職。雖然薪水也沒多少但足以支付我生活開銷又能存點小錢了,雖然狂電交大大學部畢業起薪水準來說是足夠了,科科,剩下的等 H1B 拿到後再繼續努力吧。有了正職薪水,經濟上也變得寬裕許多,大學時一個月一萬現在回頭看實在是太慘了,終於不用再憋手憋腳的讓現金流卡來卡去,能夠有更充裕的預算和規劃,想花錢就有足夠的能力,當然主力目標仍是放在存錢上。

拿到畢業證書兩天後,再度飛到 Champaign ,開始了度假般的日子 (?) ,完成了一些工作,這次待比較長的時間,加上 Sriram 有車,幾個週末去了一些地方玩耍,特別是去 Springfield 看 Lincoln ,美國的博物館我真的非常喜歡,有機會的話該帶些人去拜訪看看。工作的事情第一階段算是順利,除此之外,也和宋大師討論了許多未來的方向,並且跟胡老師算是初步談好了接下來 H1B 和綠卡的事情。比較突破自我的事情大概是再獨立完成了 project ,交付了大 project 的完整 milestone ,從念 paper ,讀 code 改 code 到 benchmark 和繳交報告基本上是我自己獨立完成,當然中間借助了宋大師和 Yun-wei 的一些想法。後期在幫忙實作一些 C++17 runtime ,雖然上個月被發現寫出來的東西有些問題後來請 Yun-wei 幫忙修掉了,不過整體的開發過程也算是非常開心,感謝 Jack 給的協助。在 Champaign 期間另一個自我突破大概是在胡老師的研究團隊上給了一場安全的 talk ,花了一個小時讓這些對網路安全幾乎沒有概念的人理解到常見的知識點和攻擊方向。老師和 Tom 似乎非常肯定我的表現,以一個剛畢業的大學生來說,當然 Tom 後來有說如果我是個 master 或 PhD 當然還差的遠,哈哈。外語能力的部份,反而帶過去的英文書沒怎麼念,搭車到公司途中倒是利用時間念些日文,單字量提高了不少。

回到台灣後的生活大概跟上半年差不多,差別只在不用上學。跟著 Kevin 和 Jack 弄些 project ,然後現在手頭在接手很可怕的怪物一直到現在 Orz.

十月底跟 Jason 到東京小旅行一趟,行程規劃非常簡單。不過這卻是我這輩子活到現在第一次出國玩,雖然只是個週末小旅行,但確真的來到了日本,身邊充斥著滿滿的日文。大二到現在上過的日文課終於派上用場,看過的動畫裏面台詞也不停的在耳朵邊出現,尤其是大四下經過了上調老師的訓練,我竟然能幾乎毫無障礙的和日本人用流利的簡單日文溝通,我自己也覺得非常意外。

今年一樣是過著沒什麼感情的一年,身邊也沒有出現想要追求的對象。日常大概就是用工作和唸書填滿自己吧。某方面來說算是一種麻痹,交友圈又這麼小,遇到適合的人的機會可能也很小吧。平常就感謝琬庭、秀、黑羊、NaNi、硬龍、君君讓我打發時間的閒聊吧,有些人能讓我這樣肆無忌憚的吐吐苦水也還是不錯的。一個人再怎麼努力的逃避也還是會有空虛寂寞的時候吧,總覺得熟悉的朋友比虛無飄渺的目標好多了。隨著工作的時間愈長,對許多人生價值觀的看法也開始不一樣,算是社畜化? 我也不知道。有了點小錢後會覺得沒有經濟能力的人實在無法相處下去,常常在推特上講 #有錢就好了 某方面來說是對於社經地位的一種期待吧。可惜我現在認識的人多數還只是學生或是在社會上打滾不久的,這方面也就沒有目標了。沒有那種想要用心追求的對象,不想跟太過幼稚、心靈層次不同的人相處,換個角度來看,大學的我真是太幼稚了,太容易把自己的感情投入到不踏實的幻想吧... 唉。

今年因為搬回家裡,倒是一天到晚和家人吵架,年中時鬧得最不開心,甚至是一度想要搬離家裡,連房子都找好了。後來從美國回台灣後,想了想先待到年底試試看,就直接撐到現在了... 也不知道以後該怎樣呢...

舊朋友新朋友有撐下去的算是運氣好吧,回台中住後明顯感到生活圈變小很多,雖然原本在新竹也沒多大。除了購物和打 ingress 之外不太出門。假日也比較傾向待在家裡休息、唸書。年底因為參加在板橋的官方活動,又重拾回 ingress 了,就當作自己運動的藉口,不知道能撐下多久呢。每天的日常還是一樣在 SNS 上面,twitter ,plurk 浪費時間,上 IRC 的時間少一些,可能跟主題漸漸不有趣也有關,WTF 頻道大家也沒什麼水量。HITCON 年底決賽的時候邀了 binja 和 TokyoWesterns 的朋友和 DSNS lab 一同吃了頓晚餐,原本只是要我自己跟他們吃個麵而已,後來因為去看了金盾決賽就合流成好大一團了。日文會話練到的日文再次起了作用,雖然還是會發生那種剛講完就馬上意識到自己文法、用字是錯誤的,再多練習吧。