Compiler engineer skill tree

My new article on 81k.today: Compiler engineer skill tree , enjoy!

Notes for iproute2, network namespace, and iptables NAT

Some random notes for stuff. Have fun!

# the ip command line tool (iproute2)

# setup an $INTERFACE and bind to $IP/$MASK
# ip link set $INTERFACE up
# ip addr add  $IP/$MASK dev $INTERFACE
# ip route add $IP/$MASK dev $INTERFACE

# delete the above settings
# ip route del $IP/$MASK dev $INTERFACE
# ip addr del $IP/$MASK dev $INTERFACE
# ip link set $INTERFACE down


# static routing
# ip route add default via $GATEWAY dev $INTERFACE
# ip route add $IP dev $INTERFACE


# search devices name
$ ls /sys/class/net/
$ ip link


# flush everything!
# ip addr flush dev $INTERFACE
# ip route flush dev $INTERFACE
# ip link set $INTERFACE down


# evil stuff

# promiscuous mode
# ip link set dev $INTERFACE promisc on

# MAC address spoofing
# ip link set dev $INTERFACE down
# ip link set dev $INTERFACE address XX:XX:XX:XX:XX:XX
# ip link set dev $INTERFACE up




# network namespace (need root permission)

# add and delete the network namespace
# ip netns add $NS_NAME
# ip netns del $NS_NAME

# ip -n shortcut
# ip -n $NS_NAME [ OPTIONS ] OBJECT { COMMAND | help }
# ip netns exec $NS_NAME ip [ OPTIONS ] OBJECT { COMMAND | help }

# run an app under $NS_NAME (remember to downgrade permission)
# ip netns exec $NS_NAME sudo -u $REGULAR_USER $application

# in scripts ...
NS_EXEC="ip netns exec $NS_NAME"


# virtual ethernet interface (create vpn0 outside and peer vpn1 under $NS_NAME)
# ip link add vpn0 type veth peer name vpn1
# ip link set vpn0 up
# ip link set vpn1 netns $NS_NAME up


# finding apps under $NS_NAME, and kill them all!
# ip netns pids $NS_NAME
# ip netns pids $NS_NAME | xargs -n1 -I{} ps -q {} -o pid=,comm=
# ip netns pids $NS_NAME | xargs kill



# sysctl settings to enable forwarding
#   save to /etc/sysctl.conf or /etc/sysctl.d/30-ipforward.conf
# sysctl net.ipv4.ip_forward=1
# sysctl net.ipv6.conf.default.forwarding=1
# sysctl net.ipv6.conf.all.forwarding=1



# iptables stuff

# allow a $PORT from $IP/$MASK for tcp/udp
#   drop: -j DROP
# iptables -A INPUT -p tcp --dport $PORT -s $IP/$MASK -j ACCEPT
# iptables -A INPUT -p udp --dport $PORT -s $IP/$MASK -j ACCEPT


# IP masquerade for NAT
#   -o output interface
#   -s source $IP/$MASK
# iptables -t nat -A POSTROUTING -o $INTERFACE -j MASQUERADE
# iptables -t nat -A POSTROUTING -s $IP/$MASK -o $INTERFACE -j MASQUERADE


# DNAT on all protocols from $IP1 to $IP2
#  -d (original) destination IP
#  --to-destination destination IP behind NAT
# iptables -t nat -A PREROUTING -d $IP1 -j DNAT --to-destination $IP2

# specify only tcp/$PORT
#   -p protocol, usually tcp, udp, icmp, icmpv6, or all
#   --to-destination could be a range [ipaddr[-ipaddr]][:port[-port]], remain the same port if not specified
# iptables -t nat -A PREROUTING -p tcp -d $IP1 --dport $PORT -j DNAT --to-destination $IP2

# use MATCH extension, see iptables-extensions(8)
#   Example: https://serverfault.com/questions/381700/iptables-when-to-use-m-in-a-rule
# iptables -t nat -A PREROUTING -i $INTERFACE -p tcp -m tcp --dport $PORT1 -j DNAT --to-destination $IP:$PORT2


# save and restore
# iptables-save
# iptables-restore


# dump current rules
#   -n no DNS lookup
#   -v verbose
#   -L list
#   -t table (default: filter)
#   --line-numbers might be helpful

# iptables -nvL
# iptables -nvL -t nat


# Reference:

ip(8), ip-address(8), ip-link(8), ip-netns(8), ip-route(8)
https://wiki.archlinux.org/index.php/Network_configuration
https://wiki.archlinux.org/index.php/MAC_address_spoofing
https://wiki.archlinux.org/index.php/iptables
https://xatierlike.blogspot.com/2013/04/ip-tools-note.html

Running a service when your ISP doesn't like you

We are running out of IPv4.

It is more common that ISPs won't give you a public IP nowadays. Especially if you live in an apartment, they only give you a DHCP port usually. This is a bit inconvenient (or annoying?) for people who want to run services, which requires a public IP.

Here's a common solution:

1. Rent a VPN on DigitalOcean ($10/mo)
2. Connect my desktop with my VPN
3. Run the server on my desktop
4. Forward traffic from my VPN to my desktop

The issue here is that OpenVPN will take over the entire interface (all the traffic will route through the tun0 device). My solution to that is jailing OpenVPN under a Linux network namespace.

Recently I modified other's script to run different kinds of VPNs under a LInux network namespace: https://github.com/xatier/rc-files/blob/master/bin/v.sh

With the help of this script, I can just simply use iptables to perform a port forwarding on my VPS's public interface to the private IP of my desktop.

The overview of the design would be like this:

---------------------------------------------------------------
| VPS public IP (eth0 / x.x.x.x)
---------------------------------------------------------------
| openVPN (tun0 / 10.8.0.1)
| ---------------------------------------------------------------
| sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 1234 -j DNAT --to 10.8.0.2:1234
| sysctl net.ipv4.ip_forward=1
===============================================================
|
| my ISP ( under NAT ... orz)
|
===============================================================
| Desktop interface (enp0s0 / 192.168.a.b)
---------------------------------------------------------------
| Linux network NS (vpn0 / 10.200.200.1)
| 
| ip netns exec  ip route add default via 10.200.200.1 dev vpn1
| iptables -t nat -A POSTROUTING -o en+ -s 10.200.200.0/24 -j MASQUERADE
| sysctl net.ipv4.ip_forward=1
| 
| Linux network NS (vpn1 / 10.200.200.2)
| ---------------------------------------------------------------
| openVPN (tun0 / 10.8.0.2)
| ---------------------------------------------------------------
| server running on port 1234
| ---------------------------------------------------------------

upgrade to awesome 4.0

awesomeWM v4.0 release came out last Christmas, I spent some time to migrate my rc.lua to v4.0. Tones of features are changed.

Recommended readings:

v4.0 NEWS: https://awesomewm.org/apidoc/documentation/89-NEWS.md.html
Porting tips From 3.5 to 4.0: https://awesomewm.org/apidoc/documentation/17-porting-tips.md.html


Here are the main changes to my personal rc.lua:

- awful.screen added new method to iterate all screens
  - only go over all screens once via the main awful.screen.connect_for_each_screen() loop
  - all for s = 1, screen.count() loops are removed

- hotkey popups : [Mod4+s] gives you the hotkeys help of awesome and vim
  - awful.key takes a table for the descriptions
  - some vim keys are predefined in awesome-git/lib/awful/hotkeys_popup/keys/vim.lua

- New wallpaper setup: the helper function set_wallpaper will check if beautiful.wallpaper is a function, if so, run it, otherwise, setup with gears.

-awful.util.pread is removed, use io.popen instead
  - Caution: this is not recommended in awesome, but I only use them for tooltips and timer updates.
  - there's a new awful.spawn.easy_async for other purposes, check awesome-git/lib/awful/spawn.lua as a reference.

- use declarative widget syntax for adding widgets to the wibox
  - imperative style is still supported

- awful.widget.textclock is renamed to wibox.widget.textclock

- titlebars are enabled for all normal clients and dialogs by default in v4.0
  - I hide the titlebar by default but add [Mod4+Shift+t] to toggle it

- mytextbox:set_text("foo") is changed to mytextbox.set_text = "foo"

- awful.util.spawn is renamed to awful.spawn

- use double quotes ("") for most of the strings

My rc.lua: https://github.com/xatier/rc-files/blob/master/rc.lua

bye, 2016

年底反省文

去年是承先啟後，今年大概是大破大立吧。(錢都燒光了意味 #有錢就好了)

今年最大的一件事情應該就是搬家了，離開了住了 23 年的熱帶太平洋小島，到冰天雪地的 Westworld 來，剛換過來除了經濟狀況有點吃力之外，感謝同事朋友的幫忙，日子過的還算順利。

說到搬家，特別感謝 Ray 學長，半買半送了大量的二手傢俱給我，連車都一起讓我開走了 可惜買不起湖邊大別墅 ，下飛機第一個晚上直接洗洗睡才是 relocation 的王道阿。工作仍然一樣的繁忙，手上又是擺滿了具有挑戰性的高難度專案，客戶又是知名大公司，壓力實在不小，每天也都是兢兢業業的忙著。

年初等 Visa 的期間實在是煩悶到瘋掉， 2016 做的最瘋狂的事情大概是把身上存款燒光，半年內來了 10 趟日本旅遊成為東京窮遊代表。東京沖繩大阪東京沖繩東京北海道東京福岡東京。東京去了好多趟還是一樣每次都發現新東西，和新朋友吃飯，去了新的地方 (到底要花幾趟才能把關東地區玩透呢?) ，沖繩除了沙灘海洋之外，幾度拜訪友人才是旅行的目的。多數時間都是自己一個人來日本拜訪日本朋友，不過也帶了老媽和好友黑羊在日本轉轉。抱著人生只活了一趟 (YOLO) 的態度，買了樂桃會員，深夜工作心情煩悶就手滑張機票到東京吃吃，不知不覺日文會話能力也提高了不少。就是年少輕狂才能這樣，半年內毫無顧慮的燒了五十萬存款這麼瘋狂的事情，應該是腦子燒了才是。

總之趁著年輕時好好玩吧，目前的人生規劃，能這樣瘋狂放蕩的時間也就這樣一次 "年少你我也曾如此風光　往往在深夜中醒來笑到天亮"。

搬家後日子其實過得平淡無奇，也就是普通鄉下肥宅工程師的生活。生活圈變得比在台灣當尼特的日子還小更多，總覺得和人有種距離感。除了公司和購物之外也不會去其他社交場合。在鄉下韜光養晦田野間隱居個幾年，剩下的事情等之後再規劃吧。

要說到今年特別的成就大概是只花兩個週末學會開車吧，然後第三週就考到駕照了 (18 天)。某個週五晚上同事 K 問我要不要學，就直接從深夜停車場學轉向，隔天下大雪，到另一個比較大的停車場開始學 溜冰 操控油門，第二個週末就直接開上路 (高速公路了)。回想起來週日晚上在兩個交流道之間來回個十幾趟簡直是神經病 XDDDDD

人在異鄉特別感到友誼的可貴，一個感恩節週末竟然接連兩個高中好友來訪，倒屣相迎是一定的。特別是因為到中國唸書而 人間蒸發 的學弟突然 show up 在這個鄉下小鎮，更讓本肥宅垂死病中驚坐起，笑問客從何處來。

2016 年蠻感謝黑羊的，常常被我無故拖出來喝咖啡吐苦水，不過可能接下來要看到大姊姊腰肉的機會應該不多惹，也喝不到大姊姊泡的咖啡和好吃的乳酪蛋糕惹 QAQ

感情的事情也不必多談，除了幾些小插曲之外，基本上沒啥進展，2017 年可能也沒什麼好說了 XD

大破大立的一年，新的故事，舊的回憶，在鄉下隱居沉潛個幾年再來大鳴大放吧。

大家新年快樂。