Wednesday, April 8, 2015

More notes for shadowsocks

I wrote a note for the usage of shadowsocks few days ago:

I spent some time to dig into the source code of the project and came up with this note, just a note for what I've found. :D


shadowsocks is under Apache 2.0.


There's a script called under shadowsocks/utils .

According to the official wiki, that is used for banning brute force crackers.

Actually, is implemented by iptables , this script looks at the log and find something like this and grab the remote remote IP out.

'2015-04-07 16:42:26 ERROR    can not parse header when handling connection from'

if 'can not parse header when' in line:
    ip = line.split()[-1].split(':')[0
    cmd = 'iptables -A INPUT -s %s -j DROP' % ip
    print(cmd, file=sys.stderr)


shadowsocks uses a very strange way to determine it's running under python2 or python3.

if bytes == str

This is True in python2 but False in python3.

Also, here's a strange logic to check the version.

I will write if info[0] == 2 and info[1] < 6 rather than the author does.

def check_python():
    info = sys.version_info
    if info[0] == 2 and not info[1] >= 6:
        print('Python 2.6+ required')
    elif info[0] == 3 and not info[1] >= 3:
        print('Python 3.3+ required')
    elif info[0] not in [2, 3]:
        print('Python version not supported')
Argument parsing

The entry points of sslocal and ssserver commands are the main functions in and, respectively. checks command line arguments and checks the configuration files, it's using getopt, I think that should should be rewrite with argparse .

Event loop and Relay:

Basically shadowsocks abstracts three kinds of polling system: epool, kqueue and system select, it will use them in order if available.

class EventLoop(object):
    def __init__(self):
        self._iterating = False
        if hasattr(select, 'epoll'):
            self._impl = EpollLoop()
            model = 'epoll'
        elif hasattr(select, 'kqueue'):
            self._impl = KqueueLoop()
            model = 'kqueue'
        elif hasattr(select, 'select'):
            self._impl = SelectLoop()
            model = 'select'
            raise Exception('can not find any available functions in select '

So basically shadowsocks has a local server connected to the SOCK5 proxy and send data to remote via TCP/UDP relays.

Both of TCP/UDP relays will encrypt the payload with specified algorithms.

Here's the diagram of the idea of shadowsocks.

browser <== SOCKS proxy ==> local <== TCP/UDP relays ==> remote => free world

browser <== plain text ==> local <= encrypted data => GFW <= encrypted data => remote => free world

Pretty similar to SSH tunnel, right?

browser <= socks proxy => ssh client <= tunnel => ssh server => free world

The feathers of SSH handshaking traffic is easily blocked by GFW, shadowsocks are just simple standard TCP/UDP traffic with unknown/encrypted payloads.

The following is from the comments of and .

TCP Relay

# for each opening port, we have a TCP Relay
# for each connection, we have a TCP Relay Handler to handle the connection
# for each handler, we have 2 sockets:
#    local:   connected to the client
#    remote:  connected to remote server

# as sslocal:
# stage 0 SOCKS hello received from local, send hello to local
# stage 1 addr received from local, query DNS for remote
# stage 2 UDP assoc
# stage 3 DNS resolved, connect to remote
# stage 4 still connecting, more data from local received
# stage 5 remote connected, piping local and remote

# as ssserver:
# stage 0 just jump to stage 1
# stage 1 addr received from local, query DNS for remote
# stage 3 DNS resolved, connect to remote
# stage 4 still connecting, more data from local received
# stage 5 remote connected, piping local and remote

UDP Relay

# ------------------
# `dest`    means destination server, which is from DST fields in the SOCKS5
#           request
# `local`   means local server of shadowsocks
# `remote`  means remote server of shadowsocks
# `client`  means UDP clients that connects to other servers
# `server`  means the UDP server that handles user requests


shadowsocks implements its own DNS query ( and LRU caching  ( system.


steal LINE stickers to telegram

1. pull all LINE stickers from you phone

adb pull /storage/sdcard0/Android/data/ .

2. find your purchased sticker pack (you can look at the "preview" file)

for example, the sticker pack of Puella Magi Madoka Magica is # 1101

Note, you also can find stickers sent by your friend.

3. convert your the files (from PNG) to the WebP format

I'm too lazy so I wrote a script to do the following stuffs:

mkdir madoka
cp ../stickers/1101/* madoka/
cd chocola/
rm *_key preview thumbnail *.tmp
for i in *; do cwebp $i -o $i.webp ; done
for i in `ls | grep -v webp`; do mv $i $i.png; done
cd ..

4. put them in your phone, done!


The first one is the file in PNG, the second one is in WebP.

Automatic script:


Saturday, April 4, 2015

netowrk speed test between two linux boxes

This trick with nc and dd can be used as speed testing between two linux boxes.

nc -vvlnp 12345 > /dev/null

dd if=/dev/zero bs=1M count=1k | nc -vvn <server IP> 12345

Wednesday, April 1, 2015

ramdisk for chromium

I just bought two new ram for my desktop.
HyperX FURY black DDR3-1866 8GB(8GBx2) (HX318C10FBK2/16)

tmpfs     /tmp/cache    tmpfs     nodev,nosuid,noatime,size=2G      0 0

$ chromium --disk-cache-dir=/tmp/cache

$ mount | grep cache
tmpfs on /tmp/cache type tmpfs (rw,nosuid,nodev,noatime,size=2097152k

manually create a ramdisk
$ mkdir -p /tmp/ram
$ sudo mount -t tmpfs -o size=1024M tmpfs /tmp/ram/

old post: