Tuesday, May 5, 2015

STL pretty print support in gdb

I program in C/C++ for a long time, it's really hateful to debug C++ programs with STL containers in gdb, but yeah, who doesn't use STL?

gdb always prints lots of useless stuffs from a container.

$ g++ -std=c++11 foo.cc -g
$ gdb -q ./a.out
Reading symbols from ./a.out...done.

(^q^) l
1 #include <vector>
2
3 int main (void) {
4    std::vector<int> v = {1, 2, 3, 4, 5};
5    return 0;
6 }

(^q^) b 5
Breakpoint 1 at 0x40082c: file foo.cc, line 5.

(^q^) r
Starting program: /tmp/a.out 

Breakpoint 1, main () at foo.cc:5
5    return 0;

(^q^) p
$2 = {
  <std::_Vector_base<int, std::allocator<int> >> = {
    _M_impl = {
      <std::allocator<int>> = {
        <__gnu_cxx::new_allocator<int>> = {<No data fields>}, <No data fields>}, 
      members of std::_Vector_base<int, std::allocator<int> >::_Vector_impl: 
      _M_start = 0x602010, 
      _M_finish = 0x602024, 
      _M_end_of_storage = 0x602024
    }
  }, <No data fields>}

(^q^) 



And we're able to use a very tricky way to print that out.


(^q^) p *(v._M_impl._M_start)@(v._M_impl._M_finish - v._M_impl._M_start)
$22 =   {[0] = 1,
  [1] = 2,
  [2] = 3,
  [3] = 4,
  [4] = 5}

(^q^) 

A container is a "container", we don't care the implementation details, what we care about is the data.

With the pretty printer, we're able to print a container out like this:

(^q^) p v
$1 = std::vector of length 5, capacity 5 = {
  [0] = 1,
  [1] = 2,
  [2] = 3,
  [3] = 4,
  [4] = 5
}


According to the gdb wiki, the gdb python pretty printer is supported since 7.0. https://sourceware.org/gdb/wiki/STLSupport

Download the latest python script from gcc.gnu.org
---
mkdir ~/.gdb
cd ~/.gdb
svn co svn://gcc.gnu.org/svn/gcc/trunk/libstdc++-v3/python
---

put the following stuffs appended to your ~/.gdbinit
---
python
import sys
sys.path.insert(0, '/home/xatier/.gdb/python')
import libstdcxx.v6
end
---

Note, we only need to import the lib, let __init__.py do the magic ;-)

Done!


If you really want to dig up the implementation details, use " p /r "


(^q^) p /r v
$1 = {
  <std::_Vector_base<int, std::allocator<int> >> = {
    _M_impl = {
      <std::allocator<int>> = {
        <__gnu_cxx::new_allocator<int>> = {<No data fields>}, <No data fields>}, 
      members of std::_Vector_base<int, std::allocator<int> >::_Vector_impl: 
      _M_start = 0x602010, 
      _M_finish = 0x602024, 
      _M_end_of_storage = 0x602024
    }
  }, <No data fields>}

(^q^)


Actually this python script will be installed and automatically loaded in Archlinux once you install gcc/gcc-multilib.


$ pacman -Ql gcc-multilib | grep libstd
gcc-multilib /usr/lib/libstdc++.a
gcc-multilib /usr/lib32/libstdc++.a
gcc-multilib /usr/share/gcc-4.9.2/python/libstdcxx/
gcc-multilib /usr/share/gcc-4.9.2/python/libstdcxx/__init__.py
gcc-multilib /usr/share/gcc-4.9.2/python/libstdcxx/v6/
gcc-multilib /usr/share/gcc-4.9.2/python/libstdcxx/v6/__init__.py
gcc-multilib /usr/share/gcc-4.9.2/python/libstdcxx/v6/printers.py
gcc-multilib /usr/share/gdb/auto-load/usr/lib/libstdc++.so.6.0.20-gdb.py


If you're using other distros like Debian, that should be here, but not auto-loaded by default :(

$ apt-file list gcc | grep libstd
gcc-snapshot: /usr/lib/gcc-snapshot/share/gcc-4.9.0/python/libstdcxx/__init__.py
gcc-snapshot: /usr/lib/gcc-snapshot/share/gcc-4.9.0/python/libstdcxx/v6/__init__.py
gcc-snapshot: /usr/lib/gcc-snapshot/share/gcc-4.9.0/python/libstdcxx/v6/printers.py


Happy debugging C++! (^q^)

Monday, May 4, 2015

Reverseing reverse engineered tools: reverse on the pyc of Easy-Card tool

A hacker called  Zhi-Wei Cai did some reverse engineering on Taipei city passport Easy Card credit querying system.


https://github.com/x43x61x69/Easy-Card


But he only released the .pyc file.

I tool a glance on the binary file and smiled, he didn't apply any code obfuscation on that, that is, it would be pretty easy for those who want to read the source code.

With the help of this tool: https://github.com/wibiti/uncompyle2

uncompyle2 easycard.pyc > easycard.py

You can decompile that yourself.




Basically the API takes 4 parameters: verify, cardID, begin, end.


verify = md5((seed * const) + salt)
where
    seed = date.month + date.day + date.hour
    salt = 'L0CalKing'
    const = 8544

cardID = base64( des3( data, key, iv, mode=DEC3.MODE_CBC) )
where
    data = 'your card ID', like '1234567889'
    key = 'EasyCardToKingay23456789
    iv = '01234567'
   
begin / end = time period


I'm really curious about how did he get these constants , but I don't want to dig up the original app. :)


Lesson: don't release .pyc file without code obfuscation if you really don't want any people try to dig up your code.


Code listing (partially omitted):

#!/usr/bin/env python2
# -*- encoding: utf8 -*-

# 2015.05.04 17:41:47 CST
import sys
import datetime
import hashlib
import urllib
import urllib2
import json
from Crypto.Cipher import DES3
import pytz
version = '0.3'
copyright = 'Copyright (C) 2015 Zhi-Wei Cai.'
key = 'EasyCardToKingay23456789'
iv = '01234567'
salt = 'L0CalKing'
const = 8544

def getID(data, isEncrypt, key, iv, encode):
    size = len(data)
    # '\x06' is the padding of DES3
    if size % 16 != 0:
        data += '\x06' * (16 - size % 16)
    des3 = DES3.new(key, DES3.MODE_CBC, iv)
    if isEncrypt:
        result = des3.encrypt(data).encode(encode).rstrip()
    else:
        result = des3.decrypt(data.decode(encode))
    return result



def getVerify(const, seed, salt):
    hash = hashlib.md5()
    hash.update(str(seed * const) + salt)
    return hash.hexdigest().upper()



def proc(data):
    e = getID(data, 1, key, iv, 'base64')
    cardID = urllib.quote_plus(e)
    date = datetime.datetime.now(pytz.timezone('Asia/Taipei'))
    seed = date.month + date.day + date.hour
    begin = '{:%Y-%m-%d}'.format(date - datetime.timedelta(days=30))
    end = '{:%Y-%m-%d}'.format(date)
    verify = getVerify(const, seed, salt)
    url = '<Easy Card API URL>'.format(verify, cardID, begin, end)
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    content = response.read()
    dict = json.loads(content)

   # the rest part of the code is omitted


if __name__ == '__main__':
    print '\n悠遊卡餘額明細查詢 v{}'.format(version)
    print '{}\n'.format(copyright)
    if len(sys.argv) > 1:
        try:
            print '\n{:=^90}\n'.format('[ 查詢開始 ]')
            proc(str(sys.argv[1]))
        except ValueError as err:
            pass
    else:
        while 1:
            try:
                data = raw_input('請輸入卡片號碼:').replace(' ', '')
                if len(data):
                    print '\n{:=^90}\n'.format('[ 查詢開始 ]')
                    proc(data)
                else:
                    break
            except ValueError as err:
                pass


#+++ okay decompyling easycard.pyc
# decompiled 1 files: 1 okay, 0 failed, 0 verify failed
# 2015.05.04 17:41:47 CST

Wednesday, April 8, 2015

More notes for shadowsocks

I wrote a note for the usage of shadowsocks few days ago: http://xatierlike.blogspot.tw/2015/03/note-for-shadowsocks.html

I spent some time to dig into the source code of the project and came up with this note, just a note for what I've found. :D


License

shadowsocks is under Apache 2.0.

autoban

There's a script called autoban.py under shadowsocks/utils .

According to the official wiki, that is used for banning brute force crackers.
https://github.com/shadowsocks/shadowsocks/wiki/Ban-Brute-Force-Crackers

Actually, autoban.py is implemented by iptables , this script looks at the log and find something like this and grab the remote remote IP out.

'2015-04-07 16:42:26 ERROR    can not parse header when handling connection from 61.157.96.193:27242'

if 'can not parse header when' in line:
    ip = line.split()[-1].split(':')[0
    ...
    cmd = 'iptables -A INPUT -s %s -j DROP' % ip
    print(cmd, file=sys.stderr)
    os.system(cmd)


Versions

shadowsocks uses a very strange way to determine it's running under python2 or python3.

if bytes == str

This is True in python2 but False in python3.

Also, here's a strange logic to check the version.

I will write if info[0] == 2 and info[1] < 6 rather than the author does.

def check_python():
    info = sys.version_info
    if info[0] == 2 and not info[1] >= 6:
        print('Python 2.6+ required')
        sys.exit(1)
    elif info[0] == 3 and not info[1] >= 3:
        print('Python 3.3+ required')
        sys.exit(1)
    elif info[0] not in [2, 3]:
        print('Python version not supported')
        sys.exit(1)
     
     
Argument parsing

The entry points of sslocal and ssserver commands are the main functions in local.py and server.py, respectively.

shell.py checks command line arguments and checks the configuration files, it's using getopt, I think that should should be rewrite with argparse .


Event loop and Relay:

Basically shadowsocks abstracts three kinds of polling system: epool, kqueue and system select, it will use them in order if available.

class EventLoop(object):
    def __init__(self):
        self._iterating = False
        if hasattr(select, 'epoll'):
            self._impl = EpollLoop()
            model = 'epoll'
        elif hasattr(select, 'kqueue'):
            self._impl = KqueueLoop()
            model = 'kqueue'
        elif hasattr(select, 'select'):
            self._impl = SelectLoop()
            model = 'select'
        else:
            raise Exception('can not find any available functions in select '
                            'package')


So basically shadowsocks has a local server connected to the SOCK5 proxy and send data to remote via TCP/UDP relays.

Both of TCP/UDP relays will encrypt the payload with specified algorithms.

Here's the diagram of the idea of shadowsocks.

browser <== SOCKS proxy ==> local <== TCP/UDP relays ==> remote => free world

browser <== plain text ==> local <= encrypted data => GFW <= encrypted data => remote => free world


Pretty similar to SSH tunnel, right?

browser <= socks proxy => ssh client <= tunnel => ssh server => free world

The feathers of SSH handshaking traffic is easily blocked by GFW, shadowsocks are just simple standard TCP/UDP traffic with unknown/encrypted payloads.


The following is from the comments of tcprelay.py and rdprelay.py .

TCP Relay

# for each opening port, we have a TCP Relay
# for each connection, we have a TCP Relay Handler to handle the connection
# for each handler, we have 2 sockets:
#    local:   connected to the client
#    remote:  connected to remote server

# as sslocal:
# stage 0 SOCKS hello received from local, send hello to local
# stage 1 addr received from local, query DNS for remote
# stage 2 UDP assoc
# stage 3 DNS resolved, connect to remote
# stage 4 still connecting, more data from local received
# stage 5 remote connected, piping local and remote

# as ssserver:
# stage 0 just jump to stage 1
# stage 1 addr received from local, query DNS for remote
# stage 3 DNS resolved, connect to remote
# stage 4 still connecting, more data from local received
# stage 5 remote connected, piping local and remote

UDP Relay

# HOW TO NAME THINGS
# ------------------
# `dest`    means destination server, which is from DST fields in the SOCKS5
#           request
# `local`   means local server of shadowsocks
# `remote`  means remote server of shadowsocks
# `client`  means UDP clients that connects to other servers
# `server`  means the UDP server that handles user requests


Misc

shadowsocks implements its own DNS query (asyncdns.py) and LRU caching  (lru_cache.py) system.


Reference:
http://vc2tea.com/whats-shadowsocks/
http://gpio.me/readcode-ShadowSocks.html

steal LINE stickers to telegram

1. pull all LINE stickers from you phone

adb pull /storage/sdcard0/Android/data/jp.naver.line.android/stickers/ .

2. find your purchased sticker pack (you can look at the "preview" file)

for example, the sticker pack of Puella Magi Madoka Magica is # 1101

Note, you also can find stickers sent by your friend.

3. convert your the files (from PNG) to the WebP format

I'm too lazy so I wrote a script to do the following stuffs:

mkdir madoka
cp ../stickers/1101/* madoka/
cd chocola/
rm *_key preview thumbnail *.tmp
for i in *; do cwebp $i -o $i.webp ; done
for i in `ls | grep -v webp`; do mv $i $i.png; done
cd ..

4. put them in your phone, done!



Test:

The first one is the file in PNG, the second one is in WebP.



Automatic script: https://gist.github.com/xatier/971e1abe16f3bcbc51d9

Reference
https://telegram.org/blog/stickers
https://developers.google.com/speed/webp/docs/using

Saturday, April 4, 2015

netowrk speed test between two linux boxes


This trick with nc and dd can be used as speed testing between two linux boxes.

Server:
nc -vvlnp 12345 > /dev/null

Client:
dd if=/dev/zero bs=1M count=1k | nc -vvn <server IP> 12345

Wednesday, April 1, 2015

ramdisk for chromium

I just bought two new ram for my desktop.
HyperX FURY black DDR3-1866 8GB(8GBx2) (HX318C10FBK2/16)
http://24h.pchome.com.tw/prod/DRAL1H-A90052AUN?q=/S/DRAL1I


/etc/fstab
tmpfs     /tmp/cache    tmpfs     nodev,nosuid,noatime,size=2G      0 0

$ chromium --disk-cache-dir=/tmp/cache

$ mount | grep cache
tmpfs on /tmp/cache type tmpfs (rw,nosuid,nodev,noatime,size=2097152k

manually create a ramdisk
$ mkdir -p /tmp/ram
$ sudo mount -t tmpfs -o size=1024M tmpfs /tmp/ram/


old post: http://xatierlike.blogspot.tw/2012/08/create-ramdisk-on-ubuntu.html


Tuesday, March 31, 2015

GnuPG notes

Some notes for GnuPG

# usually we can replace <key ID> with <user ID>
# keygen
gpg --full-gen-key (choose DSA & Elgamal here)


# editing
gpg --edit-key <key ID>


# key listing
gpg -k [ <user ID> or <key ID> ]
gpg -K [ <user ID> or <key ID> ]

-k: --list-public-keys / --list-keys
-k: --list-secret-keys


# fingerprint
gpg --fingerprint [ <key ID> ]


# import and export (backup & restore)
gpg --import filename

gpg --export <key id>
gpg --export-secret-keys <key id>

# --armor (-a): ASCII text format
gpg -a --export <key id>
gpg -a --export-secret-subkeys <key id>

gpg --enarmor filename.gpg
gpg --dearmor filename.asc


# keyserver
gpg --keyserver pgp.mit.edu --send-keys <key ID>
gpg --keyserver pgp.mit.edu --recv-keys <key ID>
gpg --keyserver pgp.mit.edu --search-key <key ID>


# encryption and decryption
gpg -e filename
gpg -r <key ID> -e filename
gpg -o filename -d filename.gpg

-e: --encrypt
-d: --decrypt
-o: --output
-r: --recipient


# signature
(in place signature)
gpg --sign filename
gpg --clearsign filename
gpg --verify filename.gpg

(saperated key)
gpg --detach-sign filename
gpg -a --detach-sign filename
gpg --verify filename.sig filename


# sign-key
gpg --sign-key <user id>


Tuesday, March 24, 2015

OpenVPN with VPN Gate

VPN Gate is a project by University of Tsukuba, Japan.
http://www.vpngate.net/en/

Basically there're thousands of relay servers hosted by volunteers around the world.

As a Linux user, the easiest way to connect to VPN Gate servers is OpenVPN.

Just install openvpn from the official repositories.

sudo pacman -S openvpn

and randomly grab a configuration file like this: vpngate_vpn197292320.opengw.net_udp_1786.ovpn

http://www.vpngate.net/en/do_openvpn.aspx?fqdn=vpn197292320.opengw.net&ip=223.223.103.92&tcp=1620&udp=1786&sid=1427186718705&hid=1019730

Then use openvpn client to read the config file and connect to the free internet:

sudo openvpn vpngate_vpn197292320.opengw.net_udp_1786.ovpn

I wrote simple Perl script to get OpenVPN configs from VPN Gate

https://gist.github.com/xatier/8911e8737089e9eaa236

That will show you a list of available VPNs and save the config file for you.

demo:



Reference:
http://www.vpngate.net/en/
https://wiki.archlinux.org/index.php/OpenVPN



Note for shadowsocks

Shadowsocks is a popular open sourced tunneling tool in China.

https://github.com/shadowsocks/shadowsocks
https://www.archlinux.org/packages/community/any/shadowsocks/

(both server and client)
sudo pacman -S shadowsocks

Salsa20 & Chacha20 support
sudo pacman -S libsodium python2-numpy python2-salsa20

/etc/shadowsocks/config.json
{
"server":"remote-shadowsocks-server-ip-addr",
"server_port":8888,
"local_address":"127.0.0.1",
"local_port":1080,
"password":"your-passwd",
"timeout":300,
"method":"aes-256-cfb",
"fast_open":false,
"workers":1
}


server
sudo ssserver -c /etc/shadowsocks/config.json --user nobody

run as daemon
sudo ssserver -c /etc/shadowsocks/config.json --user nobody -d start
sudo ssserver -d stop

client
sslocal -c /etc/shadowsocks/config.json

run as daemon
sudo sslocal -c /etc/shadowsocks/config.json -d start
sudo sslocal -c /etc/shadowsocks/config.json -d stop


Chromium: use Proxy SwitchyOmega and connect to a local socks5 proxy
https://chrome.google.com/webstore/detail/proxy-switchyomega/padekgcemlokbadohgkifijomclgjgif

Android Client
https://play.google.com/store/apps/details?id=com.github.shadowsocks

QR code for Android client
sudo pacman -S python2-qrcode
echo -n "ss://"`echo -n aes-256-cfb:password@1.2.3.4:8388 | base64` | qr


References:

https://github.com/shadowsocks/shadowsocks/wiki
https://wiki.archlinux.org/index.php/Shadowsocks_%28%E7%AE%80%E4%BD%93%E4%B8%AD%E6%96%87%29