Thursday, November 19, 2015

C++ lambda inheritance

I was thinking about if we are able to inherit from a C++ lambda (say, anonymous class)

My first try was like this, use a helper function to forward the lambda body and store that as an private member of a class. But this is a HAS-A relationship, not really inherited from the lambda.

#include <iostream>
#include <utility>

template<typename lambda>
class Foo {
  lambda _t;
  int x = 5566;

public:
  Foo(lambda &&t) : _t(std::forward<lambda>(t)) {
    std::cout << "Foo()" << std::endl;
  }

  ~Foo() {
    std::cout << "~Foo()" << std::endl;
  }

  void operator()(void) {
    _t(x);
  }

  void operator()(int x) {
    std::cout << "Foo(int): " << x << std::endl;
  }
};

template<typename lambda>
Foo<lambda> make_class(lambda &&t) {
  return { std::forward<lambda>(t) };
}


int main (void) {
  auto qq = make_class([](int x) -> void {
    std::cout << "operator(int): " << x << std::endl;
  });

  qq();
  qq(7788);

  return 0;
}


Foo()
operator(int): 5566
Foo(int): 7788
~Foo()



suhorngT improved this as a real inheritance:


#include <iostream>
#include <utility>

template<typename lambda>
class Foo : public lambda {

public:
    Foo(lambda &&t) : lambda(t) {
      std::cout << "Foo()" << std::endl;
    }

    ~Foo() {
      std::cout << "~Foo()" << std::endl;
    }
};

template<typename lambda>
Foo<lambda> make_class(lambda &&t) {
  return { std::forward<lambda>(t) };
}


int main (void) {
  auto qq = make_class([](int x) -> void {
    std::cout << "operator(int): " << x << std::endl;
  });

  qq(123);
  return 0;
}




Foo()
operator(int): 123
~Foo()

After that I took out the make_class helper, and use decltype instead, make the syntax much better.


#include <iostream>

template<typename... lambda>
class Bar : public lambda... {

public:
  Bar(lambda... l) : lambda(l)... {
    std::cout << "Bar()" << std::endl;
  }

  void foo(void) {
    std::cout << "foo()" << std::endl;
  }
};


int main(void)  {
  int a = 55;

  auto gg = [a](int b) -> void {
    std::cout << "operator(): " << a << b << std::endl;
  };

  auto qq = Bar<decltype(gg)>(gg);

  qq(66);
  qq.foo();

  return 0;
}



Bar()
operator(): 5566
foo()


Reference:
http://cpptruths.blogspot.tw/2014/05/fun-with-lambdas-c14-style-part-2.html
http://cpptruths.blogspot.com/2014/03/fun-with-lambdas-c14-style-part-1.html

Friday, November 13, 2015

gdb pretty-print python path


On some of my dev machines (freaking Ubuntu 14.04), gdb can't find the pretty print python extension.

(^q^) r
Starting program: /home/xatier/xxxxx
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'


Let's print python path:


On Ubuntu 14.04
(^q^) python print(sys.path)
['/usr/share/gdb/python', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']

On arch linux
(^q^) python print(sys.path)
['/usr/lib/../share/gcc-5.2.0/python', '/usr/share/gdb/python', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-linux', '/usr/lib/python3.5/lib-dynload', '/usr/lib/python3.5/site-packages']


See? So basically the python scripts are under /usr/share/<gcc-ver>/python

$ ls /usr/share | grep gcc
gcc-4.8


Just add this line to your ~/.gdbinit :

python sys.path.append("/usr/share/gcc-4.8/python")


Now you are good.

(^q^) python print(sys.path)
['/usr/share/gdb/python', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages', '/usr/share/gcc-4.8/python']


STL vector out-of-bound access is an undefined behavior


Found a bug today, which is a well-known pitfall in C++.

Consider the following code, std::vector::operator[] won't perform any boundary check on the index, a out-of-bound access is an undefined behavior.

#include <vector>                                                               
#include <iostream>                                                             
                                                                                
int main (void) {                                                               
  std::vector<bool> a(10);                                                      
                                                                                
  for (auto x : a)                                                              
      x = true;                                                                 
                                                                                
  if (a[11] == false)                                                           
      std::cout << "gg";                                                        
                                                                                
  return 0;                                                                     
}


gg

----

On the other hand, std::vector::at will yield an exception on out-of-bound access.


#include <vector>                                                               
#include <iostream>                                                             
 
int main (void) {                                                               
  std::vector<bool> a(10);                                                      
 
  for (auto x : a)                                                              
      x = true;                                                                 
 
  if (a.at(11) == false)                                                           
      std::cout << "gg";                                                        
 
  return 0;                                                                     
}


terminate called after throwing an instance of 'std::out_of_range'
what(): vector::_M_range_check: __n (which is 11) >= this->size() (which is 10)

Friday, July 3, 2015

ffmpeg x264 lossless video encoding

I re-encoded my previous club class video clips for my Google drive quota.

The result of is extremely good.


$ ls -lh
-rw-r----- 1 xatier staff 2.8G Jun 29 07:13 Topic 20 - [Programming 1] Python 1.mov
-rw-r--r-- 1 xatier staff 1.6G Jul 3 00:24 Topic 20 - [Programming 1] Python 1.mp4
-rw-r----- 1 xatier staff 3.1G Jun 29 07:30 Topic 21 - [Programming 2] Python 2.mov
-rw-r--r-- 1 xatier staff 1.4G Jul 3 13:43 Topic 21 - [Programming 2] Python 2.mp4


For the x264 lossless encoding, you can use the following commands:

If you don't have time, use the 'ultrafast' preset:

# fastest encoding
$ ffmpeg -i input -c:v libx264 -preset ultrafast -qp 0 -c:a copy output

If you need a compressed encoding, use the 'veryslow' preset:

# best compression
$ ffmpeg -i input -c:v libx264 -preset veryslow -qp 0 -c:a copy output


Both examples will provide the same quality output. (-qp 0)


It took me over 2 hours for this video on my 2012-mid MacbookAir (1.8 GHz Intel Core i5 / 8 GB 1600 MHz DDR3).


$ ffmpeg -i Topic\ 20\ -\ \[Programming\ 1\]\ Python\ 1.mov  -c:v libx264 -preset veryslow -qp 0 -c:a copy py1.mp4
...
frame=459704 fps= 23 q=-1.0 Lsize= 1660358kB time=02:07:41.73 bitrate=1775.3kbits/s dup=1008 drop=0
video:1387489kB audio:264157kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.527454%


Update: il gnaggnoy remembered not only qp but also both qpmax and qpmin should be set to 0 in order to force a lossless encoding in x264.

Reference:
https://trac.ffmpeg.org/wiki/Encode/H.264
https://wiki.archlinux.org/index.php/FFmpeg



Monday, June 29, 2015

List Google Drive files by size

When you run out of the quota limit of your Google Drive service, you must want to look at which file is occupying your space.

For Google Drive, you can just use this link to see the list of uploaded files by size.

https://drive.google.com/#quota

Alternatively, you can hover over your storage usage in the bottom left corner and click the Google Drive icon, that will lead you to the same link as above.

Friday, June 26, 2015

Currency conversions in Google Spreadsheet

Just learned a tip in Google Spreadsheet.

To get the currency conversion rate between TWD-USD:

=GoogleFinance("CURRENCY:TWDUSD")




That will give you the rate from Google Finance database, history is also available.

=GoogleFinance("currency:USDTWD", "price", today()-10, today())






Other API parameters:
https://support.google.com/docs/answer/3093281


Reference:
http://googledocstips.com/2011/03/09/how-to-calculate-foreign-exchange/
http://stackoverflow.com/questions/20607627/on-google-spreadsheet-how-can-you-query-googlefinance-for-a-past-exchange-rate


Monday, June 22, 2015

Graduation


Alright, I just finished a big milestone in my life -- graduated from the college with a degree in computer science. I'm not a college kid anymore.


Actually, I have no idea about this certificate. For me, the only purpose of this "paper" is matching the requirement of my H-1B visa next year.


Requirement 2 - Your job must qualify as a specialty occupation by meeting one of the following criteria:
A bachelor’s degree or higher degree or its equivalent is normally the minimum requirement for the particular position;






Anyway, happy graduation.

Thanks to my parents, my sister, Dr. Sung, jserv, ... all of my friends, ... and lots of people surrounding me.



Thursday, June 11, 2015

Interactive debugging in Python

Just put this thing anywhere in your python code.




import code                                                                    
code.interact(local=locals())                                                  

import IPython                                                                 
IPython.embed()

Sunday, May 31, 2015

Rip Audio CDs in command line

We basically use two tools: cdparanoia and lame.

cdparanoia: Compact Disc Digital Audio extraction tool https://www.archlinux.org/packages/extra/x86_64/cdparanoia/
lame: A high quality MPEG Audio Layer III (MP3) encoder https://www.archlinux.org/packages/extra/x86_64/lame/


# grab CD information
$ cdparanoia -vsQ
cdparanoia III release 10.2 (September 11, 2008)

Using cdda library version: 10.2
Using paranoia library version: 10.2
Checking /dev/cdrom for cdrom...
Testing /dev/cdrom for SCSI/MMC interface
SG_IO device: /dev/sr0

CDROM model sensed sensed: ASUS DRW-24D1ST 1.00

Checking for SCSI emulation...
Drive is ATAPI (using SG_IO host adaptor emulation)

Checking for MMC style command set...
Drive is MMC style
DMA scatter/gather table entries: 1
table entry size: 131072 bytes
maximum theoretical transfer: 55 sectors
Setting default read size to 27 sectors (63504 bytes).

Verifying CDDA command set...
Expected command set reads OK.

Attempting to set cdrom to full speed...
drive returned OK.

Table of contents (audio tracks only):
track length begin copy pre ch
===========================================================
1. 11033 [02:27.08] 0 [00:00.00] no no 2
2. 18007 [04:00.07] 11033 [02:27.08] no no 2
3. 10100 [02:14.50] 29040 [06:27.15] no no 2
4. 13237 [02:56.37] 39140 [08:41.65] no no 2
5. 9050 [02:00.50] 52377 [11:38.27] no no 2
6. 11076 [02:27.51] 61427 [13:39.02] no no 2
7. 23342 [05:11.17] 72503 [16:06.53] no no 2
8. 13141 [02:55.16] 95845 [21:17.70] no no 2
9. 16703 [03:42.53] 108986 [24:13.11] no no 2
10. 12635 [02:48.35] 125689 [27:55.64] no no 2
TOTAL 138324 [30:44.24] (audio only)





# rip stuffs
$ cdparanoia -B
cdparanoia III release 10.2 (September 11, 2008)


Ripping from sector 0 (track 1 [0:00.00])
to sector 138323 (track 10 [2:48.34])

outputting to track01.cdda.wav

(== PROGRESS == [ | 011032 00 ] == :^D * ==)

outputting to track02.cdda.wav

(== PROGRESS == [ | 029039 00 ] == :^D * ==)

outputting to track03.cdda.wav

(== PROGRESS == [ | 039139 00 ] == :^D * ==)

outputting to track04.cdda.wav

(== PROGRESS == [ | 052376 00 ] == :^D * ==)

outputting to track05.cdda.wav

(== PROGRESS == [ | 061426 00 ] == :^D * ==)


...

outputting to track10.cdda.wav

(== PROGRESS == [ | 138323 00 ] == :^D * ==)

Done.





# convert to mp3 format
$ for i in `ls`; do lame $i; done
LAME 3.99.5 64bits (http://lame.sf.net)
Using polyphase lowpass filter, transition band: 16538 Hz - 17071 Hz
Encoding track01.cdda.wav to track01.cdda.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
Frame | CPU time/estim | REAL time/estim | play/CPU | ETA
5633/5633 (100%)| 0:04/ 0:04| 0:04/ 0:04| 35.102x| 0:00
-------------------------------------------------------------------------------------
kbps LR MS % long switch short %
128.0 1.8 98.2 99.9 0.0 0.0
Writing LAME Tag...done
ReplayGain: -0.3dB
LAME 3.99.5 64bits (http://lame.sf.net)
Using polyphase lowpass filter, transition band: 16538 Hz - 17071 Hz
Encoding track02.cdda.wav to track02.cdda.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
Frame | CPU time/estim | REAL time/estim | play/CPU | ETA
9193/9193 (100%)| 0:07/ 0:07| 0:07/ 0:07| 34.246x| 0:00
-------------------------------------------------------------------------------------
kbps LR MS % long switch short %
128.0 0.3 99.7 100.0 0.0 0.0
Writing LAME Tag...done
ReplayGain: +0.1dB

...


done!

Reference: http://www.cyberciti.biz/faq/linux-ripping-and-encoding-audio-files/

Thursday, May 14, 2015

skicka: Google drive command line tool

Install go and skicka

$ sudo pacman -S go
$ mkdir ~/go
$ export GOPATH=~/go
$ export PATH=$PATH:~/go/bin
$ go get github.com/google/skicka

Initialize the configuration and client id/secret key pairs from  https://console.developers.google.com/project
Read: https://github.com/google/skicka/blob/master/README.md


$ skicka init
2015/05/14 21:44:37 created configuration file /home/xatier/.skicka.config.
$ vim ~/.skicka.config



Oauth authentication for the first time
Generate ~/.skicka.metadata.cache and ~/.skicka.tokencache.json

$ skicka ls -l /
Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?***********************************
Enter verification code: ******************************************
Updating metadata cache: 
[========================================================================================] 99.99 % 37s


Support commands:

$ skicka
usage: skicka [skicka options] [command options]

Supported commands are:
cat Print the contents of the given file
download Download a file or folder hierarchy from Drive to the local disk
df Display free space on Drive
du Report disk usage for a folder hierarchy on Drive
fsck Check consistency of files in Drive and local metadata cache
genkey Generate a new encryption key
init Create an initial skicka configuration file
ls List the contents of a folder on Google Drive
mkdir Create a new folder or folder hierarchy on Drive
rm Remove a file or folder on Google Drive
upload Upload a local file or directory hierarchy to Drive



Testing with my hinet 100/40 home use plan (roughly around 1.5MB/s):


$ skicka upload GG.mp4 /
Files: 23.83 MB / 23.83 MB 
[========================================================================================] 100.00 % 15s
2015/05/14 22:13:11 Preparation time 1s, sync time 15s
2015/05/14 22:13:11 Updated 1 Drive files, 0 local files
2015/05/14 22:13:11 23.83 MiB read from disk, 0 B written to disk
2015/05/14 22:13:11 23.83 MiB uploaded (1.58 MiB/s), 0 B downloaded (0 B/s)
2015/05/14 22:13:11 4.72 MiB peak memory used

$ skicka upload GG.mp4 /
Files: 23.83 MB / 23.83 MB 
[========================================================================================] 100.00 % 13s
2015/05/14 22:14:19 Preparation time 1s, sync time 13s
2015/05/14 22:14:19 Updated 1 Drive files, 0 local files
2015/05/14 22:14:19 23.83 MiB read from disk, 0 B written to disk
2015/05/14 22:14:19 23.83 MiB uploaded (1.72 MiB/s), 0 B downloaded (0 B/s)
2015/05/14 22:14:19 5.49 MiB peak memory used

$ skicka upload GG.mp4 /
Files: 23.83 MB / 23.83 MB 
[========================================================================================] 100.00 % 13s
2015/05/14 22:14:41 Preparation time 1s, sync time 13s
2015/05/14 22:14:41 Updated 1 Drive files, 0 local files
2015/05/14 22:14:41 23.83 MiB read from disk, 0 B written to disk
2015/05/14 22:14:41 23.83 MiB uploaded (1.71 MiB/s), 0 B downloaded (0 B/s)
2015/05/14 22:14:41 5.49 MiB peak memory used

$ skicka upload GG.mp4 /
Files: 23.83 MB / 23.83 MB 
[========================================================================================] 100.00 % 15s
2015/05/14 22:15:01 Preparation time 1s, sync time 15s
2015/05/14 22:15:01 Updated 1 Drive files, 0 local files
2015/05/14 22:15:01 23.83 MiB read from disk, 0 B written to disk
2015/05/14 22:15:01 23.83 MiB uploaded (1.51 MiB/s), 0 B downloaded (0 B/s)
2015/05/14 22:15:01 5.49 MiB peak memory used

Large file test:

$ skicka upload movie.mkv
Files: 8.39 GB / 8.39 GB 
[========================================================================================] 100.00 % 2h43m59s
2015/05/15 01:18:24 Preparation time 1s, sync time 2h 43m 59s
2015/05/15 01:18:24 Updated 1 Drive files, 0 local files
2015/05/15 01:18:24 8.39 GiB read from disk, 0 B written to disk
2015/05/15 01:18:24 8.39 GiB uploaded (893.93 kiB/s), 0 B downloaded (0 B/s)
2015/05/15 01:18:24 16.07 MiB peak memory used



Encryption

$ SKICKA_PASSPHRASE=gg skicka genkey
; Add the following lines to the [encryption] section
; of your ~/.skicka.config file.
salt=************************************
passphrase-hash=************************************
encrypted-key=************************************
encrypted-key-iv=************************************

$ SKICKA_PASSPHRASE=gg skicka upload -encrypt GG2.mp4 /
Files: 23.83 MB / 23.83 MB 
[========================================================================================] 100.00 % 25s
2015/05/14 21:59:16 Preparation time 1s, sync time 26s
2015/05/14 21:59:16 Updated 1 Drive files, 0 local files
2015/05/14 21:59:16 23.83 MiB read from disk, 0 B written to disk
2015/05/14 21:59:16 23.83 MiB uploaded (928.17 kiB/s), 0 B downloaded (0 B/s)
2015/05/14 21:59:16 4.24 MiB peak memory used