2014/09/30

Django series: Tricks on doing Test Driven Development

Preface

This post will not talk about how to write TDD in django. Instead, it covers how to do this HAPPIER (in my opinion).
In case, you do not know how to do TDD under the django framework, please take a look here.

Tricks

  • How to run test cases?

python manage.py test
  • How to colorize the above output?

Take a look here
Personally I am using the pyrg
  • How to change the TEST RUNNER?

Actually, you can change the test runner of django (EG: nose2) instead of using the default one. Take a look here.
Note that you should write this statement TEST_RUNNER='djnose2.TestRunner' instead of TEST_RUNNER=djnose2.TestRunner in order to make this work.
  • How to capture the output from the running tests (eg: pprint, logging etc) with COLOR?

pyrg -- manage.py test -- -B
The difficulty is how to pass options to your test runner.
The first -- tellspyrg that following (manage.py test -- -B) MUST BE arguments instead of OPTIONS. The second -- tells manage.py the -B MUST BE the ARGUMENT instead of the OPTIONS. Then, you can pass the -B options to your custom test runner (I am using nose2).

2014/09/22

To the world of social networking

Hello world! First blog post about this course and this blogger.

After attending first few lecture talks, I can feel the power and the potential of studying social networking. It is amazing that people invent intrinsic ways to analyze and model User Generated Contents by using Natural Language Processing and probability theory.

Actually, I am really interest on how modern social medias like facebook, google plus analyze user's behavior and how to make use of such kind of data to do advertisements pushing and promotions efficiently.

What I learnt from lecture 3 allows me to analyze users' content. Conceptually, by using term weighting, we are able to convert data from text based context to digits. Then, by using either or both of the vector space model and k-clustering, we are able to classify the type of the user context. Once we are able to categorize type of content, the remaining stuff is simply coding.

k-cluster


Besides the implementations, I am really excited about tricks on improving the accuracy of the classifications and how can those big companies profit from them. Hopefully, the following lectures will cover those issues one by one.

2014/09/15

Django series: About django-admin.py

Following documents what I have walked through from here
  • Relationship between django-admin.py and manage.py
manage.py is a wrapper of django-admin.py which take cares the following before delegated to django-admin.py:
* It puts your project’s package on sys.path.
* It sets the DJANGO_SETTINGS_MODULE environment variable so that it points to your project’s settings.py file.
* It calls django.setup() to initialize various internals of Django.
  • List of helpful commands
    • dumpdata, output all data in database related to the given apps
    • inspectdb, create a model for each table inside the database
    • loaddata, load data from fixture to the database
    • flush, erase all data from the database

2014/09/04

Java TDD = PROJECT + ANT + JUNIT

About

This blog is about what issues I have faced and tackle on writing a JAVA project with TDD (Test driven development). As you may not know, I am not an experienced Java developer. All the java coding skills came from a course 2 years ago.

Project

My goal is building up an application for fetching concert’s information from EVENTFUL. Although there is an official java binding provided, some of it’s dependencies (eg: XML) are not supported on ANDROID device by default. So, I am going to make a new one.

ANT

For brevity, I will say this tool works like grunt or makefile which ease the burdens of testing and compile your source code.
Typically, it required a makefile named as build.xml
Structure of the build.xml
XML := 1 * Project
Project := 1~N * Target
Target := 1~N * Task

FAQ

JUNIT

A popular testing framework. To be honest, all I need is the fabulous assertions.

FAQ

  • Installation on MAC BOOK AIR (MAC OS)
Command sudo port install junit works well. Make sure you have updated your port database if you are fail to issue the above command.
  • Compilation on command line
PATH is the most difficult issue in this part. Make sure you have included both junit.jar and hamcrest-core.jar on compilation.
  • How to run JUNIT on command line
2 points to note. The first one is make sure you have included all jar files on your java class path. The second one is how you call your test class. Below is an example
/opt/local/share/java/junit.jar:/opt/local/share/java/hamcrest-core.jar:.:./com/ideafactory/dataquery/test/PerformerTest.class org.junit.runner.JUnitCore com.ideafactory.dataquery.test.PerformerTest

2014/06/03

Install luci on ubuntu 12.04


Abstract


After struggling around 1 hour, I have successfully installed luci on my ubuntu VM. For future reference, here is the steps I have tried.

Dependencies


  • lua
  • cmake
  • uci
  • luci

# lua

Installation

    $> sudo apt-get install lua5.1 liblua5.1-0-dev

Notes

Hopefully, you can run lua interpreter on your terminal by typing lua directly on terminal for now on.


# cmake

Installation

$> sudo apt-get install cmake

# uci

Installation

Edited from here
# Some extra package are required:
$> sudo apt-get install cmake lua5.1

# The libubox library is required. It should to compiled from source.
# To do this, first you have to get the source from git:  
$> git clone git://nbd.name/luci2/libubox.git libubox
$> cd libubox

# Please follow the next steps to build libubox:
$> mkdir build
$> cd build
$> cmake ..
$> make ubox

# Install libubox:
$> sudo mkdir -p /usr/local/include/libubox
$> sudo cp ../*.h /usr/local/include/libubox
$> sudo cp libubox.so /usr/local/lib
$> sudo ldconfig

# Get UCI source from git:
$> git clone git://nbd.name/uci.git uci
$> cd uci

# Please follow the next steps to build uci:
$> mkdir build
$> cd build
$> cmake ..
$> make all

# Install uci:
$> sudo mkdir -p /usr/local/include/uci
$> sudo cp ../uci.h ../uci_config.h /usr/local/include/uci
$> sudo cp ../uci_blob.h ../ucimap.h /usr/local/include/uci
$> sudo cp libuci.so /usr/local/lib
$> sudo cp uci /usr/local/bin
$> sudo cp lua/uci.so /usr/local/lib/lua/5.1/
$> sudo ldconfig

# Testing:
$> mkdir test
$> cat > test/test << EOF
> config 'test' 'abc'
> option 'test_var' 'value'
> EOF

$> uci -c `pwd`/test show test
Output:
test.abc=test
test.abc.test_var=value

$> uci -c `pwd`/test set test.abc.test_var=foobar
$> uci -c `pwd`/test commit test

$> uci -c `pwd`/test show test
Output:
test.abc=test
test.abc.test_var=foobar

# luci

Installation

# According to the official news,
# luci's repository has been merged with openwrt.
# Below are updated commands for downloading their sources.
$> git clone https://github.com/openwrt/luci.git
$> cd luci
$> git checkout luci-0.12
# Remove builtin folders if fail
$> rm -fr contrib/uhttpd

# compilataion
$> make runshell

Notes


1. This is not a permanent installation

Above installation is not a permanent one, you need to run above commands for every terminal session in order to use luci.

Actually, missing environment variables are the root cause for this phenomenon. For advanced Linux users, I strongly recommend you put those environment variables (named with LUA_*) into your ~/.bashrc after calling make runshell so that you do not need to run them again and again.

Write down following contents at the end of ~/.bashrc
# make sure you have update the home directory :)
export LUA_PATH="/home/mondwan/Documents/git/luci/host//usr/lib/lua/?.lua;/home/mondwan/Documents/git/luci/host//usr/lib/lua/?/init.lua;$LUA_PATH;"
export LUA_CPATH="/home/mondwan/Documents/git/luci/host//usr/lib/lua/?.so;$LUA_CPATH;"
export LUCI_SYSROOT='/home/mondwan/Documents/git/luci/host'
export LD_LIBRARY_PATH='/home/mondwan/Documents/git/luci/host/usr/lib:'

2. Missing iwinfo.h

If it tells you about missing iwinfo.h, please read this link. Try to revert those files will be good enough. Below are extracted commands from the link.

$> git checkout 89678917~1 contrib/package/luci/Makefile
$> git checkout 89678917~1 modules/admin-full/src/luci-bwc.c

Test installation


Before doing anything, here is the script for testing whether you have installed luci successfully or not.

$> lua
> UCI = require 'luci.model.uci'
> UCI.cursor() 

if you can run them without errors you have accomplished the installation.

Further reading


References


* http://www.wakoond.hu/2013/06/using-uci-on-ubuntu.html

Update history


Update on 26/1/2015: Fix the incorrect directory structure CSS
Update on 16/2/2015: A new guide to compile luci
Update on 12/3/2015: Update text format and add notes about make runshell
Update on 21/5/2015: Add a reference forurther reading
Update on 2/6/2015:  Update bashrc contents
Update on 3/8/2015:  Update notes about installing uci
Update on 4/2/2016:  Update documents for UCI installation
Update on 26/8/2016: Update make commands for uci and luci installation

2014/05/10

Let's type cangjie on Ubuntu Hong Kong People

Tonight I am so happy that I can find (and install it successfully) a nice Cangjie software on Ubuntu which I am going to document down how to make it works.


By the way, this library originates from our Hong Kong People. Proud of you ahahahahah

Let's start.

To be honest, I believe they havn't written a good documentation for end user. Well.... common problem in open-source projects. Never mind. That's why I wrote this blog post.

Here is their PPA. There are 2 ways for installation.

Install by adding the ppa via add-apt-repository
  • sudo add-apt-repository ppa:haggenso/ibus-cangjie-ppa
  • sudo apt-get update
  • sudo apt-get install ibus-cangjie
If you fail (like me), here is another way.
  • sudo vi /etc/apt/sources.list
  • Paste following contents
deb http://ppa.launchpad.net/haggenso/ibus-cangjie-ppa/ubuntu precise main 
deb-src http://ppa.launchpad.net/haggenso/ibus-cangjie-ppa/ubuntu precise main 

  • sudo apt-get update
  • sudo apt-get install ibus-cangjie
  • Restart the keyboard icon (or logout, reboot etc) on your top-right hand corner
  • You can see a gear icon here if you get nothing wrong



















FAQ

  1. No keyboard icon?
Read this. It guides you how to setup an ibus on your ubuntu


Enjoy :D 
終於打到中文啦 哇哈哈哈哈哈

2014/04/16

Pattern explore - Decorator

Abstract

Currently, I am revising pattern design on Software Develop. In order to have a deep understanding and goooooood memorable experiences on the patterns I revised, I am going to drop down the notes or codes in this blog. So, let's start on decorator

Decorator

Intent:
1. How to implement the objects that give the new functionality.
2. How to organize the objects for each special case.

To summarize, it is a very good pattern for chaining up a sequences of commands!

Use case


Imagine that you have follow commands on your hand:

ping(), iperf(), ssh(), get_bw(), telnet()

Then, your task is doing a bandwidth measurement between 2 network devices. So you will came following command sequences.

Get_BW_BY_SSH() if (ping()) // check if those 2 devices are alive host = ssh() // connect to host ret = host.iperf() // run iperf() get_bw(ret) //process results....
It sounds perfect. However, what if I need to support the connection protocol from ssh to telnet? Then, you need to add a switch on connection part.

Get_BW(protocol) if (ping()) // check if those 2 devices are alive host = switch(protocol){ "ssh": ssh() "telnet": telnet() } ret = host.iperf() // run iperf() get_bw(ret) //process results....
Not bad. OK! what if I need you to color the results if BW higher than certain thersold ?

Get_BW_BY_SSH_WITH_COLOR(protocol, thresold) if (ping()) // check if those 2 devices are alive host = switch(protocol){ "ssh": ssh() "telnet": telnet() } ret = host.iperf() // run iperf() bw = get_bw(ret) //process results.... if (bw > thresold) color(bw)
Obviously, We need a new function with each new requirements. Moreover, client sides will need to know which one they should call in order to fulfill what they need. I hopes I can convince you that this is a bad design.

You may argue that you can keep them in one function so that we don't need to add codes on client sides. However, your function must be full of "if-then-else" or "switch" for logic control. It will totally mess up your codes.

Decorator allows you to solve them properly. Consider the following codes:

Class Task { construtor: function (Task t) { this.task = Task; } do: function (val) {//your specific code} } Class Ping extends Task { construtor: function (Task t) { this.task = Task; } do: function (val) { ret = ping(); this.task.do(ret); } } Class Iperf extends Task { construtor: function (Task t) { this.task = Task; } do: function (val) { ret = val.iperf(); this.task.do(ret); } } Class GetBw extends Task { construtor: function (Task t) { this.task = Task; } do: function (val) { ret = get_bw(val); this.task.do(ret); } } Class Color extends Task { construtor: function (Task t) { this.task = Task; } do: function (val) { ret = this.task.do(val); color(ret); } } operation = new Ping(new SSH(new Iperf(new Color(new GetBw())))) operation()
Although this is an incomplete code, it shows how decorator works.


  • Specific Tasks are inherited from Class::Task and implement the same interface (function, method, whatever you like to call) do().
  • Instantiating those specific tasks by passing another tasks to their constructor as a refernce allows chaining the operation.
  • By modifying how to instantiate those instance, you can manage which order you would like to run those tasks. 
  • The client side know nothing about it since they are still calling operation().
  • The ordering of this.task.do() and your specify code is not matter. Just like Class::Color, it calls GetBw() before running its business logic.


See? That's the power of Decorator.

Updated on: 20/4/2014

2014/04/08

Ubuntu 12.10 missing a desktop

Abstract

Today is a bad day. I found the damn ubuntu desktop gave me a blank wallpaper only while I would like to start working. Even by the help of google, it took me times for dealing with it. Damn! 

Anyway, I am going to listing down the methods I tried.

Methods

1. Re-install the graphic card driver

Actually, this is not work for me this time

2. Re-install ubuntu desktop

It works for me. However, I suggest you simply issue the following commands
sudo stop lightdm
sudo apt-get remove --purge ubuntu-desktop

sudo apt-get install ubuntu-desktop
sudo start lightdm
Note: Remember to reboot your computer for checking the solution work or not


2014/03/23

Install / Upgrade nodejs on ubuntu 12.10 / 13.04 /14.04

Abstract

According to my boss, I am going to develop a testing tool with UI and bandwidth testing within one week......
So, order is order and boss is boss. Since I need to complete this task anyway, I am going to develop on a cutting edge web application server nodejs :D

Let's go

I have checked out the official node js version on ubuntu 12.10 is 0.6.x. After googling a while, I found this link is very useful.

Recommend reference

what is node.js

Update on 2014/03/23

- This solution works on ubuntu 13.04

Update on 2016/01/26

- This solution works for Ubuntu 14.04

2013/12/12

Install php5-rrd module on ubuntu 12.04

Abstract

Since I am running php-5.4 on ubuntu 12.04, apt-get install php5-rrd is not work due to the problems of dependency. I need to do it via pecl alternatively.

Steps

1. Make sure your pecl is installed properly.
2. apt-get install php5-dev
3. pecl update-channels
4. pecl install rrd
5. cd /etc/php5/mods-available
6. cp {?.ini} rrd.ini 
//note that simply copy a file among this directory and modify the extension value to rrd.so
7. cd ../conf.d
8. ln -s ../mods-available/rrd.ini 20-rrd.ini

done

Check

php -v
php -m | grep rrd


2013/11/20

"%.*s" in printf()

Background

Currently, I am dealing with regex example in C.  There is a abnormal printf command.
......
printf ("'%.*s' (bytes %d:%d)\n", (finish - start),  to_match + start, start, finish);
......
Ouput:
$& is '1 is nice 2' (bytes 5:16)
I am wondering why there are 3 "%s" format options but there are 4 parameters to map those options.

Dig deeper...

To make the story short, "%.*s" is where the amazing happen. This option take 2 arguments: length of the string and the starting point of a string. (XXX: so ?) OK. Take a look the example code below.

#include <stdio.h>

int main(int argc, const char *argv[])
{
        char a[] = "0123456789";

        int len, offset;
        len = 4;

        for (offset = 0; offset < 3; offset++) {
                printf("%.*s\n", len, a + offset);
        }

        return 0;
}

Output:
0123
1234
2345

In other word, "%.*s" allows you to print a subset of a string without doing tedious jobs like following.

char *b = (char *) malloc(4+1);
memcpy(b, a+offset, len);
b[len+1] = '\0';
printf("%s\n",b);



2013/11/17

Introduction to enyoJS

Background

Diverged from xtuple paragraph, enyo is one of xtuple dependencies. Or, in other words, xtuple use this framework to build their client side UI.

Introduction

enyo js
http://enyojs.com/about/
http://enyojs.com/get-enyo/#Bootplate
Difference between enyo and jQuery
jQuery focus on manupliating HTML elements while enyo focus on encapsulating HTMLs as a module and reuse them

Dig deeper

1. Cloning their "blootplate" and play around

The 1st question I would like to ask is what blootplate is.
After reading a while from their reference, I understand several things.
a) You can produce a web page with a minify source easily if you can make use of their shell script and directory structure.
b) If you have not interested on building an APP they claimed like me(well... it means a HTML5 webpage), you can simply read source/App.js. This file control what "debug.html" look like.

2. Diving back to their developer guide

Well... their developer guide make more sense than then point 1 I just pass-through.
I understand 2 more keywords in enyo namespace:
kind: A kind is a JavaScript constructor for an object that's been defined using the enyo.kind method. (That's why you can write `new App()`)
enyo.Control: A kind provided by enyo's core. A factory for users like us to create enyo.instance with user-defined properties.
enyo.Control() VS enyo.kind:
var t = enyo.Control(.....);
t.renderInto(document.body);
enyo.kind({name:'test',kind:enyo.Control...});
var k = new test();
k.renderInto(document.body);


In other word, enyo.kind allows you to publish your enyo.instance to a global namespace as a constructor while enyo.Control return a pointer of your enyo.instance.

3. Walk through their example

Here is their tutorial and here is my github repository, which folk from enyo.bootplate, to play with the provided example.
Notes:
1. "published":
properties insides "published" will be given a setter and getter method.
2. "event handler":
There are 2 parameters, object that's the source of the event and the event object, will be passed into the event handler.
3. "jsonp example":
Since Twitter have updated their APIs, the original example does not work. I have coded another apps in order to try their Ajax wrapping. Not Bad :D

4. Summery

It is fun for me to code with enjo JS. However, there are no too much online sources for you. If you get problems with enjo JS, you better be able to solve it yourself........
PS: Currently, # of jquery tag on stackoverflow is 389,821 while enyo is 120....

5. Useful links