Advices

This is an article that I'd like update regularly. It should be a less technical article. I will write down some of the philosophies about life and work that I agree with or at least want to give it a try. Happy Learning :)

Pomodoro technique:

Concentrate on your work for 20 mins (without any distraction, like Facebook) and take a break for 5 mins. So you have to do something within the 20 mins. This technique also helps avoiding procrastination. 

Coding:
Sometimes you can see a problem in a different way and rewrite it so that a special case goes away and becomes the normal case.  -- Linus Torvalds

Profession:
Don't chase money, chase happiness. -- Liz Wessel

Continue reading →

Stanford Machine Learning Week 5 review

I just finished the 5th week of Stanford Machine Learning course:Neural Networks: Learning. Since this week's course is a little bit difficult. I thought I might as well write something as a reminder, so that I could look back in the future.

I do feel that it is a powerful way to predict the outcome when there's enough training data, hidden layers and intermediate neurons. A simple and practical neural network would consist of three layers: input layer, hidden layer, and output layer. Actually, one of the first versions of self-driving car was built on a three-layer neural network.

The forward propagation is straightforward and intuitive. On the other hand, it's a bit difficult for me to grasp the concept of back propagation. The good news is that I somehow managed to understand the implementation.

The Neural Network Learning Algorithm on a high level

• Calculate the cost function, given multiple matrices of thetas* (one matrice per layer). In the end, we should have a cost given thetas. The cost represents how "far" our prediction is from the "reality".
• Calculate the gradient (a.k.a. partial derivative) for each given theta. In the end, we should have a concrete numerical gradient value for each given theta. The back propagation take place on this step.
• With the ability to calculate the cost, and gradient for each theta, we can use one of the optimised functions such as fminunc (or gradient descent) to do the following iteration: random initial thetas --> calculate cost and gradients --> update thetas --> less cost and new gradients --> update thetas --> less cost and new gradients --> ... ---> until we get the minimum cost. This is a glorious moment when we get the optimised thetas, which allows the neural network to do the most accurate prediction.

*theta is the weight of X, whereas X is the feature vector used to predict outcome.

 

Questions

Though I finished the assignment of this week, there are still some parts that I need to do more research on to understand better:

• What exactly does δ (delta) represent in back propagation?
• Why do we use derivative sigmoid function g'(z) to calculate the δ (delta) from last layer back to the hidden layers?
• Why δ(l+1)*(a(l))T is the gradient (a.k.a. partial derivative) matrice at l layer?

Accomplishment

• Built a neural network that recognises 1 - 9 digital number imagines with 96% accuracy.
• Visualized hidden layer images, each of which represents a row of theta in the input layer, who calculates one neuron in the hidden layer. There are 25 neurons in the hidden layer.
• 100% code score passed.

Continue reading →

How the compiler understands your Java class

I have the following static factory method that creates a list view out of an int array: In "Effective Java", Joshua Bloch mentioned this as an

Adapter that allows an int array to be viewed as a list of Integer instances

However, I remember that Adapter pattern uses composition and the instance of the anonymous list implementation should therefore use the int[] as a member field. To verify the hypothesis and understand how the class is understood by the compiler, one can use the following command:

javac -d . -XD-printflat ListFactory.java

From the output, one can clearly understand how the final parameter is passed to the inner anonymous class implementation:

Continue reading →

Caveat - JVM cannot check a generic cast at runtime

The Java Generics is based on Erasure, which means the type check is only done at compile time, and at runtime the type information is erased.

The following code will throw an ClassCastException, because there is no way the JVM can check the cast at runtime.

 public static void main(String[] args) {
  List<String> strings = createList(1);
  String string = strings.get(0); //ClassCastException here
  System.out.println(string);
 }
 
 public static <E> List<E> createList(Object object) {
  List<E> list = new ArrayList<>();
  E element = (E)object;
  list.add(element);
  return list;
 }

At runtime, the list object's type is just List. When using type parameter E to cast the object, the element's type is just Object and it's legal to add an Object to List. But when trying to get an element from the strings, the cast (generated by compiler) throws an ClassCastException as an Integer cannot be casted into a String.

Continue reading →

wsgen generates Java classes used by JAX-WS (in particular JAXB) to marshal/unmarshal

When a SOAP web service interface is of document style, we used to use wsgen utility (a binary tool released with JDK) to generate, from the SIB (Service Implementation Bean) the artifacts - usually Java classes. These Java classes represent request and response of the operations the web service provides. You can think of these Java classes as the counterparts of XSD types.

The underlying Java WS libraries, especially the JAXB, will marshal Java instances into XML document instances (XML instances are defined in XSD in WSDL), or unmarshal an XML document instance to an Java instance.

The more recent version of JAX-WS releases the developers from the tedious wsgen, by letting the Endpoint publisher automatically generates the wsgen artifacts if the programer doesn't.

Continue reading →