Mimicked version of Alice's Adventures in Wonderland

Why?

This is an easy algorithm I coded to practise Python.

I've been studying Machine Learning for a while, mainly by following courses on Coursera. Even though I have some occasions to implement the training algorithms such as Gradient Descent for Linear Regression, Logistic Regression for Classification; I mainly used Matlab, which is an efficient and practical tool and very easy to get start with. However, I think it would also be great to learn to use some of the ML libraries in Python, such as scikit-learn, which is already tuned and optimised by a lot of data scientists.  I also want to start doing some of the problems on Kaggle, and I'm not sure if Matlab is a perfect tool for that. This is why I decided to learn python by following Google Python Class.

What's the mimicked version of Alice's Adventures in Wonderland?

Basically, you pick a random word in the book. I picked the first word in the book, which is "Alice". Then you shall follow these steps to generate your version of the book:

• Print the chosen word, and then look up what words might come next in the original novel, then pick one at random.
• Do this recursively for 200 times.

Can you show me the mimicked version?

Here's the mimicked version I generated with the algorithm mentioned above :)

Alice's elbow was gently brushing away quietly marched off together, Alice a minute, `and I can be seen: she spread his book, `Rule Forty-two. ALL RETURNED FROM HIM TWO--" why, I ought to a well?' The first saw the mistake about for her eyes were of him), while she is asleep again,' said after her: its full of the opportunity of things--I can't have signed at once set to feel very dull!' `You might belong to my youth,' said the right THROUGH the Mouse with such nonsense!' `I feared it might knock, and his pocket, and anxious.) Alice to be civil, you'd better with the sea,' the rattle of the Cat went on a week: HE was.' `I feared it was certainly was, what? Alice called after it, and Grief, they don't believe there's nothing seems to be a thick wood. `The fourth.' `Two days and then, saying to speak again. This was I the others. `Are they were nowhere to have to Alice said; but there stood the King. `Nearly two and hurried on, looking over their own business!' said Alice. `Why the stairs. Alice thought was coming. It sounded promising, certainly: but the Mock Turtle, and doesn't understand

Isn't it fun? :)

I'll continue following the rest of the ML course. Hope I could use what I learned one day to solve real world problems.

Continue reading →

Advices

This is an article that I'd like update regularly. It should be a less technical article. I will write down some of the philosophies about life and work that I agree with or at least want to give it a try. Happy Learning :)

Pomodoro technique:

Concentrate on your work for 20 mins (without any distraction, like Facebook) and take a break for 5 mins. So you have to do something within the 20 mins. This technique also helps avoiding procrastination. 

Coding:
Sometimes you can see a problem in a different way and rewrite it so that a special case goes away and becomes the normal case.  -- Linus Torvalds

Profession:
Don't chase money, chase happiness. -- Liz Wessel

Continue reading →

Stanford Machine Learning Week 5 review

I just finished the 5th week of Stanford Machine Learning course:Neural Networks: Learning. Since this week's course is a little bit difficult. I thought I might as well write something as a reminder, so that I could look back in the future.

I do feel that it is a powerful way to predict the outcome when there's enough training data, hidden layers and intermediate neurons. A simple and practical neural network would consist of three layers: input layer, hidden layer, and output layer. Actually, one of the first versions of self-driving car was built on a three-layer neural network.

The forward propagation is straightforward and intuitive. On the other hand, it's a bit difficult for me to grasp the concept of back propagation. The good news is that I somehow managed to understand the implementation.

The Neural Network Learning Algorithm on a high level

• Calculate the cost function, given multiple matrices of thetas* (one matrice per layer). In the end, we should have a cost given thetas. The cost represents how "far" our prediction is from the "reality".
• Calculate the gradient (a.k.a. partial derivative) for each given theta. In the end, we should have a concrete numerical gradient value for each given theta. The back propagation take place on this step.
• With the ability to calculate the cost, and gradient for each theta, we can use one of the optimised functions such as fminunc (or gradient descent) to do the following iteration: random initial thetas --> calculate cost and gradients --> update thetas --> less cost and new gradients --> update thetas --> less cost and new gradients --> ... ---> until we get the minimum cost. This is a glorious moment when we get the optimised thetas, which allows the neural network to do the most accurate prediction.

*theta is the weight of X, whereas X is the feature vector used to predict outcome.

 

Questions

Though I finished the assignment of this week, there are still some parts that I need to do more research on to understand better:

• What exactly does δ (delta) represent in back propagation?
• Why do we use derivative sigmoid function g'(z) to calculate the δ (delta) from last layer back to the hidden layers?
• Why δ(l+1)*(a(l))T is the gradient (a.k.a. partial derivative) matrice at l layer?

Accomplishment

• Built a neural network that recognises 1 - 9 digital number imagines with 96% accuracy.
• Visualized hidden layer images, each of which represents a row of theta in the input layer, who calculates one neuron in the hidden layer. There are 25 neurons in the hidden layer.
• 100% code score passed.

Continue reading →

How the compiler understands your Java class

I have the following static factory method that creates a list view out of an int array: In "Effective Java", Joshua Bloch mentioned this as an

Adapter that allows an int array to be viewed as a list of Integer instances

However, I remember that Adapter pattern uses composition and the instance of the anonymous list implementation should therefore use the int[] as a member field. To verify the hypothesis and understand how the class is understood by the compiler, one can use the following command:

javac -d . -XD-printflat ListFactory.java

From the output, one can clearly understand how the final parameter is passed to the inner anonymous class implementation:

Continue reading →

Caveat - JVM cannot check a generic cast at runtime

The Java Generics is based on Erasure, which means the type check is only done at compile time, and at runtime the type information is erased.

The following code will throw an ClassCastException, because there is no way the JVM can check the cast at runtime.

 public static void main(String[] args) {
  List<String> strings = createList(1);
  String string = strings.get(0); //ClassCastException here
  System.out.println(string);
 }
 
 public static <E> List<E> createList(Object object) {
  List<E> list = new ArrayList<>();
  E element = (E)object;
  list.add(element);
  return list;
 }

At runtime, the list object's type is just List. When using type parameter E to cast the object, the element's type is just Object and it's legal to add an Object to List. But when trying to get an element from the strings, the cast (generated by compiler) throws an ClassCastException as an Integer cannot be casted into a String.

Continue reading →