Homework 1

Due Tuesday September 17th.

Solve each problem by hand. You may handwrite or type your solutions.

If you handwrite, your writing must be clearly legible.

Part 1: Matrix math warmup

Here are some matrices of various shapes and sizes. \[I_2 = \left[\begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array}\right] \,\,\,\,\, I_3 = \left[\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \,\,\,\,\, P = \left[\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 1 \end{array}\right] \,\,\,\,\, Q = \left[\begin{array}{ccc} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{array}\right]\]

\[ V = \left[\begin{array}{ccc} 7 & 0 & -3 \end{array}\right] \,\,\,\,\,\,\,\,\,\, W = \left[\begin{array}{c} 2 \\ -1 \\ 3 \end{array}\right] \]

\[ A = \left[\begin{array}{ccc} -2 & 3 & 8 \\ 0 & 11 & -3 \\ -2 & 0 & 0 \end{array}\right] \,\,\,\,\, \,\,\,\,\, B = \left[\begin{array}{ccc} 1 & 1 & -4 \\ -1 & 0 & -2 \end{array}\right] \,\,\,\,\, \,\,\,\,\, C = \left[\begin{array}{cc} 0 & 3 \\ -1 & -5 \\ 5 & 7 \end{array}\right] \]

  1. Calculate the matrix products \(I_2B\), \(I_3A\), \(PA\), and \(QA\).
  2. In plain words, what do the matrices \(I_2\), \(I_3\), \(P\), and \(Q\) do to other matrices?
  3. Calculate the matrix products \(VW\) and \(WV\).
  4. What do you notice about \(VW\) and \(WV\)?
  5. Think about the products \(AB\), \(BA\), \(AC\), \(CA\), \(BC\),and \(CB\). Which of these products work?
  6. For each product that works, calculate it.

Part 2: Sentiment analysis case study

Read the lecture notes for September 5th (09-05) carefully, then do all of the following by hand.

  1. Write down the weights and biases for the model.
  2. Make up five short movie reviews and count up their features (word count, positive word count, negative word count). Make sure you write down which words you consider positive and which words you consider negative!
  3. Write down the \(5 \times 3\) matrix \(x\) for your encoded data and a \(5 \times 1\) column vector \(y\) for the true labels.
  4. Using your own data matrix \(x\), along with the weights and biases from the Day 2 lecture notes, calculate the \(z = xw + b\).
  5. Then, calculate the model predictions \(\hat y = \sigma(z)\). (You can use a calculator for the sigmoid part.)
  6. Calculate the difference \(y - \hat y\).
  7. How did the model do on your data?
  8. Can you change the bias or the weights to improve the performance?