Question

A matrix function $$ Q(t)=\left[\begin{array}{cccc} q_{11}(t) & q_{12}(t) & \cdots & q_{1 s}(t) \\ q_{21}(t) & q_{22}(t) & \cdots & q_{2 s}(t) \\ \vdots & \vdots & \ddots & \vdots \\ q_{r 1}(t) & q_{r 2}(t) & \cdots & q_{r s}(t) \end{array}\right] $$is said to be differentiable if its entries \(\left\\{q_{i j}\right\\}\) are differentiable. Then the derivative \(Q^{\prime}\) is defined by $$ Q^{\prime}(t)=\left[\begin{array}{cccc} q_{11}^{\prime}(t) & q_{12}^{\prime}(t) & \cdots & q_{1 s}^{\prime}(t) \\ q_{21}^{\prime}(t) & q_{22}^{\prime}(t) & \cdots & q_{2 s}^{\prime}(t) \\ \vdots & \vdots & \ddots & \vdots \\ q_{r 1}^{\prime}(t) & q_{r 2}^{\prime}(t) & \cdots & q_{r s}^{\prime}(t) \end{array}\right] $$ (a) Prove: If \(P\) and \(Q\) are differentiable matrices such that \(P+Q\) is defined and if \(c_{1}\) and \(c_{2}\) are constants, then $$ \left(c_{1} P+c_{2} Q\right)^{\prime}=c_{1} P^{\prime}+c_{2} Q^{\prime} $$ (b) Prove: If \(P\) and \(Q\) are differentiable matrices such that \(P Q\) is defined, then $$ (P Q)^{\prime}=P^{\prime} Q+P Q^{\prime} $$

Short answer

In this exercise, we proved two properties of differentiable matrices. First, we proved that for differentiable matrices P and Q and constants c1 and c2, the derivative of their sum with scalar multiples is equal to the sum of their derivatives with the corresponding scalar multiples: (c1P + c2Q)' = c1P' + c2Q'. Second, we proved that for differentiable matrices P and Q, the derivative of their product is equal to the sum of the product of their derivatives in a specific order: (PQ)' = P'Q + PQ'.

Step-by-step solution

Define the given matrices and their derivatives

Consider the matrices \(P\) and \(Q\) as given, such that their sum is defined. That means that \(P\) and \(Q\) have the same dimensions. Let their derivatives be \(P'\) and \(Q'\).

Calculate the derivative of the sum of scalar multiples of the matrices

Let us look at the matrix \((c_1P + c_2Q)\). To calculate its derivative, we first note that the derivative of the sum of two functions is the sum of their derivatives. That is, $$ \left(c_{1} P+c_{2} Q\right)' = c_{1} P' + c_{2} Q'. $$ This holds since the derivatives are calculated element-wise, and the distributive property of scalar multiplication holds for matrices. Hence, we have proved the first property: \(\left(c_{1} P+c_{2} Q\right)' = c_{1} P' + c_{2} Q'\). (b) Prove that \((P Q)^{\prime}=P^{\prime} Q+P Q^{\prime}\)

Define the given matrices and their derivatives

Consider the matrices \(P\) and \(Q\) as given, such that their product \(PQ\) is defined. If \(P\) is an \(m \times n\) matrix and \(Q\) is an \(n \times p\) matrix, then \(PQ\) is an \(m \times p\) matrix. Let their derivatives be \(P'\) and \(Q'\).

Calculate the derivative of the product of the matrices

To calculate the derivative of the product, we need to differentiate each element of the product matrix \(PQ\). Let \((PQ)_{ij}\) be the element in the \(i\)-th row and \(j\)-th column of the product. We have: $$ (PQ)_{ij} = \sum_{k=1}^n P_{ik}Q_{kj}. $$ Now, we differentiate it with respect to \(t\) to find the derivative, \((PQ)'_{ij}\): $$ (PQ)'_{ij} = \frac{d}{dt}\left(\sum_{k=1}^n P_{ik}Q_{kj}\right) = \sum_{k=1}^n \frac{d}{dt}(P_{ik}Q_{kj}). $$ Applying the product rule of differentiation and using the fact that the derivatives of \(P\) and \(Q\) are \(P'\) and \(Q'\), we get $$ \sum_{k=1}^n \left(\frac{d P_{ik}}{dt}Q_{kj} + P_{ik}\frac{d Q_{kj}}{dt}\right) = \sum_{k=1}^n (P'_{ik}Q_{kj} + P_{ik}Q'_{kj}). $$ Recall that \((P'Q)_{ij} = \sum_{k=1}^n P'_{ik}Q_{kj}\) and \((PQ')_{ij} = \sum_{k=1}^n P_{ik}Q'_{kj}\). Therefore, the expression above becomes $$ (PQ)'_{ij} = (P'Q)_{ij} + (PQ')_{ij}. $$ As this holds for all \(i\) and \(j\), we have proved the second property: \((PQ)' = P'Q + PQ'\).

Key concepts

Differentiable Matrices

Understanding differentiable matrices is the first step in grasping matrix calculus, which is a crucial tool in various fields, including machine learning and theoretical physics. A matrix is considered differentiable if each of its entries is a differentiable function with respect to some variable, typically time or another scalar parameter. Mathematically, if we have a matrix function

\[ Q(t) = \begin{bmatrix} q_{11}(t) & q_{12}(t) & \cdots & q_{1n}(t) \ q_{21}(t) & q_{22}(t) & \cdots & q_{2n}(t) \ \vdots & \vdots & \ddots & \vdots \ q_{m1}(t) & q_{m2}(t) & \cdots & q_{mn}(t) \end{bmatrix} \]
each entry function \( q_{ij}(t) \) must have a derivative \( q_{ij}'(t) \). The derivative of the matrix \( Q \) is then straightforwardly defined as the matrix of the derivatives of its entries:

\[ Q'(t) = \begin{bmatrix} q_{11}'(t) & q_{12}'(t) & \cdots & q_{1n}'(t) \ q_{21}'(t) & q_{22}'(t) & \cdots & q_{2n}'(t) \ \vdots & \vdots & \ddots & \vdots \ q_{m1}'(t) & q_{m2}'(t) & \cdots & q_{mn}'(t) \end{bmatrix} \]
This concept is akin to differentiating a vector of functions component-wise. However, for matrices, each entry is differentiated separately to form the derivative matrix.

Product Rule for Matrices

An extension of the single-variable product rule to matrices is the product rule for matrices. Just as with scalars, when differentiating a product of matrices, one cannot simply differentiate each matrix separately and then multiply the results. Instead, it requires careful application of the product rule, taking into account that matrix multiplication is not commutative.

The matrix product rule states that if you have two differentiable matrices \( P \) and \( Q \) of appropriate dimensions such that their product \( PQ \) is defined, the derivative of their product is given by:

\[ (PQ)' = P'Q + PQ' \]
Here, \( P' \) signifies the derivative of \( P \) with respect to our variable, and similarly, \( Q' \) is the derivative of \( Q \). This formula is derived by applying the product rule to each element of the product matrix, considering the fact that each element is a sum of products of entries from matrices \( P \) and \( Q \). Understanding this concept is crucial as it is commonly used when differentiating expressions involving matrix equations, which appear frequently in applied mathematics and optimization problems.

Derivative of Matrix Product

The derivative of a matrix product builds upon our understanding of the matrix product rule. To elaborate on the provided exercise, let's dive into the step-by-step differentiation of the product of two matrices \( P \) and \( Q \). Suppose \( P \) is an \( m \times n \) matrix and \( Q \) is an \( n \times p \) matrix, resulting in their product \( PQ \) being an \( m \times p \) matrix.

Each entry of \( PQ \) results from the summation of the products of corresponding entries from rows of \( P \) and columns of \( Q \):

\[ (PQ)_{ij} = \sum_{k=1}^n P_{ik}Q_{kj} \]
The derivative of this sum-product expression can be found by applying the product rule to each term individually:

\[ (PQ)'_{ij} = \sum_{k=1}^n (P'_{ik}Q_{kj} + P_{ik}Q'_{kj}) \]
When everything is put together, we get the derivative of the entire matrix product as a sum of two products:

\[ (PQ)' = P'Q + PQ' \]
Here, \( P'Q \) represents the matrix formed by multiplying each entry of the derivative matrix \( P' \) by the corresponding entry of matrix \( Q \), and vice versa for \( PQ' \). This understanding not only helps in theoretical exercises but also in practical applications, such as when dealing with the sensitivity analysis of systems described by matrix equations.