I have posed a question here regarding the determination of the rank of a matrix expression, which has yet to receive a complete answer. Therefore, I attempted to symbolically define these matrices in MATLAB for a specific case (N=n=3, m=p=1) and wrote a program to calculate their rank, hoping this would provide insights for determining the rank of the matrix expression. Since I was not successful in this attempt, I provide my code in hopes of receiving guidance.

This is my code:

syms A  [3 3] matrix
syms B  [3 1] matrix
syms C  [1 3] matrix
B_N=[A*A*B A*B B]
O_N=[C;C*A;C*A*A]
T_N=[0 0 0;C*B 0 0;C*A*B C*B 0]
M=[B_N-(A^3)*O_N^(-1)*T_N  (A^3)*O_N^(-1)]
rank(M)

When I run the code this error occurs:

Error using svd
First input must be single or double.

Error in rank (line 14)
s = svd(A);

Error in Rank_of_M_symbolic_math (line 8)
rank(M)

My questions:

1.How should I change my code to achieve the goal?

2.Is MATLAB suitable for this purpose? If not, what is the best tool for that?

Edit:

I know that my MATLAB code is not sufficient to achieve the goal because, for example, the full-rank condition of matrix O_N has not been taken into account. However, I don’t know how to implement this. It seems that the rank of matrix M should be expressed in terms of the ranks of other matrices