This is called colinearity - you can check for it by comparing the rank of the matrix to its number of columns. In R qr(X)$rank. Good to add this to your EDA workflow.