Mathematical Foundation: Time to Dust Up That Linear Algebra

Matrices, complex numbers, and tensor products are the holy trinity of quantum computing. During my quantum learning curve on the mathematical background, I wondered as you may: Where is the quantum magic in all this? How can a complex number or a matrix tell me about superposition or entanglement? Where is the spookiness? My immature understanding of superposition was that a system existed in multiple states at the same time: Like the cat in the box, it is dead and alive at once. However that is incorrect, the cat cannot be dead and alive at the same time just like a quantum system cannot exist in two simultaneous states.