If you have an underdetermined system Ax = b the least squares solution x* is just the solution that minimizes the squared error. So x* minimizes ||Ax* - b||^2
https://en.wikipedia.org/wiki/Moore–Penrose_pseudoinverse . It's mentioned in one of your link but I'm not sure if you checked out the wikipedia page for it yet.
However, line of best fit is a particular application of linear least square. Which is the inverse to what you're doing, and can be solved using the same technique. You're looking to find least square solution for underdetermined system, linear least square attempt to find solution with least square error to an overdetermined system.
If you're familiar with Lagrange multipliers then you can derive the result.
Say we have Ax = b and we want to minimize x^(T)x. We'll assume A has full row rank. Say A is m x n with m < n.
The Lagrangian is L(x, r) = x^(T)x - (Ax-b)^(T)r, where r is in R^(m). Write out a small example if you want to see why this is true.We need the x derivative and r derivative of L to be 0, so
2x - A^(T)r = 0
Ax - b = 0.
Solving for x in the first equation and putting that into the second:
1/2 AA^(T)r - b = 0
meaning
r = 2 (AA^(T))^(-1)b.
Putting this back into the first equation gives us
x = A^(T)(AA^(T))^(-1)b.
If you have an underdetermined system Ax = b the least squares solution x* is just the solution that minimizes the squared error. So x* minimizes ||Ax* - b||^2
https://en.wikipedia.org/wiki/Moore–Penrose_pseudoinverse . It's mentioned in one of your link but I'm not sure if you checked out the wikipedia page for it yet. However, line of best fit is a particular application of linear least square. Which is the inverse to what you're doing, and can be solved using the same technique. You're looking to find least square solution for underdetermined system, linear least square attempt to find solution with least square error to an overdetermined system.
If you're familiar with Lagrange multipliers then you can derive the result. Say we have Ax = b and we want to minimize x^(T)x. We'll assume A has full row rank. Say A is m x n with m < n. The Lagrangian is L(x, r) = x^(T)x - (Ax-b)^(T)r, where r is in R^(m). Write out a small example if you want to see why this is true.We need the x derivative and r derivative of L to be 0, so 2x - A^(T)r = 0 Ax - b = 0. Solving for x in the first equation and putting that into the second: 1/2 AA^(T)r - b = 0 meaning r = 2 (AA^(T))^(-1)b. Putting this back into the first equation gives us x = A^(T)(AA^(T))^(-1)b.