[Ether]

For completeness, here's how to solve the problem using constrained nonlinear optimization.

It's a bit easier to set up. All you need is the length and area as a function of a and b.

The area is the objective (to be maximized), and the length is a constraint on the values of a and b.