Second Derivative Test - If f'(x) = 0 at a point and f''(x) > 0 at this point, then f(x) has a local minimum at this point.

Let $s$ be the side of square and $a$ be side of equilateral triangle.

Given,

$4s + 3a = 36$

$ a = \frac{36-4s}{3}$ -- ( 1 )

Total area $A = s^2 + \frac{\sqrt{3}}{4}a^2$

Substituting for $a$ from (1) above,

$A = s^2 + \frac{\sqrt{3}}{4} ( \frac{36-4s}{3} )^2 $

$A = s^2 + \frac{4\sqrt{3}}{9} (9-s)^2 $

$\frac{dA}{ds} = 2s + \frac{4\sqrt{3}}{9}(2s-18)$

$\frac{{d^2}A}{ds^2} = 2 + \frac{8\sqrt{3}}{9} > 0 $

Hence based on second derivative test, a minimum happens for A wherever $\frac{dA}{ds} = 0 $

So,

$\frac{dA}{ds} = 2s + \frac{4\sqrt{3}}{9}(2s-18) = 0$

$s = \frac{8\sqrt{3}}{2 + \frac{8\sqrt{3}}{9}} = \frac{36\sqrt{3}}{9 + 4\sqrt{3}}$

Substituting in ( 1 ) and simplifying, we get

$a = \frac{108}{9 + 4\sqrt{3}}$

So size of two pieces are $4s$ and $3a$ with $s$ and $a$ with values as above.