A large window has the shape of a rectangle surmounted by an equilateral triangle.

Let the dimensions of the rectangle be $x$ and $y$.

Perimeter of the window $=x+y+x+x+y=12$

$\Rightarrow 3 x+2 y=12$

$\Rightarrow y=\frac{12-3 x}{2}$          ......(1)

Area of the window $=x y+\frac{\sqrt{3}}{4} \mathrm{x}^{2}$

$\Rightarrow A=x\left(\frac{12-3 x}{2}\right)+\frac{\sqrt{3}}{4} x^{2}$

$\Rightarrow A=6 x-\frac{3 x^{2}}{2}+\frac{\sqrt{3}}{4} x^{2}$

$\Rightarrow \frac{d A}{d x}=6-\frac{6 x}{2}+\frac{2 \sqrt{3}}{4} x$

$\Rightarrow \frac{d A}{d x}=6-3 x+\frac{\sqrt{3}}{2} x$

$\Rightarrow \frac{d A}{d x}=6-x\left(3-\frac{\sqrt{3}}{2}\right)$

For maximum or a minimum values of A, we must have

$\frac{d A}{d x}=0$

$\Rightarrow 6=x\left(3-\frac{\sqrt{3}}{2}\right)$

$\Rightarrow x=\frac{12}{6-\sqrt{3}}$

Substituting the value of $x$ in eq. $(1)$, we get

$y=\frac{12-3\left(\frac{12}{6-\sqrt{3}}\right)}{2}$

$\Rightarrow y=\frac{18-6 \sqrt{3}}{6-\sqrt{3}}$

Now,

$\frac{d^{2} A}{d x^{2}}=-3+\frac{\sqrt{3}}{2}<0$

Thus, the area is maximum when $x=\frac{12}{6-\sqrt{3}}$ and $y=\frac{18-6 \sqrt{3}}{6-\sqrt{3}} .$

Disclaimer: The solution given in the book is incorrect. The solution here is created according to the question given in the book.