Step 1:
We have $\cos^2A+\cos ^2B+\cos ^2C$
$\sin ^2B+\cos^2B=1$
$1-\sin^2B=\cos^2B$
$\Rightarrow \cos^2A+(1-\sin ^2B)+\cos ^2C$
$\Rightarrow (\cos ^2A-\sin^2B)+\cos^2C+1$
$\Rightarrow \cos(A+B)\cos(A-B)+\cos^2C+1$
Step 2:
The above equation can be written as
$\cos(\pi-C)\cos(A-B)+\cos^2C+1$
$\Rightarrow -\cos C\cos(A-B)+\cos^2C+1$
$\Rightarrow -\cos C[\cos (A-B)-\cos C]+1$
$\Rightarrow -\cos C[\cos(A-B)-\cos\{\pi-(A+B)\}]+1$
$\Rightarrow -\cos C[\cos(A-B)+\cos(A+B)]+1$
$\cos(A-B)+\cos(A+B)=2\cos A\cos B$
$\Rightarrow -\cos C(2\cos A\cos B)+1$
$\Rightarrow 1-2\cos A\cos B\cos C$
Hence (d) is the correct answer.