To Set the viewer to binary:
m_cViewer.GetPalette().SetType(4096);
I don't know where 4096 is defined...
To Set the viewer to grayscale:
m_cViewer.GetPalette().SetType(2048);
Again, I don't know where this is defined. To get these values I just inserted the line,
long paletteType=m_cViewer.GetPalette().GetType();
and ran the debugger to see what it set paletteType to when I changed the palette using the viewer's palette menu.