This error is usually caused by an invalid resource name, so if you are using VISA and want to connect to an instrument at address 2 on your first gpib board, use "GPIB0::2::INSTR".
Also, check that you can communicate with your instrument in Measurement & Automation Explorer (MAX). You might also try the interactive control to make sure that the GPIB communication is working.
http://www.ni.com/support/gpib/max/ibic.htm
If this isn't helpful, let us know what development environment you are using and the functions / VIs you are using to try and connect to your instrument.