Review the control and function help in the function panel for ibrsp. Note the difference between the Serial Poll Response (char, passed by reference) and the Status Returned (int).
The Serial Poll Response is a char and will always be less than 256. When you cast a char as an int, if the most significant bit is a 1 (if the char is >= 128), the int will be negative. You don't need to cast the char as an int to interpret it. You can do bit-wise operations directly on the char. The function panel help for the Serial Poll Response states "If bit 6 of the response byte is set, the device is requesting service. Other bit values are device dependent." You'll need to see the documentation for the device you're talking to to interpret the rest of the bits.
The Status Ret
urned is an int and may be greater than 256. It is not "an int written on a byte". See the control help in the function panel for the definition of return values.
int device, statusReturned;
char pollResponse;
...
statusReturned = ibrsp (device, &pollResponse);
if (pollResponse & 0x40)
printf("Device %d requests service.\n", device);
if (statusReturned & ERR)
printf("Serial poll returned error.\n");