That particular conference phone only has an effective mic range of about 7 feet. After 5 feet or so, there would be a noticable drop in volume. How far are you from the phone?
Also, excessive background noise such as fans or even the HVAC system in the room can effect the echo cancellation on the phone which can limit the range of the mic below what it normally have. You could try taking it into a different room and testing to see if you notice a difference.
