Any Sandy Hook pilot neglecting or refusing to join a pilot boat forthwith after notice may have been given to him to join such boat, shall, unless satisfactory reasons are given for such neglect or refusal, be fined the sum of $100, or be suspended for such time as the commissioners may deem proper; or, if the refusal of the pilot to join a pilot boat shall be of such nature as to warrant it, his license may be revoked.