Whenever any wharf, pier, bulkhead, piling or harbor structures, within the limits of said cities, has not been used for a period of three years and is no longer capable of being used for the purpose for which it was constructed, the Director of Commerce of such city may serve written notice upon the owner and lessee thereof, if any, requiring that, within 60 days from service of said notice, work commence for the removal of said wharf, pier, bulkhead, pilings or other harbor structure and to complete such removal within a period of one year from the date of service of such notice.