The provisions of this subtitle do not apply to agricultural land management practices, construction of agricultural structures, or, except in Calvert County, to construction of single-family residences or their accessory buildings that disturb an area of less than one-half acre and occur on lots of two acres or more. Regardless of planning, zoning, or subdivision controls, a county or municipality may not issue a permit for grading or construction of any building, other than those matters exempted by the provisions of this section, unless the grading or construction conforms with plans approved as provided in this subtitle.