This Federal-style building was built in 1840 as the county office building and was used as such until the courts moved in 1891.

A private owner purchased the building and the Haverhill Library Association moved in in 1916. In 1929, the property was sold to the Haverhill Library Association and for over 100 years it has been used as the community's library.

It was originally constructed with 5 by 3 bays, but two bays were added to make it 7 by 3. It has two stories, with Flemish and common bond brick exterior and a slate roof.

In the gable ends are triangular, louvered openings framed by a double row of bricks and a granite sill.

The high pitched gable roof has interior end chimneys at both the east and west elevations, each linked at the base forming a parapet.

The lateral eaves have an entablature made of brick with buck dentils. The close verges of the raking eaves are not included in the parapet. The 6/6 double-hung windows have granite lintels, with sills at the front elevation and louvered shutters.

The Entry was originally in the center of the front facade, but with the addition of two bays, it is now off-center.

The 8-panel door is slightly recessed with three-quarters sidelights, a 5-light transom window, rectangular granite lintel, and granite block side framing. There are paneled mullions between the door, the transom lights, and the sidelights which intersect at the corner blocks with a bull's-eye motif.

The entry at the north (back) elevation is a 6-panel door with three-quarters sidelights, plain enframement, and a rectangular granite lintel.