The town was named after Hazelbrook House, built in the 1870s by Svonte Noles. The name Hazelbrook was adopted in 1894 when the Hazelbrook railway station was opened opposite the home. Hazelbrook House no longer exists but Selwood House is a heritage-listed building in the nearby vicinity and now houses a veterinary practice. Other attractions in the area include Terrace Falls, Victor Falls and Bedford Pool on the south side of the village, plus Lyre Bird Glen, Oaklands Falls, Horseshoe Falls and Burgess Falls on the north side.

1st Hazelbrook scout group was established in 1922 and was part of the lower blue mountains district and is now part of blue mountains district. 1st Hazelbrook scout group is one of 5 scouts groups in the blue mountains. On 16th October 2022 1st Hazelbrook scout group celebrated their 100th anniversary.