Head toward the forest and just after you cross the covered bridge you'll see a ruin beside the road, which was once the home of the Fletcher family.
Inside is a fireplace with a carving at the back that used to show a man making arrows, but it's hard to make out these days.
Just below and beyond that is a concealed ladder that takes you into the cellar.
Go through the hole in the wall down there and you'll be in the river.  From there, just swim upstream until you reach the waterfall.