Jumping on the gravel with enough force should break it and allow you to fall through into hidden areas. Look for gravel all around the map when exploring.