There are more Sylvari then just the ones that grew out of the Pale Tree that we know.

Also, remember that the cave that had the seed for the Pale Tree had other seeds in it.