Risk description:
There is no particular security risk in having a robots.txt file. However, this file is often misused to try to hide some web pages from the users.
This should not be done as a security measure because these URLs can easily be read from the robots.txt file.
Recommendation:
We recommend you to remove the entries from robots.txt which lead to sensitive locations in the website (ex. administration panels,
configuration files, etc).
More information about this issue:
https://www.theregister.co.uk/2015/05/19/robots