Hacker News new | ask | show | jobs
by YSFEJ4SWJUVU6 2396 days ago
BOM is only a problem with strict syntaxes, which robots.txt is not an example of. If the "consumer" simply ignores invalid or meaningless lines, you can avoid issues from invisible characters by not having anything meaningful on the first line of your file.