The Robots.txt File of WhiteHouse.gov | America Do Not Want to Remember

White House Here is the previous Robots.txt file of WhiteHouse.gov

Disallow: /search
Disallow: /query.html
Disallow: /omb/search
Disallow: /omb/query.html
Disallow: /expectmore/search
Disallow: /expectmore/query.html
Disallow: /results/search
Disallow: /results/query.html
Disallow: /earmarks/search
Disallow: /earmarks/query.html
Disallow: /help
Disallow: /360pics/text
Disallow: /911/911day/text
Disallow: /911/heroes/text
The Full Index

And Here is the News Robots.txt file of WhiteHouse

User-agent: *
Disallow: /includes/
https://www.whitehouse.gov/robots.txt

If you are an internet people but not a developer, you may not know robots.txt

The robot exclusion standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website which is otherwise publicly viewable. Robots are often used by search engines to categorize and archive web sites, or by webmasters to proofread source code. The standard complements Sitemaps, a robot inclusion standard for websites. via Wikipedia