@samgrover Yep. In a way, well-defined user agents and robots.txt is a convention for that "saying no" part.