by Hubert Chathi.
If you check that a URL is reachable, you have to be able to handle pages that the user is able to reach, but the server is not able to, for whatever reason. e.g. the page is password-protected, or the server is behind a restrictive firewall.
You'd probably also want to limit the characters that you would try chopping off -- probably just to punctuation characters. e.g. in the URL http://example.com/foo/bar)., the comma, period, and closing parenthesis are questionable whether they belong to the URL or not, but the rest of the URL is not questionable. This way, you don't hammer some other server with 100 requests when someone types in an incorrect URL.