What are 301 Redirects?
A redirect to a URL other than the one typed in or clicked on by the user.
For example, www.mydomain.com/coolproduct could issue a 301 redirect to www.mydomain.com/products/categoryq/itemw/product1.html.
According to RFC 2616, 301 redirects are defined as follows:
"The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise."
Most organizations use 301 redirects as Vanity URLs. Vanity URLs are simply short catchy links which a human user can remember. Taking the example above, 'coolproduct' is much easier to remember than 'products/categoryq/itemw/product1.html.'. URL shorteners such as bit.ly and goo.gl use the same 301 redirect to point to a longer URL. You can see this by clicking this link to 301 Redirects - How Permanent is "Permanently"? which will have you come back to this line of text.
If you use a tool such as HTTPFox, you could find that very first time you click on the short URL it will actually go to the server at goo.gl and get a redirect response from that server. Any new clicks on the short URL will actually take the redirect response from the cache.
So, what is the Problem?
Previously, webmasters used to use the 301 redirects for the purpose of vanity URLs. Today, www.mydomain.com/coolproduct could be pointing to www.mydomain.com/products/categoryq/itemw/product1.html. But tomorrow, you might want it to point it to www.mydomain.com/products/categoryq/itemw/product1_ver10.html.
In this manner the webmasters had control over where the user went when he clicked on a link.
But after 14 years (note that the world wide web is currently 20 years old), browsers decided to change the way "Permanently" was interpreted.
Earlier when an user typed in www.mydomain.com/coolproduct, the browser would go to the server and get a 301 Redirect and take the user to the URL specified by the server. If you did this today you would go to product1.html and if you did this tomorrow, you would go to version 10 of the product.
But since of all late, all browsers cache the 301 Redirect from the server. For a typical internet user, his Browser Cache would last the entire lifetime of the PC or laptop he or she is using.
So the webmaster has lost all control over a 301 redirect and "Permanent" has become "forever"!
So, is there a solution?
If you have inherited a 301 Redirect, you are in deep trouble. But if you are creating new 301 redirects, you could be smart and avoid future headaches. I will not be going into the solutions, but will point to some excellent articles I have found on the net.