HSTS or Strict Transport Security is a standard defined in RFC 6797, by which a web server can declare to a client that it should only be accessible via HTTPS. The web server or crawler will then make all future requests over HTTPS. This will be the case even if you are following a link to an HTTP URL. From there, SEO Spider displays a status code of 307 and an HSTS policy status and HSTS policy redirect type.
This redirect is internal representation in SEO Spider and the browser. This differs from a 301 or 302 in that it is not sent by the web server as it is reversed internally. When a web server declares that it should be contacted over HTTPS, an expiration on it – this 307 response is ideal because it signifies a temporary redirect.
The HSTS protocol is based on sending a single header by the server. This is called Strict-Transport-Security and is only sent over HTTPS as if sent over HTTP it was ignored. The header requires 2 associated directives max-age and includeSubDomains.
Max-age is mandatory and allows the server to know the number of seconds during which it can only be contacted by HTTPS. IncludeSubDomains is an optional field which, if set, indicates that the HSTS policy applies to all subdomains.
There are several advantages to using HTTP -> HTTPS Redirection. It reduces communication over unsecured protocols, reduces the load on the web server, and improves performance because a round trip is avoided when the HTTP link is encountered.
Site wide HTTP-> HTTPS redirect is always required due to the Strict-Transport-Security header ignoring it, unless sent over HTTPS. If the first visit to your site is not over HTTPS, you still need that initial redirect to HTTPS to provide the Strict-Transport-Security header. Given this, you can’t expect to see a 307 in the SEO Spider, but make an HTTP request for the robots.txt file, receive a 301 at the HTTPS version of the site, and then receive the Strict header. -Transport-Security, so will then report 307 for the first URL crawled. If robots.txt is disabled, the SEO spider check will report a 301.
How to turn off HSTS
This can be easily done by unchecking: Configuration « Respect HSTS policy » under « Configuration> Spider> Advanced » in SEO Spider.
The SEO Spider will ignore the HSTS completely and report any underlying redirects and status codes.