MODX Community

MODx is serving HTTP 404 to Googlebot on a page that renders in the brower

#1

-Hello All

My website, https://nolaarttherapy.com/, is serving HTTP/2 404 to Googlebot (and every other crawler I’ve tested) for a page that it is serving HTTP/2 200 to browsers. This was happening to the entire site. I changed the resource alias (URLs) for the interior pages and that cleared up the issue for the interior pages. Unfortunately changing the resource alias for the home page has no effect.

I have other MODx sites on the server and none of them have this issue.

I’m running
MODX Revolution 2.7.1-pl
PHP 7.2.12
nginx 1.14.2

Thanks,
Richard

1 Like
#2

can you please share Screen shot of your error? i am not understanding whats going on your side.

#3

You will have to check webserver logs to find out what is actually happening. There is no way we could help as an outsider here.

#4

Here are screenshots from Google Search Console.

404
404.jpg1472×1232 308 KB

#5

url-inspection
url-inspection.jpg1472×1232 303 KB

#6

crawl-anomaly
crawl-anomaly.jpg1472×596 142 KB

#7

I checked the error log and the only thing I see are things like this.

[Sun Oct 13 22:31:43 2019] [error] [client 79.137.69.95] ModSecurity: Warning. Match of “rx ^POST$” against “REQUEST_METHOD” required. [file “/etc/apache2/mod_security/custom/wpbrute.conf”] [line “15”] [hostname “nolaarttherapy.com”] [uri “/gs-bin/php-latest/index.php”] [unique_id “XaQIPs26uA4AAIBbc-sAAAAR”]

#8

what is /gs-bin/php-latest/index.php ?

Also can you try to find in log the IP of the Googlebot or any other crawler to find out the exact message.
From the look of the message, it seems when Googlebot hits your website, something tries to POST data somewhere and it gets blocked by ModSecurity.

#9

Thanks for the suggestions.

I don’t have access to /gs-bin/php-latest/index.php. From what I’ve researched it looks like it contains the latest PHP version. “/gs-bin/php-latest/index.php” frequently comes up as the the value for “ORIG_SCRIPT_NAME”.

I collected a list of Googlebot IP addresses from my access log. None of the Googlebot IP addresses from the access log are in the error log.