Hey Martin, it looks like we resolved this issue. Apparently one of the APIs would fail if it was crawled too many times. They put a caching on it and the pages are getting and staying indexed now. Thanks for looking into it but wanted to let you know it's resolved now too.

Had a higher up send the article to me today to make sure our strategies aligned with what is mentioned in the article. I said I've been aligned for the last 7 years....

Hello! I wanted to see if there was any guidance from your team on what the crawlers are seeing from a stability stand point. We have Pingdom set up on several URLs to see if there are any 404s, but not seeing any. Appreciate any info that could be passed along to troubleshoot

Hello, just wanted to follow up and see if you had a chance to look into this. Thanks again!

I know I sent this info late on Friday night - so I'm pinging in case you missed it @divingfor.fun. Thanks for any insight you can provide

The rendering looks accurate.

Our homepage uses the same page building tool as our blogs and landing pages. I can see the index is picking up the content of the page in GSC but guessing since Google is choosing the canonical as the homepage, it is rendering the homepage as a soft 404 too.

It appears that sometimes these pages render as a soft 404 error, and when that happens, the pages loads the 404 error page and "Uh Oh" appears as the H1. Crawling the pages with Botify shows this as well as they show up as Duplicate H1s. But when putting it into the Inspect URL tool in GSC...

That resolved a lot of the issues, however it seems to be a problem again. Here is an example URL: www.art.com/blog/tips-ad...

Not seeing a DM option - basically most of our non-PLP and non-PDP pages are not getting indexed as Google is choosing a different canonical than our declared one. Pages come and go out of the index. We found a line in robots.txt that was blocking an API and removed it several months ago and...

I was going to say at least it was a cheese pizza - @johnmu.com's was to work cheese into the presentation

Wrong image, but you get the idea

We thought it was due to some robots.txt blocking urls, but we've removed those lines from robots.txt. The website moved to SFCC in June and it has been a problem since then. CDN is controlled by Salesforce clearing cache isn't something I can verify has been cleared.

...because of Google declaring their own canonical to our 404 page. The pages render fine now in the live test so I'm getting frustrated. I think we may have to change the URLs to get them indexed again

In browser it loaded fine for the user, in GSC Inspect URL live test, it was showing a 404 error in rendering and declared ~80 URLs to a different canonical due to the 404 error. We fixed the issue via robots.txt and resubmitted the URLs and got them indexed again, but now G removed them again...

The response HTML seemed to have no issue, which is what is leading me to think that Bing doesn't (or doesn't do well) render JS.