Add sitemap index and serve sitemap for subprojects

[cedk]

The sitemap index lists all the sitemap locations for a project including the subprojects.
The sitemap of subprojects are also served from the parent domain.

Closes #6841

---

📚 Documentation previews 📚

• User's documentation (docs): https://docs--10629.org.readthedocs.build/en/10629/

• Developer's documentation (dev): https://dev--10629.org.readthedocs.build/en/10629/

[ericholscher]

Is there any more info on what the sitemap_index is for? I'm not super familiar with this code, but it seems like a nice addition!

[cedk]

The sitemap index is a file to list sitemaps of a website: https://en.wikipedia.org/wiki/Sitemaps#File_format
So the idea is that when you use subproject, you want to have a single sitemap file that list all the sitemaps so search engine can discover all the pages.

[cedk]

I'm wondering if the robots.txt should refer to the sitemap index instead of the project sitemap. Such that all projects with subproject will gain sitemap discoverability of subproject automatically without having to declare the sitemap index to the search engine.

[cedk]

How to relaunch the tests?

[humitos]

I'm not sure what happened here. Maybe it's a CircleCI configuration that doesn't allow to trigger test on contributor's PRs? 🤷🏼