XML Sitemaps landing in WordPress Core

The Google team is working on a feature plugin to add XML sitemaps to the WordPress core:

https://github.com/GoogleChromeLabs/wp-sitemaps

I’ve tested the MVP and it works great.

The URLs can be filtered with this code. That means we can use it for Frontity :slight_smile:

<?php 
function change_urls( $url_list ) {
	for ( $i = 0; $i < count( $url_list ); $i++ ) {
		foreach ( $url_list[ $i ] as $attr => $value ) {
			if ( 'loc' === $attr ) {
				$parsed_url              = wp_parse_url( $value );
				$parsed_url['scheme']    = 'https';
				$parsed_url['host']      = 'frontity-domain.com';
				$url_list[ $i ][ $attr ] = unparse_url( $parsed_url );
			} 
		}
	}
	return $url_list;
}
 
add_filter( 'core_sitemaps_posts_url_list', 'change_urls' );
add_filter( 'core_sitemaps_taxonomies_url_list', 'change_urls' );
add_filter( 'core_sitemaps_users_url_list', 'change_urls' );
8 Likes

Hey @luisherranz thanks for the research, this plugin is going to be really helpful :blush:

Could you elaborate a bit more why this plugin is compatible with our “direct to Frontity” installation? It’s not totally clear for me.

Sure. With the code I added here we can change the URL of the sitemap links from the WordPress URL to the Frontity URL. For example, a link that is https://wp.domain.com/post-1 will become https://www.domain.com/post-1.

Then, the only thing we would need to do is to add the sitemap URL (https://wp.domain.com/sitemap.xml) to the robots.txt file of the Frontity site:

Sitemap: https://wp.domain.com/sitemap.xml
User-agent:*
Disallow:

It’s not the only way to add sitemap support though, there are more ways. For example, we could add a Frontity package that requests the sitemap from the WordPress site and filters the URLs before returning it.

This is a very simplified version, but something like this:

packages: [
  {
    name: "@frontity/sitemap",
    state: {
      sitemap: {
        origin: "https://wp.domain.com"
      }
    }
  }
]
export const server = ({ app }) => {
  app.use(
    get("/sitemap-*.xml", async ctx => {
      const origin = ctx.settings.state.sitemap.orign;
      const frontityUrl = ctx.settings.state.frontity.url;
      // Get the original sitemap from the WordPress site.
      const response = await fetch(`${origin}/${ctx.path}`);
      const body = await response.text();
      // Replace the URLs of WordPress for URLs of Frontity.
      ctx.body = body.replaceAll(origin, frontityUrl);
      // Do not cache this.
      ctx.set("cache-control: no-cache");
    })
  );
};
1 Like

Now it’s totally clear for me, thanks a lot @luisherranz

My site does not have a sitemap yet. I use Yoast seo, but could not find a solution to make it work with Frontity. I tried to use this, but without succes.
I tried your solution, but I get **Fatal error** : Uncaught Error: Call to undefined function unparse_url.
Can you share that function? Or do you have a good solution with Yoast Seo?

I was able to make the code work with replacing unparse_url( $parsed_url ) with str_replace('//wp.mysite.com/', '//mysite.com/' $value).

And how can I add it to robots.txt?

I created an npm package from your code: https://www.npmjs.com/package/basic-sitemap-for-frontity
But it does not work… :stuck_out_tongue: I get always 400 error.

1 Like

Wow @koli14 that package is awesome, but we haven’t finished the server extensibility yet :sweat_smile:


There’s another, easier way, to solve this: Use a robots.txt file in the root.

It should be fairly easy to do because it’s simply replicating what we have for the favicon.ico file, but with a robots.txt file. If it’s not present, it should return the default robots.txt file we have defined now.

If you are willing to contribute with this feature, you can start here: https://docs.frontity.org/contributing/code-contribution-guide

And if you want to go ahead with a PR and need help, let us know! :slightly_smiling_face:

2 Likes

is it possible to do something to write the original domain in robots.txt, not a subdomain?
For example, for nodejs to take the content at wp.domain.com/sitemap.xml, and give it to the address domain.com/sitemap.xml

Where can i add this? in functions.php? any can i help me with this and the ads?

Where can i add this? in functions.php? any can i help me with this and the ads?

Yes, you need to add it in functions.php. And then add a robots.txt pointing to the generated Sitemap: Use a robots.txt file in the root

1 Like

Hi @prainua

Apologies for the long delay in getting back to you on this.

Support for robots.txt was added in the last release. Please see: Use a robots.txt file in the root