301 redirects stored in WordPress database

luisherranz · 2 February 2021 10:51

I will write the Final Implementation as soon as possible

juanma · 9 February 2021 10:26

Great! Let us know (@documentation-team) once you have written the Final Implementation so we can check it for documentation purposes

orballo · 27 February 2021 04:39

Hi guys, I’m having some trouble making this work, and I think it is because I’m still using a beta version of the wp-source package because of the infinite scroll. Is there a release of the beta packages that includes the redirects?

luisherranz · 1 March 2021 17:30

The infinite scroll PR is merged in dev. We didn’t do a release today because the unit tests started failing for an unrelated issue that is already fixed in this PR.

So if you can wait until tomorrow…

orballo · 3 March 2021 11:13

Sure thing. Please let me know when it’s released!

SantosGuillamot · 3 March 2021 13:34

We did the release yesterday, although we haven’t prepared the announcement yet. It’s ready

SantosGuillamot · 3 March 2021 13:43

We have seen that our hosting provider, in this case Pantheon, change the HEAD methods and use GET instead. This way, if you are using the Embedded mode and the setting redirections: "404", it causes and Infinite loop that breaks the site as explained in this diagram:

If your hosting isn’t changing the HEAD method, it just works fine. We aren’t sure why Pantheon is doing this, but we think we should solve it anyway. We have made this video explaining the issue and the possible solutions:

And this is the diagram we have used. Let us know your thoughts .

luisherranz · 3 March 2021 15:35

We should also add another safety measure: Do not return a 301 which redirects again to the same link.

That could happen if the domains are different and the WordPress domain is redirected to the Frontity domain.

The logic should be something like this: if the redirection is internal and the pathname is the same, don’t do the redirection.

const linkParams = new URL(link);
const locationParams = new URL(location);

const linkURI = linkParams.path + linkParams.search;
const locationURI = locationURI.path + locationURI.search;

// Only return the redirection if the location is different.
if (!isExternal && linkURI !== locationURI) {
  return redirection;
}

I haven’t tested this code, it’s just an example.

Excalidraw | Hand-drawn look & feel • Collaborative • Secure

orballo · 9 March 2021 08:17

@luisherranz I can confirm that the infinite loop happens when visiting a non existent url. Not sure what other cases I should test, but if you go to https://frontity.es.aleteia.org/2021/02/23/fake/ for us it enters in a loop.

orballo · 9 March 2021 08:24

By the way, congrats on this solution, it seems to work with all our use cases, very nice job!

luisherranz · 9 March 2021 10:27

Thanks Edu!

Ok. We’ll let you know once it is solved.

SantosGuillamot · 10 March 2021 08:09

I have created an issue for this → Infinite loop with 404 redirections in Embedded mode in some hostings · Issue #743 · frontity/frontity · GitHub

I labeled it as priority: high but I wasn’t sure if it should be considered critical. So if you have a different opinion just let me know.

mmczaplinski · 17 March 2021 03:24

Final Implementation

Pull Request

https://github.com/frontity/frontity/pull/601

Rationale

Many users store their 30x redirections in the WordPress database, e.g. via a Redirection plugin. WordPress redirects a user to a new link, for example after a user renames a post and tries to access it using the “old” link. However, no such functionality exists out-of-the-box for the REST API. Since Frontity uses the REST API to retrieve content, it needs a way of handling such redirections. For a bit more in-depth explanation, I recommend watching the Demo (also linked at the end).

General mechanism

The redirections functionality works by making an additional request to the WordPress instance to check if a redirection exists for a particular URL. By WordPress instance, we mean the location of your WordPress installation, not the Frontity app. Normally the state.source.url points to that location.

We introduce a single setting state.source.redirections which dictates under what circumstances should Frontity make that request to the check if a redirection exists. The state.source.redirections accepts the following values:

"no" - Does not handle redirections at all. This is the default.
"all" - Always make an additional request to the WordPress instance to check if there exists a redirection. This means that every time you navigate to a new link, Frontity will make 2 requests: one to the REST API to try to fetch the content and another one to the WordPress instance to check if a redirection exists. Frontity will wait for both requests to finish before proceeding.
"404" - Only send the additional request to the WordPress instance if the original request to the REST API has returned a 404. This would happen for example if try to access a post that has been renamed.
string - A string that contains a regex pattern. The string must start with RegExp:. This pattern will be matched against the current route and if matched, Frontity will make an additional request to the WordPress instance to check if there exists a redirection for that route. Note that the shorthand character classes will have to be escaped, so for example instead of \d, you will need to write \\d.
string[] - An array of strings, which can contain the “404” value as well as any number of strings starting with "RegExp:" which represent regular expressions. An additional request will be sent to Wordpress to check for the redirection if any of the regular expressions match the current route. If the array also contains a "404", an additional request will also be made if the original request has returned a 404.

Usage

In order for the redirections to work correctly you will need to set up CORS headers in your WordPress installation. If you are using the Redirections plugin, it’s quite simple:

In order to use the redirections, there is no need to install any new npm package. There is a new property exposed by the wp-source package, state.source.redirections which is used to handle the redirections. The recommended way of using it is by setting it in your frontity.settings.js file.

state.source.redirections accepts the options outlined above and described by those types

Examples of valid values:

// frontity.settings.js

{
  name: "@frontity/wp-source",
  state: {
    source: {
      url: "https://test.frontity.org",

      // always check if there exists a redirection.
      redirections: "all",

      // match the url `/some-post` exactly
      redirections: "/some-post/",

      // match urls like `/some-post/1`, `/some-post/2`, etc.
      redirections: "RegExp:/some-post/(\\d*)",

      // match urls like: /some-post/42, /some-otherpost/5
      redirections: "RegExp:/post-(\\w*)/(\\d*)",

      // match a combination of multiple options
      redirections: ["404", "/some-post/", "RegExp:/another-post/(\\d)"],
    },
  },
}

Alternative usage

Redirections work internally by assigning a special RedirectionData object to state.source.data[link] (details below in Technical details).

A consequence of that is that you can alternatively define redirections directly in the state or using a custom handler as mentioned by Luis previously.

Single redirection populating state.source.data with RedirectionData directly:

const state = {
  source: {
    data: {
      "/old-url/": {
        isReady: true,
        isRedirection: true,
        is301: true,
        redirectionStatus: 301,
        isExternal: false,
        location: "/new-url",
      },
    },
  },
};

A custom handler which assigns properties of RedirectionData object for the current route:

const categoryRedirection = {
  pattern: "/category/:slug",
  priority: 5,
  func: ({ link, params }) => {
    state.source.data[link].isReady = true;
    state.source.data[link].isRedirection = true;
    state.source.data[link].is301 = true;
    state.source.data[link].redirectionStatus = 301;
    state.source.data[link].isExternal = false;
    state.source.data[link].location = `/categoria/${params.slug}`;
  },
};

Note on functionalities

The redirections support not only redirecting to other pages in your WordPress site but also redirecting to external pages.
You can define 301, 302, 307 or 308 Redirections.
We respect the settings of the Redirections plugin with respect to the query parameters:

Screen Shot 2021-03-15 at 20.12.01994×516 65.6 KB

Technical details

The redirections are first handled inside of actions.source.fetch(). If the setting for state.source.redirections contains an “eager” value (for example if it equals "all") it is fetched before calling the handler for the current route. “Fetching a redirection” refers to making a request to the WordPress instance to check if a redirection exists for a particular URL.

If a redirection is not “eager” (for example if state.source.redirections is "404") then we fetch the redirection only after the request to the REST API has returned a 404.

The actual logic for fetching the redirection from the WordPress instance is different on the client and server because of platform differences.

In both cases, if a redirection does exist for a a particular route, Frontity populates state.source.data[link] with a RedirectionData object instead of a “typical” Data object like PostData or AuthorData. This object contains all the information about a redirection that Frontity needs in order to handle it. You can check the type of RedirectionData to see all of its properties.

Once a RedirectionData object is in the state, the behavior of Frontity is different on client and server.

On the server, we check the data object for the current route in the beforeSSR() action of tiny-router. If that object contains a redirection, Frontity will set the correct HTTP status, and redirect using ctx.redirect(data.location) where data.location contains the final URL .

On the client, we listen to changes to the state.source.data object and if the data object for current route contains a redirection, Frontity calls actions.router.set(data.location) where data.location contains the final URL.

The above flow looks something like:

Demo

A demo has been created by Mario:

juanma · 17 March 2021 17:07

@mmczaplinski Thanks for the Final Implementation info. It’s really helpful and allowed me to understand pretty well how the redirections work. The Demo video is also very useful.

Just one question

Fow what I understand this feature works w/ the default redirections managed internally by wordpress (for example when we rename a slug) and for those defined w/ the Redirection plugin (that it seems to the popular choice for most WordPress users)

Will this feature work w/ some other redirections plugins?

Just to provide some feedback I think this is very detailed and technical explanation really useful to understand how the use the feature and how it works internally. I don’t think we’ll explain the technical details in the docs, but I love they’re explained here because they can be easily referenced from the docs

mmczaplinski · 17 March 2021 17:31

We have not tested it with other redirection plugins because the Redirection plugin is by far the most popular but I see no reason why it should work differently with another plugin. They should all work using the same principles of HTTP

awesome, thanks

luisherranz · 18 March 2021 15:56

I couldn’t agree more. Top work Michal!

mmczaplinski · 19 March 2021 00:03

Hey @luisherranz

Let me try to understand this use case a bit better. Let’s take a single post /some-post/ as an example.

We have some WP + Frontity app where:
- state.source.redirections === "404".
- There is no post with a slug /some-post/.
- There is a redirect for that post which points to itself (/some-post/ → /some-post/).
What we expect is the user to see a 404, but we end up with an ERR_TOO_MANY_REDIRECTS. This much is clear and it’s indeed a problem.
state.source.redirections === "/some-post/" or it’s an array that includes /some-post/. In this case, it doesn’t matter if the post exists or not. We will always check the redirection for this post before fetching from the REST API. In this case, there is a redirection and we are again in the infinite loop. But in this case this is expected behavior. I don’t think there is any other reasonable thing for Frontity to do if a user decides to create a redirection like that.
Same, if state.source.redirections === "all". We end up with an infinite loop, but that is the expected behavior.

Can you confirm that the issue is just 1) as I outlined or is there something I’m missing?

luisherranz · 19 March 2021 14:36

Good question. I think it should work the same way. If the location and the link are the same, we don’t return the 301. In that case, instead of the 404, I think actions.source.fetch() should continue the execution.

This would be the flow:

Let me know if that makes sense to you

mmczaplinski · 19 March 2021 23:52

Okay, I see.

So, for the “eager” redirections like "all", string and string[], if link === location we just ignore the fact that WordPress has returned a redirection and continue fetching the data.

I guess that makes sense, but it still seems a bit strange to me because it means that ultimately we don’t respect the redirections setting if the redirection is “eager” and the link redirects to itself. (excalidraw link) . I don’t think this is necessarily wrong, just a bit weird for lack of a better word.

luisherranz · 22 March 2021 16:38

But in this case, the redirection from WordPress is to the Frontity server itself, isn’t it?

Let’s try to go over an example:

WordPress URL is https://wp.domain.com
Frontity URL is https://domain.com
WordPress has 301 redirections from https://wp.domain.com to https://domain.com.
A user tries to access https://domain.com/some-post.
Frontity wants to know if there is a redirection, and asks WordPress about /some-post using https://wp.domain.com/some-post.
WordPress tells Frontity that there is a redirection to https://domain.com/some-post indeed.
Frontity returns a 301 redirection that points to itself, causing the too many redirections loop.

Can you think of a case where Frontity should return the redirection even if link === location?