Make the backend URL a global setting

đź—Ł Feature Discussions Released
1

Description

The WordPress backend URL is only on state.source.api now, and it may (or may not) contain a prefix, usually /wp-json.

We have started to see more and more examples of different packages that need to know the WordPress URL, not only to access the REST API, but to other uses. Extracting the WordPress URL from state.source.api is not trivial.

Examples

  • Replace the URL of a link or a meta
    For example, changing links from https://wp.domain.com/my-link to https://www.domain.com/my-link for the @frontity/head-tags package.
  • Link or use a resource or asset that is stored in WordPress
    For example, linking to the RSS feed https://wp.domain.com/feed, sitemap https://wp.domain.com/sitemap.xml or service worker https://wp.domain.com/sw.js.
  • Other source packages could have different API urls
    If we stick with this approach and a package like @frontity/wpgraphql-source is released, it will break packages depending on state.source.api to get the WordPress URL.

I know we’ve seen more examples of this but I don’t remember them right now :sweat_smile:

Possible solution

Create a global setting for the backend, similar to state.frontity.url. Then, get rid of it on @frontity/wp-source, but keep the options that are relevant to the REST API.

export default {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
      title: "Test Frontity Blog",
      description: "Useful content for Frontity development",
      backend: "https://test.frontity.org",
    },
  },
  packages: [
    {
      name: "@frontity/wp-source",
      state: {
        source: {
          prefix: "/wp-json", // This will be the default.
          prefix: false, // To use `?rest_route=`.
          isWpCom: true, // To support the WP.com API.
        },
      },
    },
  ],
};

This should be a backward-compatible change.

Drawbacks:

I guess there could be more than one backend for a single site.

@dev-team opinions?

2

Sounds good to me!

I cant think of any counterargument at the moment. Also, a side-effect would be that we can have zero-configuration for the wp-source package in that case :slight_smile:

I don’t like the name backend that much though because it’s quite generic… Maybe backendUrl or wordpressUrl …?

  • backend
  • backendUrl
  • wordpressUrl

0 voters

(edit the poll to add other suggestions if you like) :slightly_smiling_face:

3

I like the idea :slightly_smiling_face: Apart from being useful for the uses cases you listed, I think the settings would be clearer this way.

4

Does this change need to be backwards compatible?

5

Hey @nicholasio.oliveira

I’m afraid that yes. :slight_smile:

What use case do you have in mind that would require making it backwards-incompatible and in what way would it be incompatible?

6

I’m just thinking of packages and themes that potentially rely on state.source.api and therefore would break if we just remove it.

7

To make it backwards compatible state.source.api can be a derived state built with state.frontity.backend and state.source.prefix, so packages relying on state.source.api will still work:

const state = {
  source: {
    prefix: "/wp-json",
    api: ({ state }) => state.frontity.backend + state.source.prefix,
  },
};

If someone overwrites state.source.api in the frontity.settings.js file, it will still work because it will overwrite the derived state.

By the way, I wouldn’t name it wordpressUrl because now it’s going to be a global setting. Even though the framework (and the company) is focused on WordPress, there’s yet not a single line of the core hardcoded for WordPress and I would like it to remain that way. If someone else wants to start using Frontity for another CMS, they should be able to do so. All the WordPress integrations done so far and/or planned ahead are always done at a package level :slightly_smiling_face:

1 Like
8

Looking at the results of the poll and taking into account that we should avoid the WordPress name in the core, I think we can go with backendUrl. And yes, as @mmczaplinski mentioned, this would make wp-source a zero-config package :smile: :tada:

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
      backendUrl: "https://test.frontity.org",
    },
  },
  packages: ["@frontity/wp-source"],
};

Thoughts?

4 Likes
9

backendUrl sounds pretty ugly. I propose dataSourceUrl or sourceDataUrl instead.

10

Uhm, I like that. What about sourceUrl?

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
      sourceUrl: "https://test.frontity.org",
    },
  },
  packages: [...],
};

And we could also keep it in state.source but call it url instead of api:

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
    },
  },
  packages: [
    {
      name: "@my-source-package",
      state: {
        source: {
          url: "https://test.frontity.org",
        },
      },
    },
  ],
};

Packages that need access to it would use state.source.url instead of state.frontity.sourceUrl.

In the end, it’s the same, it’s just a matter of how to organize it to make more sense 🤷

The important thing here is that we remove the /wp-json prefix.

11

By the way, this would be equivalent, as there are no restrictions on what to include on those state blocks:

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
    },
    source: {
      url: "https://test.frontity.org",
    },
  },
  packages: ["@my-source-package"],
};
1 Like
12

By the way, I’ve just noticed that state.source.api is not part of the source namespace and currently belongs to wp-source only. I think that if we finally go with state.source.src, we have to add it to the source namespace, as any source package will have to expose that setting for the other packages to use, not only wp-source.

EDIT: I’ve moved state.source.api to the source namespace in this commit.

1 Like
13

I wonder if we could make this work with relative URLs as well.

Imagine I’m using the Embedded mode:

It’d make sense to make the Backend URL relative to the domain, something like this:

const settings = {
  packages: [
    {
      name: "@my-source-package",
      state: {
        source: {
          url: "/",
        },
      },
    },
  ],
};

So requests in mydomain.com go to mydomain.com/wp-json and requests in mydomain.local go to mydomain.local/wp-json without having to configure anything else.

14

My 2 cents here…

Agree :+1:

:+1: I think having the main domain (without /wp-json) of the source of data (API REST) stored in state.source.url is the way to go
But… I don’t really get the purpose of this Feature Discussion as it starts saying…

What would be the difference of extracting the value from state.source.api vs extracting the value from state.source.url?

Why don’t we go a step further and make this value the default one? So if there¡s no definition of state.source.url, Frontity will assume / as the default value

15

@juanma The idea would be that the user can only provide the state.source.url from now on and the state.source.api can be constructed from it like mentioned in: Make the backend URL a global setting

I’d like to put together an implementation proposal because I think we should implement this together with the 301 Redirections. Having this setting ready would simplify that feature and prevent us from having to extract the url from state.source.api there which as Luis has mentioned is “not trivial” :slight_smile:

Side note about “prefix”

I think what we’ve been calling a prefix in this thread, should really be a suffix or perhaps a different name altogether. A “prefix” is something that goes before the main name, so this might be confusing I think. I will keep using the suffix for the time being.

The Implementation Proposal

Types:

I think that we’ll need to require either url or api or both. With TS this should be:

/// packages/source/types.ts
interface SourceBase {
  suffix?: string | false;
  data: Record<string, Data>;
  /* all the other properties here */
}
interface SourceWithAPI extends SourceBase {
  api: string;
}
interface SourceWithURL extends SourceBase {
  url: string;
}
type FinalSource = SourceWithAPI | SourceWithURL;

interface Source<T = null> extends Package {
  state: {
    source: FinalSource; 
  }
}

Extending wp-source:

const state: WpSource["state"]["source"] = {
  // ...
  api: ({ state }) => {
    if (state.source.url) {
      const suffix = state.source.suffix ?? "?rest_route=";
      return state.source.url + suffix;
    }
    return "";
  },  
  suffix: "/wp-json", // This is the default
  //...
}

Example usage:

const settings = {
  packages: [
    {
      name: "@my-source-package",
      state: {
        // The most simple example, uses the /wp-json default
        source: {
          url: "https://my-wordpress-site.com",
        },

        // custom suffix
        source: {
          suffix: "/custom-suffix"
          url: "https://my-wordpress-site.com",
        },

        // use "?rest_route=" as the suffix
        source: {
          suffix: false
          url: "https://my-wordpress-site.com",
        },

        // custom REST api location
        source: {
          api: "https://my-wordpress.site.com/api/"
          url: "https://my-wordpress-site.com",
        },

        // Everything should still work even if 
        // you don't specify the state.source.url
        source: {
          api: "https://my-wordpress.site.com/api/"
        },
      },
    },
  ],
};

Outstanding questions:

  1. Some features will depend on this new setting (like 301 Redirections). Can we (or should we) fallback to extracting the url from the state.source.api or should any future features throw an error if state.source.url is not available?
  2. About making this work with relative URLs for the embedded mode… I haven’t included that in the proposal, but I think this could be added now or in a future update.
16

I saw a problem yesterday with this proposal regarding WordPress.com sites (I already mentioned this to Michal).

If we simply replace state.source.api by state.source.url we would lose context as, right now, state.source.api is the one used to compute state.source.isWpCom.

That is because the REST API URL for WordPress.com sites is slightly different.

For example, with state.source.api:

{
  state: {
    source: {
      api: "https://public-api.wordpress.com/wp/v2/sites/my-wp-site.com",
      isWpCom: true // <-- This value is derived from `state.source.api`.
    }
  }
}

With the url proposal:

{
  state: {
    source: {
      url: "https://my-wp-site.com",
      isWpCom: true // <-- We can't derive this value from `url`.
    }
  }
}

This could be solved making isWpCom to be assignable in frontity.settings.js.

17

I don’t disagree with you, as this can be considered a suffix of the URL. But when we “inherit” concepts from WordPress, we always have to try to stay as close as possible to the names already defined in WordPress itself, to avoid confusion.

There is already a filter in WordPress which calls this prefix: https://developer.wordpress.org/reference/functions/rest_get_url_prefix/. It also makes sense, because it’s also the prefix of the URI: [wp-json]/wp/v2/.... So I would stick to prefix because of that.

I really like that idea. It’ll solve the problem very elegantly.

I guess new features don’t need to be backward compatible with state.source.api, because when we instruct people to set state.router.redirections when can ask people to also set state.source.url, right? But we have to remember to put that requirement in the docs.

I really like @juanma idea. If state.source.url is not defined, it fallbacks to state.frontity.url.

My feedback about the proposal:

1. Move state.source.prefix to state.wpSource.prefix

This is in preparation for Source v2, where we want packages from different APIs, like the REST API and GraphQL, to coexist.

const mergedState = {
  source: {
    url: "https://mybackend.com",
  },
  wpSource: {
    prefix: "/wp-json",
  },
  wpGraphqlSource: {
    prefix: "/graphql",
  },
};

I know that state.source.api is still coupled to wpSource, but that’s something we cannot change now to keep backward compatibility.

2. If state.source.url is missing, use state.frontity.url

As Juanma suggested, we can ask people to delete state.source.url if they are using the Embedded mode, and it will fall back to state.frontity.url.

3. Support for ?rest_route=, but using the prefix field

That’s a good point, thanks for adding it here. My only opinion is that I don’t see a reason why people should not set it in the prefix, instead of setting it to false:

const settings = {
  state: {
    wpSource: {
      prefix: "?rest_route=",
    },
  },
};

We have to make sure that it works in libraries.source.api.get though.

4. Keep backward compatibility with WordPress.com

We have to make sure that we keep backward compatibility with WordPress.com. Right now, state.source.isWpCom is populated automatically.

Juanma made a series of tests a while ago and discovered that the long public-api URL (https://public-api.wordpress.com/wp/v2/sites/site.wordpress.com) is only used for free sites that use the site.wordpress.com URL.

Thanks to that, I think we can still detect it and keep backward compatibility.

export default {
  state: {
    wpSource: {
      prefix: "/wp-json",
    },
    source: {
      // The default when `state.source.url` is not overwritten in
      // frontity.settings.js is to fallback to `state.frontity.url`.
      url: ({ state }) => state.frontity.url,

      // Keep backward compatibility when `state.source.api` is not
      // overwritten in frontity.settings.js.
      api: ({ state }) => {
        // Check if it's a free WordPress.com site.
        if (/^https:\/\/[\w-]\.wordpress\.com/.test(state.source.url))
          return `https://public-api.wordpress.com/wp/v2/sites/${state.source.url}`;

        return normalize(state.source.url + state.wpSource.prefix);
      },

      // Keep backward compatibility.
      isWpCom: ({ state }) =>
        state.source.api.startsWith("https://public-api.wordpress.com"),
    },
  },
};
1 Like
18

Awesome, thanks @luisherranz! Not much to add, I agree with pretty much everything:

oh, I see, I was not aware of that. This makes sense.

yep, that’s what I had in mind as well.

You’re right, that would be more intuitive for Frontity users. I was following the WordPress here where ?rest_route= is the default (hence the false)

:+1:

1 Like
19

The normalization of URLs is quite a mess right now. We are going to need to make something much better for Source v2.

Also, I think that the ?rest_route= prefix doesn’t work now. But nobody seems to have requested it yet, so I guess we can fix it later of even wait for the Source v2.

Finally, we have to rethink all the types of the source namespace for Source v2:

  • What goes into state.source in @frontity/source: this is meant to be included by any source package.
  • What goes into state.source in @frontity/wp-source: this is meant to be included only by wp-source. I’m not sure if this makes sense anymore.
  • What goes into state.wpSource in @frontity/wp-source: this is meant to be included only by wp-source.

I have approved the PR now Michal. Feel free to merge and update the FD with the Final Implementation :slightly_smiling_face:

1 Like
20

Final implementation

The final implementation consists of the following:

  1. Introduce a new property state.source.url in wp-source that points to the URL of the users’ WordPress backend. The default value of that property is derived from state.frontity.url.

  2. The default value of state.source.api has changed. We now derive that value from the state.source.url. This should have no impact on existing applications, but has an important effect on new apps: You don’t have to specify the state.source.api!

    You now can just specify the state.source.url and the state.source.api is computed from it by adding the “prefix” (see point 3. :slightly_smiling_face: ).

  3. Introduce a new property state.wpSource.prefix where the user can specify prefix of the REST API, for example "/wp-json" or "?rest_route=/". The default value is "/wp-json".

  4. Introduce the state.wpSource.api and state.wpSource.isWpCom. For the moment their values are just derived from state.source.api and state.source.isWpCom respectively. This is added in preparation for the Source V2.

For completeness, here is the actual, final implementation:

// packages/wp-source/src/index.ts

const wpSource = (): WpSource => ({
  // ... 
  state: {
    wpSource: {
      // Just copy the value of `state.source.api` until we deprecate it and
      // move its logic to `state.wpSource.api` in the v2.
      api: ({ state }) => state.source.api,
      // Just copy the value of `state.source.isWpCom` until we deprecate it and
      // move its logic to `state.wpSource.isWpCom` in the v2.
      isWpCom: ({ state }) => state.source.isWpCom,
      prefix: "/wp-json",

      // ...
    },
  },
}

// packages/wp-source/src/state.ts

const state: WpSource["state"]["source"] = {
  // ...
  // Keep backward compatibility when `state.source.api` is not
  // overwritten in frontity.settings.js.
  api: ({ state }) => {
    // Check if it's a free WordPress.com site.
    if (/^https:\/\/(\w+\.)?wordpress\.com/.test(state.source.url))
      return addFinalSlash(
        `https://public-api.wordpress.com/wp/v2/sites/${state.source.url}`
      );

    return addFinalSlash(
      addFinalSlash(state.source.url) + state.wpSource.prefix.replace(/^\//, "")
    );
  },
  url: ({ state }) => {
    if (!state.frontity?.url)
      error(
        "Please set either `state.source.url` (or at least `state.frontity.url` if you are using Embedded mode) in your frontity.settings.js file."
      );
    return addFinalSlash(state.frontity.url);
  },

  //... 
};
3 Likes