Make the backend URL a global setting

[luisherranz]

@mmczaplinski

I think what we’ve been calling a prefix in this thread, should really be a suffix or perhaps a different name altogether.

I don’t disagree with you, as this can be considered a suffix of the URL. But when we “inherit” concepts from WordPress, we always have to try to stay as close as possible to the names already defined in WordPress itself, to avoid confusion.

There is already a filter in WordPress which calls this prefix: https://developer.wordpress.org/reference/functions/rest_get_url_prefix/. It also makes sense, because it’s also the prefix of the URI: [wp-json]/wp/v2/.... So I would stick to prefix because of that.

@juanma

Why don’t we go a step further and make this value the default one? So if there¡s no definition of state.source.url , Frontity will assume / as the default value

I really like that idea. It’ll solve the problem very elegantly.

@mmczaplinski

Some features will depend on this new setting (like 301 Redirections). Can we (or should we) fallback to extracting the url from the state.source.api or should any future features throw an error if state.source.url is not available?

I guess new features don’t need to be backward compatible with state.source.api, because when we instruct people to set state.router.redirections when can ask people to also set state.source.url, right? But we have to remember to put that requirement in the docs.

@mmczaplinski

About making this work with relative URLs for the embedded mode… I haven’t included that in the proposal, but I think this could be added now or in a future update.

I really like @juanma idea. If state.source.url is not defined, it fallbacks to state.frontity.url.

---

My feedback about the proposal:

1. Move state.source.prefix to state.wpSource.prefix

This is in preparation for Source v2, where we want packages from different APIs, like the REST API and GraphQL, to coexist.

const mergedState = {
  source: {
    url: "https://mybackend.com",
  },
  wpSource: {
    prefix: "/wp-json",
  },
  wpGraphqlSource: {
    prefix: "/graphql",
  },
};

I know that state.source.api is still coupled to wpSource, but that’s something we cannot change now to keep backward compatibility.

2. If state.source.url is missing, use state.frontity.url

As Juanma suggested, we can ask people to delete state.source.url if they are using the Embedded mode, and it will fall back to state.frontity.url.

3. Support for ?rest_route=, but using the prefix field

That’s a good point, thanks for adding it here. My only opinion is that I don’t see a reason why people should not set it in the prefix, instead of setting it to false:

const settings = {
  state: {
    wpSource: {
      prefix: "?rest_route=",
    },
  },
};

We have to make sure that it works in libraries.source.api.get though.

4. Keep backward compatibility with WordPress.com

We have to make sure that we keep backward compatibility with WordPress.com. Right now, state.source.isWpCom is populated automatically.

Juanma made a series of tests a while ago and discovered that the long public-api URL (https://public-api.wordpress.com/wp/v2/sites/site.wordpress.com) is only used for free sites that use the site.wordpress.com URL.

Thanks to that, I think we can still detect it and keep backward compatibility.

export default {
  state: {
    wpSource: {
      prefix: "/wp-json",
    },
    source: {
      // The default when `state.source.url` is not overwritten in
      // frontity.settings.js is to fallback to `state.frontity.url`.
      url: ({ state }) => state.frontity.url,

      // Keep backward compatibility when `state.source.api` is not
      // overwritten in frontity.settings.js.
      api: ({ state }) => {
        // Check if it's a free WordPress.com site.
        if (/^https:\/\/[\w-]\.wordpress\.com/.test(state.source.url))
          return `https://public-api.wordpress.com/wp/v2/sites/${state.source.url}`;

        return normalize(state.source.url + state.wpSource.prefix);
      },

      // Keep backward compatibility.
      isWpCom: ({ state }) =>
        state.source.api.startsWith("https://public-api.wordpress.com"),
    },
  },
};

[mmczaplinski]

Awesome, thanks @luisherranz! Not much to add, I agree with pretty much everything:

@luisherranz

There is already a filter in WordPress which calls this prefix: https://developer.wordpress.org/reference/functions/rest_get_url_prefix/. It also makes sense, because it’s also the prefix of the URI: [wp-json]/wp/v2/.... So I would stick to prefix because of that.

oh, I see, I was not aware of that. This makes sense.

@luisherranz

I guess new features don’t need to be backward compatible with state.source.api, because when we instruct people to set state.router.redirections when can ask people to also set state.source.url, right?

yep, that’s what I had in mind as well.

@luisherranz

That’s a good point, thanks for adding it here. My only opinion is that I don’t see a reason why people should not set it in the prefix, instead of setting it to false:

You’re right, that would be more intuitive for Frontity users. I was following the WordPress here where ?rest_route= is the default (hence the false)

@luisherranz

I really like @juanma idea. If state.source.url is not defined, it fallbacks to state.frontity.url.

[luisherranz]

The normalization of URLs is quite a mess right now. We are going to need to make something much better for Source v2.

Also, I think that the ?rest_route= prefix doesn’t work now. But nobody seems to have requested it yet, so I guess we can fix it later of even wait for the Source v2.

Finally, we have to rethink all the types of the source namespace for Source v2:

• What goes into state.source in @frontity/source: this is meant to be included by any source package.
• What goes into state.source in @frontity/wp-source: this is meant to be included only by wp-source. I’m not sure if this makes sense anymore.
• What goes into state.wpSource in @frontity/wp-source: this is meant to be included only by wp-source.

---

I have approved the PR now Michal. Feel free to merge and update the FD with the Final Implementation

[mmczaplinski]

Final implementation

The final implementation consists of the following:

1. Introduce a new property state.source.url in wp-source that points to the URL of the users’ WordPress backend. The default value of that property is derived from state.frontity.url.

2. The default value of state.source.api has changed. We now derive that value from the state.source.url. This should have no impact on existing applications, but has an important effect on new apps: You don’t have to specify the state.source.api!

   You now can just specify the state.source.url and the state.source.api is computed from it by adding the “prefix” (see point 3. ).

3. Introduce a new property state.wpSource.prefix where the user can specify prefix of the REST API, for example "/wp-json" or "?rest_route=/". The default value is "/wp-json".

4. Introduce the state.wpSource.api and state.wpSource.isWpCom. For the moment their values are just derived from state.source.api and state.source.isWpCom respectively. This is added in preparation for the Source V2.

For completeness, here is the actual, final implementation:

// packages/wp-source/src/index.ts

const wpSource = (): WpSource => ({
  // ... 
  state: {
    wpSource: {
      // Just copy the value of `state.source.api` until we deprecate it and
      // move its logic to `state.wpSource.api` in the v2.
      api: ({ state }) => state.source.api,
      // Just copy the value of `state.source.isWpCom` until we deprecate it and
      // move its logic to `state.wpSource.isWpCom` in the v2.
      isWpCom: ({ state }) => state.source.isWpCom,
      prefix: "/wp-json",

      // ...
    },
  },
}

// packages/wp-source/src/state.ts

const state: WpSource["state"]["source"] = {
  // ...
  // Keep backward compatibility when `state.source.api` is not
  // overwritten in frontity.settings.js.
  api: ({ state }) => {
    // Check if it's a free WordPress.com site.
    if (/^https:\/\/(\w+\.)?wordpress\.com/.test(state.source.url))
      return addFinalSlash(
        `https://public-api.wordpress.com/wp/v2/sites/${state.source.url}`
      );

    return addFinalSlash(
      addFinalSlash(state.source.url) + state.wpSource.prefix.replace(/^\//, "")
    );
  },
  url: ({ state }) => {
    if (!state.frontity?.url)
      error(
        "Please set either `state.source.url` (or at least `state.frontity.url` if you are using Embedded mode) in your frontity.settings.js file."
      );
    return addFinalSlash(state.frontity.url);
  },

  //... 
};

[luisherranz]

Awesome work Michal! Thanks

[SantosGuillamot]

This feature has been included in the latest release : Read the full Release Announcement .

[SantosGuillamot]

There are different options here to define the new source.url and I am not sure what is best:

1. We can define it in the general settings, like frontity.url.
2. We can define it in the source package, like the old source.api.

Both of them will work, but we should reach an agreement to see how we document and explain it.

If we take mars-theme as example, the initial frontity.settings.js would look like this:

1. Defining source.url in the general settings

const settings = {
  name: "new-project",
  state: {
    frontity: {
      url: "https://test.frontity.org",
      title: "Test Frontity Blog",
      description: "WordPress installation for Frontity development",
    },
   source: {
     url: "https://test.frontity.org",
   },
  },
  packages: [
    {
      name: "@frontity/mars-theme",
      state: {
        theme: {
          menu: [
            ["Home", "/"],
            ["Nature", "/category/nature/"],
            ["Travel", "/category/travel/"],
            ["Japan", "/tag/japan/"],
            ["About Us", "/about-us/"],
          ],
          featured: {
            showOnList: false,
            showOnPost: false,
          },
        },
      },
    },
    "@frontity/wp-source",
    "@frontity/tiny-router",
    "@frontity/html2react",
  ],
};

export default settings;

The wp-source package would come up with no configuration, and if users want to add some settings, like postTypes for example, they would need to change it from a string to an object, like the theme.

If we release new source packages like wpgraphql in the future, users don’t have to change this setting.

2. Defining source.url in each source package

const settings = {
  name: "redirects",
  state: {
    frontity: {
      url: "https://test.frontity.org",
      title: "Test Frontity Blog",
      description: "WordPress installation for Frontity development",
    },
  },
  packages: [
    {
      name: "@frontity/mars-theme",
      state: {
        theme: {
          menu: [
            ["Home", "/"],
            ["Nature", "/category/nature/"],
            ["Travel", "/category/travel/"],
            ["Japan", "/tag/japan/"],
            ["About Us", "/about-us/"],
          ],
          featured: {
            showOnList: false,
            showOnPost: false,
          },
        },
      },
    },
    {
      name: "@frontity/wp-source",
      state: {
        source: {
          url: "https://test.frontity.org",
        },
      },
    },
    "@frontity/tiny-router",
    "@frontity/html2react",
  ],
};

export default settings;

For new source packages, users would have to add this setting to this package, even though it is the same for all the source packages.

I personally don’t have a strong opinion here about which option is better.

@juanma @mburridge Which option do you think fits better in the learning journey you are working on? How would be easier to explain Frontity?

[luisherranz]

I vote to keep doing it the same way we have done it up until now because that part of the state belongs to the source package.

{
  name: "@frontity/wp-source",
  state: {
    source: {
      url: "https://test.frontity.org",
    },
  },
}

I think it would be confusing to start defining some settings for some packages outside of the package definitions.

[juanma]

@luisherranz

I vote to keep doing it the same way we have done it up until now because that part of the state belongs to the source package.

I agree, I think it’s easier to understand that changes in specific namespaces of the state are done in the settings of the package assigned to that namespace

But, if this release includes the use of state.frontity.url when state-source.url is not defined I would start promoting this

const settings = {
  name: "new-project",
  state: {
    frontity: {
      url: "https://test.frontity.org",
      title: "Test Frontity Blog",
      description: "WordPress installation for Frontity development",
    }  
  },
  packages: [
    {
      name: "@frontity/mars-theme",
      state: {
        theme: {
          menu: [
            ["Home", "/"],
            ["Nature", "/category/nature/"],
            ["Japan", "/tag/japan/"],
            ["About Us", "/about-us/"],
          ]
        },
      },
    },
    "@frontity/wp-source",
    "@frontity/tiny-router",
    "@frontity/html2react",
  ],
};

export default settings;

that (in my opinion):

• it encourages the idea of zero-config
• featured settings can be let as the default ones defined for the theme
• less code, easier to understand
• we only write the URL of WordPress once

[mburridge]

I agree that having the WP URL in two places can be confusing. I think that state.frontity.url should be the one that we document, and just mention that state.source.api exists for backward compatibility, or that you can use state.source.url if you’re adding more properties to the source package.

Otherwise, I think that @juanma’s example above looks much cleaner and less cluttered, and so is less confusing. I think that’s the version of frontity.setting.js that should be in a new install of Frontity, i.e. with just the single line "@frontity/wp-source", and the url under state.frontity.url.

My understanding is that state.source.api and state.source.url have precedence over state.frontity.url so if we keep one of those we need to document that it should be removed so that state.frontity.url is the one that will be used, or that that is the one that needs to be configured with the WP API url - I think this would be messy and confusing.

[mburridge]

I think I might have found a problem with wordpress.com sites where the owner is using a custom URL. I’ve explained it in this loom video

[SantosGuillamot]

@juanma

But, if this release includes the use of state.frontity.url when state-source.url is not defined I would start promoting this

I am not sure we should do that. Right now, at least until the embedded mode is ready and promoted, users will always have to define both frontity.url and source.url, because that is needed for the Decoupled mode.

• frontity.url: This is the frontend url, the one pointing to node.js. Ex: mysite.com
• source.url: This is the backend url, the one pointing to WordPress. Ex: wp.mysite.com

@mburridge

I’ve explained it in this loom video

I think this could be the issue you are facing @mburridge. You are defining the WordPress url in the frontity.url setting. It should go in the source.url. In your case:

• frontity.url: http://localhost:3000.
• source.url: http://mysite.wordpress.com

[mburridge]

Hi @SantosGuillamot

I thought that the frontity.url means that you don’t need the source.url - though you can still use it, or even source.api if you want. Is my understanding incorrect?

I made the change you specified and the problem still exists with internal links when a wordpress.com site is using a custom domain.

[luisherranz]

@mburridge, it works like Mario described it:

state.frontity.url: The URL of Frontity.
state.source.url: The URL of the backend (WordPress).

• Decoupled mode (two different domains):
  • Frontity: state.frontity.url -> mars.frontity.org
  • Backend: state.source.url -> test.frontity.org
• Embedded mode (same domain for both):
  • Frontity: state.frontity.url -> frontity.org
  • Backend: state.source.url -> frontity.org

Only in Embedded mode, where both Frontity and the backend are in the same domain, you can omit state.source.url and if you do, it will take the value of state.frontity.url as a fallback.

• Embedded mode (same domains for both):
  • Frontity: state.frontity.url -> frontity.org
  • Backend: missing state.source.url -> state.frontity.url -> frontity.org

[mburridge]

@luisherranz thanks for the clarification. That makes sense. The problem with the link processor on wordpress.com sites using a custom domain is still an issue though.

[SantosGuillamot]

Sorry I said the state.frontity.url had to be localhost:3000. I’ve rewatched the video and it seems the frontend you are expecting is https://artistintheshed.com/, is that right? If that’s the case state.frontity.url should be that one.

[mburridge]

@SantosGuillamot

OK, I’m confused.

• Frontity is running locally on localhost:3000.
• The wordpress.com site is at https://artistintheshed.wordpress.com.
• The custom URL is https://artistintheshed.com.

What should my settings in frontity.settings.js be for:

• state.frontity.url
• state.source.url

?

[SantosGuillamot]

I think I misunderstood you as well

It seems that https://artistintheshed.wordpress.com is redirected to https://artistintheshed.com, and if we take a look at the API, https://public-api.wordpress.com/wp/v2/sites/artistintheshed.com/, seems to work fine.

With this in mind I assume that:

• state.source.url: https://artistintheshed.com.
• state.frontity.url: http://localhost:3000 or wherever your Frontity app is.

I guess you will have to define somewhere that state.source.url is a WP.com site, but I am not sure what is the best way to do this, I guess you would need to set state.source.isWpCom: true.

Could you help us here @mmczaplinski please?

[mburridge]

If state.source.url is set to https://artistintheshed.com then I get a 404 - I guess because that’s not a valid link to the API. Adding state.source.isWpCom: true doesn’t make a difference.

If state.source.url is set to https://artistintheshed.wordpress.com then it works as that points to the API at wordpress.com.

But then the link component problem that I describe in the loom video occurs - I guess because the internal links pointing to artistintheshed.com/whatever don’t match the hostname part of the URL which is artistintheshed.wordpress.com as far as Frontity is concerned.

By the way, this site is just an example because I know they have a custom URL. This problem will occur for any wordpress.com site with a custom URL.

[mmczaplinski]

@SantosGuillamot Let me know if this is what you expected me to explain

---

I think that there is some confusion because we have this bug that @mburridge has reported: https://github.com/frontity/frontity/issues/622.

As a matter of fact, the solution that Michael posted in the github issue is probably correct (or very close to what we it should be) so thanks Michael!

So, assuming that the bug I’ve mentioned is fixed, we should be able to do either of two things:

Version 1.

• state.source.url: https://artistintheshed.wordpress.com

• state.frontity.url: http://localhost:3000 or wherever your Frontity app is.

  The url points directly to a site that ends in wordpress.com so frontity can detect that and set up state.source.api correctly.

Version 2.

• state.source.url: https://artistintheshed.com

• isWpCom: true

• state.frontity.url: http://localhost:3000 or wherever your Frontity app is.

  In this case, frontity does not know that https://artistintheshed.com is a wp.com site, so we have to tell Frontity that

Hope this helps!