Make the backend URL a global setting

luisherranz · 28 August 2020 10:34

Uhm, I like that. What about sourceUrl?

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
      sourceUrl: "https://test.frontity.org",
    },
  },
  packages: [...],
};

And we could also keep it in state.source but call it url instead of api:

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
    },
  },
  packages: [
    {
      name: "@my-source-package",
      state: {
        source: {
          url: "https://test.frontity.org",
        },
      },
    },
  ],
};

Packages that need access to it would use state.source.url instead of state.frontity.sourceUrl.

In the end, it’s the same, it’s just a matter of how to organize it to make more sense 🤷

The important thing here is that we remove the /wp-json prefix.

luisherranz · 28 August 2020 10:36

By the way, this would be equivalent, as there are no restrictions on what to include on those state blocks:

const settings = {
  state: {
    frontity: {
      url: "https://mars.frontity.org",
    },
    source: {
      url: "https://test.frontity.org",
    },
  },
  packages: ["@my-source-package"],
};

luisherranz · 15 September 2020 20:31

By the way, I’ve just noticed that state.source.api is not part of the source namespace and currently belongs to wp-source only. I think that if we finally go with state.source.src, we have to add it to the source namespace, as any source package will have to expose that setting for the other packages to use, not only wp-source.

EDIT: I’ve moved state.source.api to the source namespace in this commit.

luisherranz · 15 October 2020 16:34

I wonder if we could make this work with relative URLs as well.

Imagine I’m using the Embedded mode:

Frontity URL: mydomain.com
Backend URL: mydomain.com

It’d make sense to make the Backend URL relative to the domain, something like this:

const settings = {
  packages: [
    {
      name: "@my-source-package",
      state: {
        source: {
          url: "/",
        },
      },
    },
  ],
};

So requests in mydomain.com go to mydomain.com/wp-json and requests in mydomain.local go to mydomain.local/wp-json without having to configure anything else.

juanma · 16 October 2020 07:28

My 2 cents here…

Agree

I think having the main domain (without /wp-json) of the source of data (API REST) stored in state.source.url is the way to go
But… I don’t really get the purpose of this Feature Discussion as it starts saying…

What would be the difference of extracting the value from state.source.api vs extracting the value from state.source.url?

Why don’t we go a step further and make this value the default one? So if there¡s no definition of state.source.url, Frontity will assume / as the default value

mmczaplinski · 21 October 2020 00:48

@juanma The idea would be that the user can only provide the state.source.url from now on and the state.source.api can be constructed from it like mentioned in: Make the backend URL a global setting

I’d like to put together an implementation proposal because I think we should implement this together with the 301 Redirections. Having this setting ready would simplify that feature and prevent us from having to extract the url from state.source.api there which as Luis has mentioned is “not trivial”

Side note about “`prefix`”

I think what we’ve been calling a prefix in this thread, should really be a suffix or perhaps a different name altogether. A “prefix” is something that goes before the main name, so this might be confusing I think. I will keep using the suffix for the time being.

The Implementation Proposal

Types:

I think that we’ll need to require either url or api or both. With TS this should be:

/// packages/source/types.ts
interface SourceBase {
  suffix?: string | false;
  data: Record<string, Data>;
  /* all the other properties here */
}
interface SourceWithAPI extends SourceBase {
  api: string;
}
interface SourceWithURL extends SourceBase {
  url: string;
}
type FinalSource = SourceWithAPI | SourceWithURL;

interface Source<T = null> extends Package {
  state: {
    source: FinalSource; 
  }
}

Extending `wp-source`:

const state: WpSource["state"]["source"] = {
  // ...
  api: ({ state }) => {
    if (state.source.url) {
      const suffix = state.source.suffix ?? "?rest_route=";
      return state.source.url + suffix;
    }
    return "";
  },  
  suffix: "/wp-json", // This is the default
  //...
}

Example usage:

const settings = {
  packages: [
    {
      name: "@my-source-package",
      state: {
        // The most simple example, uses the /wp-json default
        source: {
          url: "https://my-wordpress-site.com",
        },

        // custom suffix
        source: {
          suffix: "/custom-suffix"
          url: "https://my-wordpress-site.com",
        },

        // use "?rest_route=" as the suffix
        source: {
          suffix: false
          url: "https://my-wordpress-site.com",
        },

        // custom REST api location
        source: {
          api: "https://my-wordpress.site.com/api/"
          url: "https://my-wordpress-site.com",
        },

        // Everything should still work even if 
        // you don't specify the state.source.url
        source: {
          api: "https://my-wordpress.site.com/api/"
        },
      },
    },
  ],
};

Outstanding questions:

Some features will depend on this new setting (like 301 Redirections). Can we (or should we) fallback to extracting the url from the state.source.api or should any future features throw an error if state.source.url is not available?
About making this work with relative URLs for the embedded mode… I haven’t included that in the proposal, but I think this could be added now or in a future update.

David · 22 October 2020 07:26

I saw a problem yesterday with this proposal regarding WordPress.com sites (I already mentioned this to Michal).

If we simply replace state.source.api by state.source.url we would lose context as, right now, state.source.api is the one used to compute state.source.isWpCom.

That is because the REST API URL for WordPress.com sites is slightly different.

For example, with state.source.api:

{
  state: {
    source: {
      api: "https://public-api.wordpress.com/wp/v2/sites/my-wp-site.com",
      isWpCom: true // <-- This value is derived from `state.source.api`.
    }
  }
}

With the url proposal:

{
  state: {
    source: {
      url: "https://my-wp-site.com",
      isWpCom: true // <-- We can't derive this value from `url`.
    }
  }
}

This could be solved making isWpCom to be assignable in frontity.settings.js.

luisherranz · 22 October 2020 07:35

I don’t disagree with you, as this can be considered a suffix of the URL. But when we “inherit” concepts from WordPress, we always have to try to stay as close as possible to the names already defined in WordPress itself, to avoid confusion.

There is already a filter in WordPress which calls this prefix: https://developer.wordpress.org/reference/functions/rest_get_url_prefix/. It also makes sense, because it’s also the prefix of the URI: [wp-json]/wp/v2/.... So I would stick to prefix because of that.

I really like that idea. It’ll solve the problem very elegantly.

I guess new features don’t need to be backward compatible with state.source.api, because when we instruct people to set state.router.redirections when can ask people to also set state.source.url, right? But we have to remember to put that requirement in the docs.

I really like @juanma idea. If state.source.url is not defined, it fallbacks to state.frontity.url.

My feedback about the proposal:

1. Move `state.source.prefix` to `state.wpSource.prefix`

This is in preparation for Source v2, where we want packages from different APIs, like the REST API and GraphQL, to coexist.

const mergedState = {
  source: {
    url: "https://mybackend.com",
  },
  wpSource: {
    prefix: "/wp-json",
  },
  wpGraphqlSource: {
    prefix: "/graphql",
  },
};

I know that state.source.api is still coupled to wpSource, but that’s something we cannot change now to keep backward compatibility.

2. If `state.source.url` is missing, use `state.frontity.url`

As Juanma suggested, we can ask people to delete state.source.url if they are using the Embedded mode, and it will fall back to state.frontity.url.

3. Support for `?rest_route=`, but using the `prefix` field

That’s a good point, thanks for adding it here. My only opinion is that I don’t see a reason why people should not set it in the prefix, instead of setting it to false:

const settings = {
  state: {
    wpSource: {
      prefix: "?rest_route=",
    },
  },
};

We have to make sure that it works in libraries.source.api.get though.

4. Keep backward compatibility with WordPress.com

We have to make sure that we keep backward compatibility with WordPress.com. Right now, state.source.isWpCom is populated automatically.

Juanma made a series of tests a while ago and discovered that the long public-api URL (https://public-api.wordpress.com/wp/v2/sites/site.wordpress.com) is only used for free sites that use the site.wordpress.com URL.

Thanks to that, I think we can still detect it and keep backward compatibility.

export default {
  state: {
    wpSource: {
      prefix: "/wp-json",
    },
    source: {
      // The default when `state.source.url` is not overwritten in
      // frontity.settings.js is to fallback to `state.frontity.url`.
      url: ({ state }) => state.frontity.url,

      // Keep backward compatibility when `state.source.api` is not
      // overwritten in frontity.settings.js.
      api: ({ state }) => {
        // Check if it's a free WordPress.com site.
        if (/^https:\/\/[\w-]\.wordpress\.com/.test(state.source.url))
          return `https://public-api.wordpress.com/wp/v2/sites/${state.source.url}`;

        return normalize(state.source.url + state.wpSource.prefix);
      },

      // Keep backward compatibility.
      isWpCom: ({ state }) =>
        state.source.api.startsWith("https://public-api.wordpress.com"),
    },
  },
};

mmczaplinski · 22 October 2020 18:00

Awesome, thanks @luisherranz! Not much to add, I agree with pretty much everything:

oh, I see, I was not aware of that. This makes sense.

yep, that’s what I had in mind as well.

You’re right, that would be more intuitive for Frontity users. I was following the WordPress here where ?rest_route= is the default (hence the false)

luisherranz · 2 November 2020 11:04

The normalization of URLs is quite a mess right now. We are going to need to make something much better for Source v2.

Also, I think that the ?rest_route= prefix doesn’t work now. But nobody seems to have requested it yet, so I guess we can fix it later of even wait for the Source v2.

Finally, we have to rethink all the types of the source namespace for Source v2:

What goes into state.source in @frontity/source: this is meant to be included by any source package.
What goes into state.source in @frontity/wp-source: this is meant to be included only by wp-source. I’m not sure if this makes sense anymore.
What goes into state.wpSource in @frontity/wp-source: this is meant to be included only by wp-source.

I have approved the PR now Michal. Feel free to merge and update the FD with the Final Implementation

mmczaplinski · 3 November 2020 23:01

Final implementation

The final implementation consists of the following:

Introduce a new property state.source.url in wp-source that points to the URL of the users’ WordPress backend. The default value of that property is derived from state.frontity.url.
The default value of state.source.api has changed. We now derive that value from the state.source.url. This should have no impact on existing applications, but has an important effect on new apps: You don’t have to specify the state.source.api!

You now can just specify the state.source.url and the state.source.api is computed from it by adding the “prefix” (see point 3. ).
Introduce a new property state.wpSource.prefix where the user can specify prefix of the REST API, for example "/wp-json" or "?rest_route=/". The default value is "/wp-json".
Introduce the state.wpSource.api and state.wpSource.isWpCom. For the moment their values are just derived from state.source.api and state.source.isWpCom respectively. This is added in preparation for the Source V2.

For completeness, here is the actual, final implementation:

// packages/wp-source/src/index.ts

const wpSource = (): WpSource => ({
  // ... 
  state: {
    wpSource: {
      // Just copy the value of `state.source.api` until we deprecate it and
      // move its logic to `state.wpSource.api` in the v2.
      api: ({ state }) => state.source.api,
      // Just copy the value of `state.source.isWpCom` until we deprecate it and
      // move its logic to `state.wpSource.isWpCom` in the v2.
      isWpCom: ({ state }) => state.source.isWpCom,
      prefix: "/wp-json",

      // ...
    },
  },
}

// packages/wp-source/src/state.ts

const state: WpSource["state"]["source"] = {
  // ...
  // Keep backward compatibility when `state.source.api` is not
  // overwritten in frontity.settings.js.
  api: ({ state }) => {
    // Check if it's a free WordPress.com site.
    if (/^https:\/\/(\w+\.)?wordpress\.com/.test(state.source.url))
      return addFinalSlash(
        `https://public-api.wordpress.com/wp/v2/sites/${state.source.url}`
      );

    return addFinalSlash(
      addFinalSlash(state.source.url) + state.wpSource.prefix.replace(/^\//, "")
    );
  },
  url: ({ state }) => {
    if (!state.frontity?.url)
      error(
        "Please set either `state.source.url` (or at least `state.frontity.url` if you are using Embedded mode) in your frontity.settings.js file."
      );
    return addFinalSlash(state.frontity.url);
  },

  //... 
};

luisherranz · 5 November 2020 17:07

Awesome work Michal! Thanks

SantosGuillamot · 16 November 2020 10:02

This feature has been included in the latest release : Read the full Release Announcement .

SantosGuillamot · 17 November 2020 08:49

There are different options here to define the new source.url and I am not sure what is best:

We can define it in the general settings, like frontity.url.
We can define it in the source package, like the old source.api.

Both of them will work, but we should reach an agreement to see how we document and explain it.

If we take mars-theme as example, the initial frontity.settings.js would look like this:

1. Defining source.url in the general settings

const settings = {
  name: "new-project",
  state: {
    frontity: {
      url: "https://test.frontity.org",
      title: "Test Frontity Blog",
      description: "WordPress installation for Frontity development",
    },
   source: {
     url: "https://test.frontity.org",
   },
  },
  packages: [
    {
      name: "@frontity/mars-theme",
      state: {
        theme: {
          menu: [
            ["Home", "/"],
            ["Nature", "/category/nature/"],
            ["Travel", "/category/travel/"],
            ["Japan", "/tag/japan/"],
            ["About Us", "/about-us/"],
          ],
          featured: {
            showOnList: false,
            showOnPost: false,
          },
        },
      },
    },
    "@frontity/wp-source",
    "@frontity/tiny-router",
    "@frontity/html2react",
  ],
};

export default settings;

The wp-source package would come up with no configuration, and if users want to add some settings, like postTypes for example, they would need to change it from a string to an object, like the theme.

If we release new source packages like wpgraphql in the future, users don’t have to change this setting.

2. Defining source.url in each source package

const settings = {
  name: "redirects",
  state: {
    frontity: {
      url: "https://test.frontity.org",
      title: "Test Frontity Blog",
      description: "WordPress installation for Frontity development",
    },
  },
  packages: [
    {
      name: "@frontity/mars-theme",
      state: {
        theme: {
          menu: [
            ["Home", "/"],
            ["Nature", "/category/nature/"],
            ["Travel", "/category/travel/"],
            ["Japan", "/tag/japan/"],
            ["About Us", "/about-us/"],
          ],
          featured: {
            showOnList: false,
            showOnPost: false,
          },
        },
      },
    },
    {
      name: "@frontity/wp-source",
      state: {
        source: {
          url: "https://test.frontity.org",
        },
      },
    },
    "@frontity/tiny-router",
    "@frontity/html2react",
  ],
};

export default settings;

For new source packages, users would have to add this setting to this package, even though it is the same for all the source packages.

I personally don’t have a strong opinion here about which option is better.

@juanma @mburridge Which option do you think fits better in the learning journey you are working on? How would be easier to explain Frontity?

luisherranz · 18 November 2020 10:01

I vote to keep doing it the same way we have done it up until now because that part of the state belongs to the source package.

{
  name: "@frontity/wp-source",
  state: {
    source: {
      url: "https://test.frontity.org",
    },
  },
}

I think it would be confusing to start defining some settings for some packages outside of the package definitions.

juanma · 19 November 2020 11:58

I agree, I think it’s easier to understand that changes in specific namespaces of the state are done in the settings of the package assigned to that namespace

But, if this release includes the use of state.frontity.url when state-source.url is not defined I would start promoting this

const settings = {
  name: "new-project",
  state: {
    frontity: {
      url: "https://test.frontity.org",
      title: "Test Frontity Blog",
      description: "WordPress installation for Frontity development",
    }  
  },
  packages: [
    {
      name: "@frontity/mars-theme",
      state: {
        theme: {
          menu: [
            ["Home", "/"],
            ["Nature", "/category/nature/"],
            ["Japan", "/tag/japan/"],
            ["About Us", "/about-us/"],
          ]
        },
      },
    },
    "@frontity/wp-source",
    "@frontity/tiny-router",
    "@frontity/html2react",
  ],
};

export default settings;

that (in my opinion):

it encourages the idea of zero-config
featured settings can be let as the default ones defined for the theme
less code, easier to understand
we only write the URL of WordPress once

mburridge · 19 November 2020 14:02

I agree that having the WP URL in two places can be confusing. I think that state.frontity.url should be the one that we document, and just mention that state.source.api exists for backward compatibility, or that you can use state.source.url if you’re adding more properties to the source package.

Otherwise, I think that @juanma’s example above looks much cleaner and less cluttered, and so is less confusing. I think that’s the version of frontity.setting.js that should be in a new install of Frontity, i.e. with just the single line "@frontity/wp-source", and the url under state.frontity.url.

My understanding is that state.source.api and state.source.url have precedence over state.frontity.url so if we keep one of those we need to document that it should be removed so that state.frontity.url is the one that will be used, or that that is the one that needs to be configured with the WP API url - I think this would be messy and confusing.

mburridge · 19 November 2020 14:32

I think I might have found a problem with wordpress.com sites where the owner is using a custom URL. I’ve explained it in this loom video

SantosGuillamot · 20 November 2020 08:09

I am not sure we should do that. Right now, at least until the embedded mode is ready and promoted, users will always have to define both frontity.url and source.url, because that is needed for the Decoupled mode.

frontity.url: This is the frontend url, the one pointing to node.js. Ex: mysite.com
source.url: This is the backend url, the one pointing to WordPress. Ex: wp.mysite.com

I think this could be the issue you are facing @mburridge. You are defining the WordPress url in the frontity.url setting. It should go in the source.url. In your case:

frontity.url: http://localhost:3000.
source.url: http://mysite.wordpress.com

mburridge · 20 November 2020 09:46

Hi @SantosGuillamot

I thought that the frontity.url means that you don’t need the source.url - though you can still use it, or even source.api if you want. Is my understanding incorrect?

I made the change you specified and the problem still exists with internal links when a wordpress.com site is using a custom domain.