The Settings API

luisherranz · 28 February 2019 12:17

Ok, the first thing we need is a package system for the settings.

There may be different packages in the future, depending on where the settings are stored:

file-settings: save settings in a setting.json file in your project folder.
wp-org-settings: save settings in wp mysql database
mongo-settings: save settings in mongo
graphql-settings: save settings using graphql

and so on…

I think it’s ok if users need to define their settings package in the frontity.config.js file. I also think it’s ok that there’s only one settings package activated at a time.

// frontity.config.js
module.exports = {
  settingsPackage: '@frontity/file-settings' 
}

The first package we are going to develop is file-settings and it’s going to be the default one. This means there’s no need for a frontity.config.js to start using Frontity.

I’ve done a quick draft of the server architecture:

Frontity must import the correct package and use their getSettings method.

Ok, the first problem is: Frontity must import the correct package. I’ve opened a new thread about that because it’s important and we also need it for the “App Packages”: How to resolve dynamic imports in Webpack

Now, I’ll continue assuming we’ve solved that

Constrains

The settings must be serializable, they cannot contain code.
This is not very important for file-settings but it is for packages that save them to a database.
The developer must be able to force the use of another site using a query ?site=my-other-site.

I’ll continue working on this tomorrow.

luisherranz · 6 March 2019 08:58

All yours @orballo.

I’d work first on the constrains of this design

orballo · 6 March 2019 09:14

I’m afraid I don’t know the constrains nor the features.

The only thing I could understand from the things I read is that you want something like:

{
   	name: "example-name",
   	url: "https://mysite.com",
   	mode: {
		default: "*",
   		amp: "\/amp\/?$"
   	},
  	packages: [
  		{
  			name: "@frontity/theme",
  			namespace: "theme",
 			mode: "default",
            settings: {}
   		}
   	]
}

I’m completely blocked on this issue.

luisherranz · 6 March 2019 09:24

Features: things we want the settings API to have.
Constrains: things that must be done in a certain way for some reason and may limit the design.

It’s not what I want. Think about what we need and what would make a good settings API that follows our design principles. Use the experience we have from the old framework and the new things we want to achieve with this one.

This issue may be helpful as well. I’ve been writing about some features and constraints that directly relate to the settings API.

orballo · 6 March 2019 14:57

Ok, this is the way I imagine the settings being used:

The settings need to be serializable, and can be an array or an object. The settings being an array is useful to store the settings of different sites in just one file.

The API should be as follows:

module.exports = {
  name: "example-name", // Useful only when there is more than one site.
  mode: {
    pwa: "*", // Default.
    amp: "\/amp\/?$", // Default.
    // More modes can be defined here.
  },
  bundle: {
    babel: "old useragent match", // Default.
    es6: "new useragent match", // Default.
  },
  packages: [
    {
      name: "@frontity/theme",
      settings: {
    	color: "#FFF",
    	mode: {
    	  amp: {
            color: "#000",
    	  }
        }
      }
    }
  ]
}

If we remove the default values the API would be as follows:

module.exports = {
  packages: [
    {
      name: "@frontity/theme",
      settings: {
    	color: "#FFF",
    	mode: {
    	  amp: {
            color: "#000",
    	  }
        }
      }
    }
  ]
}

`name`

The name of the settings that you want to use in your current app. This is an useful identifier in case you are developing one theme for many different sites. Setting it as a query param will identify which are the required settings. If this param is not provided, the first settings in the array should be used.

`mode`

The different modes that Frontity supports and it’s patterns to identify them in the url.

`bundle`

The different bundles that can be served and the pattern to identify them in the user-agent of the request.

`packages`

An array of the packages used in the Frontity app.

name the name of the package declared.
settings the settings of the declared package.
mode specific settings if needed for the different modes.

Some notes

I can’t think of a use for the url field and I think the WP API url should be defined inside the wp-source package settings.

I also think that developers shouldn’t need to be mindful of the namespaces exposed by the packages they use and shouldn’t need to define them here, if there is a way to avoid it. But I’m not sure what’s the use of namespaces here, I’m assuming it’s to avoid including stores for those namespaces if they are not used in the app.

Please let me know what do you think and what I’m missing.

luisherranz · 6 March 2019 18:08

The problem (that you actually pointed out when I was working on this) is that, without a namespace, we don’t know where those settings are going to end up. For example:

    {
      name: "@frontity/some-theme",
      settings: {
    	aThemeSetting: "#FFF",
        aRouterSetting: true
      }
    }

How do we know that aThemeSetting goes to settings.theme and that aRouterSetting goes to settings.router?

As I commented here, I don’t like the pwa name. It doesn’t make sense anymore. I don’t quite like default either. We need to find something better

In our old framework the URL was used by several packages. I think it’s important. But I agree with you with the WP REST API URL. That should go in the wp-source package.

I think we need a way to store general settings. Not only the URL but things that are needed by all the extensions like the timezone or the language.

Last two things:

I don’t quite understand what you are trying to accomplish with bundle.
We need a way to distinguish between different sites. Remember, now the domain can point directly to Node, so no siteId is involved. I proposed to use a match field that contains regexps for the URL:

exports default [{
  name: "main",
  match: "https://site.com/.*"
  ...
}, {
  name: "blog",
  match: "https://blog.site.com/.*"
  ...
}]

luisherranz · 7 March 2019 08:34

I’ve been think a lot about what you said and I think you’re right, we should not require a frontity.package.js file. That information is already in the frontity.settings.js anyway.

I don’t know if you’ve already thought about the API of the settings packages but in my head it was something like :

getSettings({ url, site }): retrieves the single settings for a site, using either its url or its site name.
getPackages({ url, site }): retrieves all the packages of a site, using either its url or its site name.

We would need to add another one:

getAllPackages(): retrieves all the packages of all the sites.

Then use that getAllPackages() function to create our frontity.packages.js file internally before Webpack runs.

orballo · 7 March 2019 09:15

If we follow our design principles, this field, somehow, should have a default value. Maybe the package should be the responsible of defining those namespaces? Even if it can be overridden in the settings later. Don’t know if is something that can be achieved by overmind’s namespaced, for example, I don’t actually know what that function does

It seems to me that the url value can be known in the request, and also is something that belongs more to build maybe? I see build like derived state, so if we can get these values from somewhere else, maybe we should avoid to define them in settings.

I agree. I didn’t think of any but, yes, I think they should just go in the root level.
I’m worried here about mixing content-related settings with Frontity settings. I think we should try to have a separation of concerns. Not sure if this makes sense, though.

I know, it makes no sense. I thought that maybe the dev would like to change the user-agent match that determines what bundle to send, but the best option should be used whenever possible.

Yes, you are right. I missed that one.

orballo · 7 March 2019 09:19

I didn’t know that different sites could use different packages. That way the bundle will be bigger than neccessary, right?

I’m OK with that API.

orballo · 7 March 2019 13:32

Are the settings packages some kind of special package? Or are they treated exactly the same than other packages. I mean, for example, in the way the package is used by Frontity.

luisherranz · 8 March 2019 07:42

They export them, but they can export several.

Sure. Let us know if you come up with a solution!

Each Frontity namespace will end up in an Overmind namespace, but that doesn’t help us know what settings go to what namespace:
https://overmindjs.org/api/config?view=react&typescript=false#config-namespaced

Where would derive the URL from?

I think one file/place for all the settings of a site is better than two, so my vote goes for keeping the general settings in the frontity.settings.js file. It seems too complex:

// frontity.settings.js
{
  name: "my-site",
  settings: {
    "my-theme": {
    ...
    },
    "wp-source": {
    ...
    }
  }
}
// frontity.general.settings.js
{
  name: "my-site",
  settings: {
    url: "https://www.my-site.com",
    language: "en",
    timezone: "+2"
  }
}

The configuration of the server itself (webpack, babel…) already goes to the frontity.config.js file.

Oh, both esModules and es5 are always included in the HTML because it’s the browser the one who chooses the right one

Yes, that problem is explained here in detail: How to resolve dynamic imports in Webpack

Yes, they are not regular Frontity packages because they are consumed by the server before the SSR starts.

luisherranz · 8 March 2019 07:48

An update of the constrains:

Constrains

Settings can be retrieved by either the URL or the “site name”.
A site configuration can contain “general settings” as well, not belonging to any package.
Different modes can be defined for each site.
Different modes can have different packages and/or settings configuration.
There must be a way to know which settings belong to which namespace of a package.
The settings must be serializable, they cannot contain code.

I’d remove this constrain because it has nothing to do with this API:

The developer must be able to force the use of another site using a query ?site=my-other-site.

orballo · 8 March 2019 15:50

Ok, after what we talk in the meeting, I think this is the final (at least for now) API schema that we are going to use:

export default [
  {
    // The name of the settings. Only needs to be populated in case there is more than one site. Defaults `null`.
    name: "settings-name-1",
    // The url of the site. Defaults to `null`.
    url: "https://site.com/",
    // Match used against the url passed in `getSettings` to identify the needed settings.
    match: "/https:\/\/site.com\//",
    // Matches used against the url passed in `getSettings` to identify the render mode.
    mode: {
      pwa: "*", // This name should be changed. This is the default value.
      amp: "/\/amp\/?$/", // This is the default value.
    },
    // Number defining the offset from UTC. Defaults to 0.
    timezone: 2,
    // Defaults to "en".
    language: "en",
    // Defaults to `null`.
    title: "Awesome Title",
    // Used packages and it's settings. Defaults to [].
    packages: [
      {
        // Package name in `npm`. This field is mandatory.
        name: "@frontity/theme",
        // Allows for the dev to deactivate a package without removing its settings. Defaults to `true`.
        active: true,
        // Mode in which this package is used. It can be a string or a string[]. Defaults to ["pwa", "amp"].
        modes: "amp",
        // Namespaces used. Defaults to an array containing all the namespaces ["theme", "comments", "h2r"].
        namespaces: "theme",
        // Settings for this package.
        settings: {
          color: "#FFF",
          // Specific mode settings that override the general package settings.
          mode: {
            amp: {
              color: "#000",
            }
          }
        }
      }
    ]
  }
]

[WIP] There are some things missing yet.

luisherranz · 11 March 2019 09:01

I’ve been thinking that it’d be great if both frontity.config.js and the default settings of each package could have typings. It is only useful for file-settings. Other settings package will have to find a way to type their own settings if they want.

This is just an idea:

import Settings from "@frontity/types-settings" // Base typings
import { Settings as ThemeSettings } from "my-theme"
import { Settings as WpSourceSettings } from "@frontity/wp-source"

const settings: Settings<ThemeSettings, WpSourceSettings> = {
  name: "my-site",
  url: "https://site.com",
  packages: [
    {
      name: "my-theme",
      settings: {
        // this has typings now!
      }
    },
    {
      name: "@frontity/wp-source",
      settings: {
        // and this!
      }
    }
  ]
};

export default settings;

And let the packages export their default settings (the same typing is used):

type ThemeSettings = {
  color: "white" | "black";
  featuredImage: boolean;
}

const defaultSettings: ThemeSettings = {
  color: "white",
  featuredImage: false
};

const theme = {
  root: ThemeRoot,
  store: ThemeStore,
  Settings: ThemeSettings, // These are the types
  settings: defaultSettings, // These are the defaults
};

export default {
  theme
}

luisherranz · 11 March 2019 09:10

Two additional constrains:

Constrains

Settings can be retrieved by either the URL or the “site name”.
A site configuration can contain “general settings” as well, not belonging to any package.
Different modes can be defined for each site.
Different modes can have different packages and/or settings configuration.
There must be a way to know which settings belong to which namespace of a package.
Settings must be serializable, they cannot contain code.
Packages can be either “active” or “inactive”.
Namespaces of a package can be either “active” or “inactive”.

luisherranz · 11 March 2019 12:00

Quick idea: what if we remove the modes regexp and use match to distinguish between them?

export default {
  "my-site": [
    {
      mode: "amp",
      match: "https://my-site.*/amp$",
      settings: {...},
      packages: [ ... ]
    },
    {
      mode: "html",
      match: "https://my-site.*",
      settings: {...},
      packages: [ ... ]
    }
  ]
}

Is it simpler or more complex?

Default mode is "html" and default match is ".*":

export default {
  "my-site": {
    settings: {...},
    packages: [ ... ]
  }
}

luisherranz · 11 March 2019 12:02

It may be more work if most of the settings are the same but it is clearer when they are not. We also reduce from two configurations (regexp for mode, regexp for site) to one.

orballo · 11 March 2019 13:39

I don’t dislike the idea, but I’d aim for the simplest case to look like:

export default {
  settings: { ... }, // Optional
  packages: [ ... ]
}

luisherranz · 11 March 2019 16:01

And do you like it?

orballo · 11 March 2019 16:06

You know is not the same