Customize Webpack configuration

This is the meeting we had, with a summary of how the dev and build scripts generate the client and server bundles, and with an special emphasis on the Webpack configuration and how we thought packages should be able to extend that configuration.

It also has an explanation of a changes we plan to do: Instead of generating the all the client bundles for all the sites in a single Webpack run, use a single Webpack run for each site.

These are the drawings: Excalidraw | Hand-drawn look & feel • Collaborative • Secure

1 Like

@nicholasio.oliveira: could you please add a small explanation about the use case of Webpack extensibility that you need? Thanks! :slightly_smiling_face:

Hi there,

I need to add this to the webpack file

module: {
  rules: [
    {
      exclude: [/node_modules\/(?!(swiper|dom7)\/).*/, /\.test\.js(x)?$/],
      test: /\.js(x)?$/,
      use: [{ loader: 'babel-loader' }],
    }
  ],
}

Could you please point me to the right direction?

Thanks,
Jeff

Hey @jeffceirello :wave:

This is the webpack file that contains the module definition: frontity/modules.ts at dev · frontity/frontity · GitHub

You can patch it using patch-package - npm if you want.

I’ve started looking into this and recorded a small video to get some early feedback.

1 Like

I love it! :heart::grinning_face_with_smiling_eyes:

My only feedback would be to use a named export in the frontity.config.js file, to be able to add more stuff in the future:

// frontity.config.js
export const webpack = ({ ... }) => ...

Other than that, it looks great!

By the way, we also had a FD to customize babel configuration (Customize Babel configuration). But maybe we can close that for now, because configuring Babel will be possible using this:

// frontity.config.js
export const webpack = ({ config }) => {
  const babel = config.module.rules.find(
    (rule) => rule.use.loader === "babel-loader"
  ).options;

  // Add a new plugin.
  babel.plugins.push("my-own-babel-plugin");
};

The other option would be to get the frontity.config files at this point: frontity/index.ts at dev · frontity/frontity · GitHub and run the babel config through an babel exports of the frontity.config.js files:

// frontity.config.js
export const babel = ({ config }) => {
  // Add a new plugin.
  config.plugins.push("my-own-babel-plugin");
};

Then, pass the babel config to getWebpack() and run the webpack config through the webpack exports of the frontity.config.js files.

What do you think?

Another question that comes to my mind is if we should pass target as a parameter of the function, or pass a config object with all targets:

export const webpack = ({ config, target }) => {
  // Add something only on the server.
  if (target === "server") {
    config.someConfig = "...";
  }

  // Add something only on the client.
  if (target !== "server") {
    config.someOtherConfig = "...";
  }
};
export const webpack = ({ config }) => {
  // Add something only on the server.
  config.server.someConfig = "...";
  // Add something only on the client.
  config.module.someOtherConfig = "...";
  config.es5.someOtherConfig = "...";
};

Same for babel (if we add it).

EDIT: Nevermind, passing target as a function argument as you did seems much nicer.

Oh you are right! That’s a good call.

Indeed, that’s doable. Wondering how often would someone need to only adjust the babel configuration and not the whole config? :thinking: On the flip side, having to walk the webpack config only to get to the babel loader it’s painful. Gonna test it out.

One more thing I was meaning to ask is that currently how I thought about it, is that only the configuration files from within packages are read and apply. Should we allow a site specific configuration as well? Meaning:

/packages
/packages/theme/[...]
/packages/theme/frontity.config.ts # <--This is allowed and works
[...]
/frontity.settings.ts
/frontity.config.ts  # <-- Should we allow this?

What are your thoughts?

Recorded another video about it Loom | Free Screen & Video Recording Software

When I first thought about the frontity.config.js file, I thought that a site/root one would be cool, but right now I can’t think of any configuration that it is not better done at a package-level.

So to avoid confusion and anti-patterns, I would not include it at first. If we discover a real need for it later, we can always add it :slightly_smiling_face:

1 Like

By the way, for context, @cristianbote has started a PR to add this feature: Custom webpack config by cristianbote · Pull Request #812 · frontity/frontity · GitHub

I mentioned there this:

If we include the frontity configuration as well, as you showed in the video, we can create a package @frontity/vercel that uses the new .vercel_build_output folder for the build if process.env.VERCEL exists.

Another thing we want to add later to the frontity.config.js file is the ability to modify the site settings and add a package priority.

For example, the @frontity/amp package could set mode: "amp" automatically using this:

// High priority, run before any other package.
export const priority = 1

// Set the mode to "amp" so the user doesn't have to.
export const settings = ({ settings }) => {
  settings.mode = "amp";
}

So definitely it’s a good idea to use named exports :slightly_smiling_face::clap:

The PR is now merged :tada::tada:

3 Likes

I would like to test it first and decide after that. If it’s simple maybe with a CodeSandbox it’s enough. I will let you know.

1 Like

I can surely try! :smiley:

1 Like

Final Implementation

Pull Request

Functionalities

Frontity uses webpack as the build tool, to bundle and generate the needed JavaScript to run on the server but also on the client. We use a specially crafted set of configurations to generate them, which takes into account the target for the bundles – which can be server, module and es5, and because of that extending the configuration or customising it was not an easy task. One would have to patch the frontity package or give up on their customisations. That is not ideal, as you can imagine, since customising the bundling configuration is usually something that site frameworks are allowing and encourages the community to take it further.

Webpack is a powerful and established tool. It allows one to use build time configurations to effectively enhance the generated output, regardless of the target.

Context

The inner architecture of frontity is composed of three main configuration, which in the end are funnelled into webpack’s configuration.

  • Frontity configuration
  • Babel configuration
  • Webpack configuration

Goals

The main goal is to allow one to define a frontity.config.(j|t)s file that can modify the configuration by mutating the passed config. A generic example is this:

// `example` here is a placeholder for frontity, babel or webpack
export const example = ({
  // config, is the default configuration
  config,
  // target, refers to the three possible targets: server, module and es5
  target,
  // mode, is either development or production
  mode
}) => {
  // inside here you can modify them as you wish
};

Out of scope

The config file is designed to be defined by package and used by a site. It can not be defined at the site level. This is intended and it is like this to keep each custom configuration in it’s own context and scope.

Technical explanation

The proposed solution is the one that reads the packages configuration files, and creates a dictionary

Read the config files

First we need to read up the configuration files from all the packages, before running webpack. In order to do that, we have to construct a sort of dictionary where we can collect each defined configuration function.

const dictionary = {
  frontity: [
    // list of config functions for frontity context
  ],
  babel: [ /* ... */],
  webpack: [ /* ... */]
};

Why the need of a dictionary?
Multiple packages can export and define customisations for each of the three contexts. That means that we need to be able to run all of the packages customisations.

After we have the above dictionary computed, we can pass it along to the getConfig function in the scripts file.

Run each configuration context

All the magic happens in the getConfig function. This function runs the frontity, babel and webpack functions to generate the configurations to be used as configurations for the compiler. So, that means at this point we need to run the customisation functions. This results in something like:

const config = ({
  mode,
  extraConfigurations,
  ...rest // other configurations
}) => {
  // Default config for frontity
  const frontity = getFrontity();

  // Run the entries for frontity.
  extraConfigurations.frontity.forEach(fn => {
    fn({ config: frontity, mode });
  });

  // Default config for babel. Keep in mind this configuration is split by target.
  const babel = getBabel();

  // For each target run the extra configuration for babel
  for (const target in babel) {
    extraConfigurations.babel.forEach(fn => {
      fn({ config: babel, mode, target });
    });
  }

  // This returns the default webpack configuration split for each target as well.
  const webpack = getWebpack({
    mode,
    babel,
    frontity,
    ...rest // rest of the passed configs
  });

  // For each target run the extra configuration for webpack
  for (const target in babel) {
    extraConfigurations.webpack.forEach(fn => {
      fn({ config: webpack, mode, target });
    });
  }

  return {
    babel,
    webpack,
    frontity,
  };
};

That’s it!

At this point, each package extra configuration ran, and the returned configurations for frontity, babel and webpack are customised.

1 Like

@luisherranz @SantosGuillamot let me know if the IP needs more deep dive or a better explanation.

The final implementation is so simple that I think a video demo isn’t needed :clap:. I’ve created this simple CodeSandbox to show an example of how to extend both WebPack and Babel. Some comments about it:

  • We could have added the frontity.config.js file to the mars-theme , but I wanted to show how easy it is to create a new package for this.
  • I changed the Devtool property for WebPack in development mode, but you could change anything in the WebPack configuration.
  • I added the simple Babel plugin transform-remove-console to remove console.logs only in production, but again you can change anything in the Babel configuration.
  • If you want to check the different behaviour in the own CodeSandbox, you can change the package.json dev script (this is what CodeSandbox uses), add the flag --production , and restart the server. Without the flag you can see “Development Mode” in the console, and if you add the flag you shouldn’t see it after restarting the server.
  • This is the code I added for this example:
export const webpack = ({ config, mode }) => {
  if (mode === "development") {
    config.devtool = "eval-cheap-source-map";
  }
};

export const babel = ({ config, mode }) => {
  if (mode === "production") {
    config.plugins.push([
      "transform-remove-console",
      {
        rules: [
          {
            exclude: ["error", "warning"]
          }
        ]
      }
    ]);
  }
};

3 Likes

I think the explanation is great, thanks Cristian :slightly_smiling_face: Just a minor comment. I think you used the Implementation Proposal template instead of the Final Implementation template. It’s true that in this Feature Discussion we didn’t have an initial implementation proposal, but as it is already released and done, I think we can write the Final Implementation directly. Anyway, I think the explanation you provided should be enough. Could you please slightly update the titles to reflect it is the Final Implementation please?

1 Like

Oh no! :see_no_evil: on it. Thank you!