AMP package

Description

Implement an easy way of implementing AMP in Frontity projects.

User Stories

As a Frontity user
I want to be able to use AMP in my project
so that I can easily speed up the loading time of web pages on mobile devices

Possible solution

We were about to add support to AMP using settings’ mode: "amp", but it doesn’t play nice with the current new implementation of entry points.

This is the list of things that need to happen to make Frontity render valid AMP HTML:

  1. The entry points are switched to amp.js or /amp/index.js.
    – That is precisely what is happening now when set mode: "amp" in the settings.
  2. There should not be a client bundle, only a server one.
    – That is not happening right now.
  3. The HTML template needs to be changed to a AMP-HTML template.
    – That is hardcoded right now (bad).
  4. Some setting must be set to "amp" so packages can check if AMP is on.
    – That is done now with state.frontity.mode which is populated with the settings mode.

My proposal is to switch to a package that does all these things. It could be called @frontity/amp.

We could then remove mode from settings, but need to expose both entryPoints and bundles. This makes it much more hackable and extensible.

Let me explain how it works:

  1. People who want to configure a site in AMP need to add the @frontity/amp package in their frontity.settings.js file:
{
  name: "my-site-amp",
  match: "\\/amp\\/?($|\\?|#)", // This matches any url ending with /amp/.
  packages: [
    "@frontity/amp",
    "@frontity/mars-theme",
    "@frontity/wp-source"
  ]
}

That’s it, people don’t need to do more.

  1. Internally, the AMP package uses its own frontity.config.js to change the settings of that site:
export default {
  settings: site => {
    // This looks for amp.js entry points if they exist.
    site.entryPoints = "amp";  
    // This avoids the creation of the client bundle for this site.
    site.bundles = ["server"];
  }
}
  1. It also adds a state.frontity.mode equal to "amp" in its index.js:
export default {
  state: {
    frontity: {
      mode: "amp"
    }
  }
}
  1. And finally changes the HTML template extending the server in the koa.js file.
    – the name of the koa.js file is yet to be determined.
import AmpTemplate from "./template";

export default app => {
  app.use(ctx => {
    ctx.template = AmpTemplate;
  });
}

I think the coolest thing about this approach is that there won’t be anything hardcoded in our AMP support. Any person could have done this package by extending Frontity. So in the future, people can create other packages for other modes following similar patterns.

Any ideas, suggestions or feedback… is welcomed.

6 Likes

The problem in emotion with <Global> and AMP has been solved and will be released in their next release.

We can continue with this.

Released in version @emotion/[email protected]. We need to update to that minimum version and check that not everything ends up in the <head>.

1 Like

Can anyone tell if I can now use AMP mode safely for production or it will be in the next release??

I’m afraid that AMP hasn’t been developed yet. We were waiting for WP to add some AMP info in the REST API, but it’s taking more time than expected, so maybe we just have to look for another solution. I’ve moved this topic to the Features category, where you can vote for it and share your opinion. If we see that a lot of people are interested in it, we can start working on it again.

@luisherranz any updated ETA on this? I see wp-core merged the patch for extending schema, unblocking your amp-wp PR, yes?

Yes, absolutely. Sorry for not keeping this topic updated.

It seems like my patch is going to be merged in 5.4 and that means we will be able to finish the amp-wp PR to expose AMP in the REST API, but I simply haven’t had time lately to keep working on that PR.

AMP is quite important for some big publishers currently migrating to Frontity, so don’t worry, this is still a top priority for us!

2 Likes

This is the implementation proposal for the AMP package.

Description

The AMP package will be a Frontity package that will allow the creation of Google AMP pages with the Frontity framework. We want to do this with a package, @frontity/amp, instead of hardcoding it in the framework.

Thanks to the multisite feature of Frontity, themes can be reused for both AMP pages and React sites.

Glossary

  • AMP-aware
    Some function/component/package smart enough to check for itself if the site is an AMP site or not, and adapt to the situation.

Context / Background

We already had AMP support in the old version of the framework and it worked great. About 90% of the theme code was reused among the React site and the AMP pages, including all the layout and CSS. AMP support was hardcoded in the framework core.

Goals

As a Frontity developer
I want to use the same React codebase to create the AMP version of my site
so that I don’t have to maintain two separate codebases.

Out of Scope

This is out of the scope of this Feature Discussion but will be acomplished in a future FD.

As a Frontity developer
I want to use AMP content translated with the WP AMP plugin
so that I don’t have to use Html2React for that

Existing Solution / Workaround

There’s no way to use AMP right now with Frontity in Decoupled mode.

In Embedded mode, people could exclude the URLs that end in /amp and use the AMP plugin to do AMP. But, of course, the theme of that AMP version won’t be anything similar to the Frontity theme.

Implementation Proposal

To be able to generate valid AMP HTML, this package needs to achieve several tasks.

Change the entry point of the packages

Some packages may require a different entry point for AMP because sometimes they are going to contain code that is not needed in the React version and we need to provide tools to make the bundle size of the React version as slim as possible.

This means that instead of using src/index.js, packages can use src/amp.js or src/amp/index.js as entry points when the site is an AMP site. This is of course optional, and the packages that don’t need it or don’t implement this can still use src/index.js.

It will be managed by a new site setting on the frontity.settings.js file:

export default {
  name: "my-amp-site",
  entryPoints: "amp",
  state: {
    //...
  },
  packages: [
    //...
  ],
};

This feature requires a new FD. I will edit this IP as soon as it is created.

Don’t generate client bundles

We don’t need to generate bundles for the client because the Google AMP framework forbids the use of external JavaScript.

It will be managed by a new site setting on the frontity.settings.js file:

export default {
  name: "my-amp-site",
  bundles: ["server"],
  state: {
    //...
  },
  packages: [
    //...
  ],
};

This feature requires a new FD. I will edit this IP as soon as it is created.

Modify site settings with packages

The @frontity/amp package needs to be able to change the entryPoints and bundles settings.

This will be managed by a settings export in the frontity.config.js file of this package.

// packages/amp/frontity.config.js
export const settings = ({ settings }) => {
  settings.entryPoints = "amp";
  settings.bundles = ["server"];
};

The settings export of the frontity.config.js file of each package will be executed right after getting the settings from the frontity.settings.js file.

This feature requires a new FD. I will edit this IP as soon as it is created.

Package priorities

The @frontity/amp package needs to be executed before any other package because it is going to change the entryPoints setting and that will affect how the rest of the packages are imported.

This will be managed by a priority export in the frontity.config.js file of this package.

// packages/amp/frontity.config.js
export const priority = 2;

Users can override this setting in the frontity.settings.js file:

export default {
  name: "my-amp-site",
  packages: [
    {
      name: "some-package",
      priority: 5, // Override default priority for this package.
      state: {
        // ...
      },
    },
  ],
};

This feature requires a new FD. I will edit this IP as soon as it is created.

Overwrite the HTML template

The Google AMP framework needs a specific HTML template because a normal HTML document doesn’t validate.

For that reason, the @frontity/amp package needs to be able to overwrite the hardcoded HTML template.

This can be done with a beforeSSR function but needs the template to be exposed in libraries.frontity.

// packages/amp/src/index.js
import ampTemplate from "./templates/amp";

export default {
  actions: {
    amp: {
      beforeSSR: ({ libraries }) => {
        libraries.frontity.template = ampTemplate;
      },
    },
  },
};

The template should be something like this: https://github.com/frontity/frontity/blob/31d34e4d16042ebc73b4c30a9f53dfae36c633a2/packages/core/src/server/templates/amp.ts

Render to Static Markup

The Google AMP framework doesn’t need the custom data- attributes added by React, so instead of using renderToString, the @frontity/amp package needs to make Frontity use renderToStaticMarkup.

This can be done with a beforeSSR function. It’ll be necessary to create a separate server.js file because we don’t want to add react-dom/server to the client bundle.

// packages/amp/src/server.js
import { renderToStaticMarkup } from "react-dom/server";

export default {
  actions: {
    amp: {
      beforeSSR: ({ libraries }) => {
        libraries.frontity.render = renderToStaticMarkup;
      },
    },
  },
};

It needs the render function to be exposed in libraries.frontity.

Move CSS to the <head>

The Google AMP framework requires that all the CSS is placed in a <style> tag of the head.

For that reason, the @frontity/amp package needs to use Emotion’s extractCritical API and move the extracted CSS to the <head>.

This can be done with a beforeSSR function. It’ll be necessary to create a separate server.js file because we don’t want to add create-emotion-server and react-dom/server to the client bundle.

// packages/amp/src/server.js
import { CacheProvider } from "@emotion/core";
import createEmotionServer from "create-emotion-server";
import createCache from "@emotion/cache";
import { renderToStaticMarkup } from "react-dom/server";
import ampTemplate from "./templates/amp";

export default {
  actions: {
    amp: {
      beforeSSR: ({ libraries }) => {
        // Create an Emotion instance for this SSR.
        const cache = createCache();
        const { extractCritical } = createEmotionServer(cache);

        libraries.frontity.render = (App) => {
          // Add the Provider to the App.
          App = (
            <CacheProvider value={cache}>
              <App />
            </CacheProvider>
          );

          // Run extract critical.
          const { html, css, ids } = extractCritical(
            libraries.frontity.render(App)
          );

          // Replace the template injecting the CSS in the `<head>`.
          libraries.frontity.template = ampTemplate(css, ids);

          return html;
        };
      },
    },
  },
};

We also need to be able to add the CacheProvider to the client. That can be done with a beforeCSR function.

// packages/amp/src/client.js
import createCache from "@emotion/cache";

export default {
  actions: {
    amp: {
      beforeCSR: ({ libraries }) => {
        const cache = createCache();

        libraries.frontity.App = () => (
          <CacheProvider value={cache}>
            <libraries.frontity.App />
          </CacheProvider>
        );
      },
    },
  },
};

For context, this is the way Emotion worked in Frontity at first. This is the final commit, including a fix for the Global component: https://github.com/frontity/frontity/commit/e8c3430

After that, we removed extractCritical because both labels and source maps don’t work with that approach. Now it’s up to the @frontity/amp package add extractCritical again for AMP sites. This is where I removed that approach: https://github.com/frontity/frontity/commit/d5af653

Looking at the new docs it seems to me that we don’t need to do a separate hydrate() on the ids anymore, as they are present in the data-emotion-css attribute of the style tag:

<style data-emotion-css="${ids.join(' ')}">${css}</style>

More info about the extractCritical approach on https://emotion.sh/docs/ssr#advanced-approach

Modify link to point to the canonical URL

There are three standard ways to use AMP. This package needs to support all of them.

  1. Using a subdomain, like https://amp.mydomain.com/some-post.
  2. Using a query parameter, like https://www.mydomain.com/some-post?amp=true.
  3. Using an extra folder in the path, like https://www.mydomain.com/some-post/amp.

1 and 2 are fine because the slug used by @frontity/wp-source will be some-post, but 3 is not.

We need to make this action and derived state point to /some-post/:

actions.source.fecth("/some-post/amp");
state.source.get("/some-post/amp");

Ideally, this should be done with hooks, but we don’t have hooks yet and they seem too much to make them a dependency of this feature right now.

We’ve been searching for a workaround but the only thing that seems feasible right now is to hardcode a replacement in the normalize function and change it once we have hooks.

export const normalize = (route: string): string => {
  route = route.replace(/\/amp$/, "");
  return paramsToRoute(routeToParams(route));
};

This will need to contain a bit more logic to avoid some known URLs that shouldn’t be changed, like:

  • A page called amp: /amp.
  • A category/tag called amp: /category/amp or /tag/amp.

I wouldn’t try to make it perfect though, taking into account that this will be replaced by a hook in the (hopefully) near future.

AMP processors

A theme that needs to supports AMP should contain AMP-aware components, but the HTML that is inside of post.content.rendered is outside of the scope of the theme because it’s managed by Html2React.

The package @frontity/amp should use processors to translate the HTML tags that need changes in the AMP framework, like images, iframes, and so on.

We already did a lot of related work in the old version of the framework, where many of the components used by processors were AMP-aware, like this one: https://github.com/wp-pwa/h2r/blob/dev/src/components/LazyIframe/index.js#L62-L87

This time we don’t need to check if we are in an AMP render, as this processors will only be used in an AMP render because we know that the @frontity/amp was the one that added them.

// packages/amp/src/index.js
import amp from "./processors";

export default {
  state: {
    amp: {},
  },
  libraries: {
    processors: [...amp],
  },
};

Alternatively, we will be able to use the AMP Plugin to do this in the future once this PR is finished and merged: https://github.com/ampproject/amp-wp/pull/2827

AMP-aware components

The components that we export in @frontity/components need to be aware of an AMP render and act accordingly.

For example, the Image component needs to return <amp-img> instead of <img> and so on.

const Image = ({ state, src }) => {
  if (state.amp) {
    return <amp-img src={src} />;
  } else {
    return <img src={src} />;
  }
};

Make official Frontity packages AMP-aware

There are some packages that need to be modified to make them work in AMP sites. Right now only:

  • @frontity/comscore-analytics
  • @frontity/google-analytics
  • @frontity/google-tag-manager-analytics

But in the future other packages, like ads packages.

Test Plan / Acceptance Criteria

Apart from normal e2e tests, we need to make sure that the HTML generated is always valid AMP HTML.

As far as I know, Cypress injects jQuery in the HTML, so we cannot validate the HTML directly, but we can do a request of the HTML from Cypress and validate the HTML using https://www.npmjs.com/package/amphtml-validator. Or maybe we don’t even need Cypress for this and we can just use Jest. That is going to depend on how our e2e system works.

It’d also be cool to test against a WordPress instance loaded with this content
https://github.com/WPTT/theme-unit-test which covers a lot of content and blocks, to make sure they all validate correctly with the amphtml-validator package.

cc: @mmczaplinski maybe you want to have this new example in mind while designing the e2e system

Dependencies

We need new Feature Discussions for:

  • New setting to change the entry point of the packages of a site.
  • New setting to change what bundles are generated (client and/or server).
  • Allow packages to modify the settings of a site.
  • Package priorities.

Individual Tasks

This is a non-exhaustive list of tasks that need to be accomplished once the dependencies have been created.

  • Expose in libraries.frontity:
    • The HTML template.
    • The render function (currently renderToString).
    • The App.
  • Create the @frontity/amp package.
    • Add frontity.config.js file for:
      • Setting the entryPoints to amp.
      • Setting the bundles to ["server"].
      • Setting priority to 2.
    • Overwrite the HTML template.
    • Move all the CSS to the <head>.
    • Change the render to be renderToStaticMarkup.
    • Create the AMP processors to translate the post content to AMP.
    • Make all the components and hooks of @frontity/components and @frontity/hooks AMP-aware.
  • Remove /amp from link in actions.source.fetch and state.source.get to point to the canonical URL.

Beta version

If we want to release a beta version to test this out as soon as possible, we can avoid the release of all the dependencies and replace them with workarounds:

Setting to change the entry point of the packages of a site
No package that we know of is using the /src/amp.js entry point so far. Also, this feature is required to optimize the React bundle size so it’s not critical.

Setting to change which bundles are generated (client and/or server)
Frontity can generate the client bundle for AMP. It won’t be used, but everything should work just fine.

Allow packages to modify the settings of a site
We don’t need this until the previous two features have been implemented.

Package priorities
We can add the package at the beginning of the packages array in the frontity.settings.js file and it will have the same effect.

We can also skip modifying/creating some of the processors and components and the frontity.config.js file for the beta.

Documentation

  1. I think for this feature it would be great to have a new section in our docs that explains how to use this package:

    • How to create a second site in your project
    • How to use match to use the site for some URLs, for example, those that end with /amp.
    • How to make AMP-aware components.
    • How to use Html2React to add custom AMP components.
    • How to use different entry points to avoid including extra KBs in the client bundles.
  2. A code example of a project using two sites, one configured for React and the other for AMP.

  3. Make both mars-theme and twentytwenty-theme AMP-aware.

Open Questions

Expose SSR/CSR tools in ctx.frontity or libraries

I’m not 100% convinced yet on exposing the objects/functions necessary for the SSR/CSR on the Koa context. Another option could be to use libraries for this.

For example, using the Koa context to change the template and the App:

export default {
  actions: {
    amp: {
      beforeSSR: () => ({ ctx }) => {
        ctx.frontity.template = myOwnTemplate;
        ctx.frontity.App = (props) => (
          <MyProvider>
            <ctx.frontity.App {...props} />
          </MyProvider>
        );
      },
    },
  },
};

Or just exposing the same under libraries. It could be libraries.frontity or something more specific like libraries.rendering.

export default {
  actions: {
    amp: {
      beforeSSR: ({ libraries }) => {
        libraries.frontity.template = myOwnTemplate;
        libraries.frontity.App = (props) => (
          <MyProvider>
            <libraries.frontity.App {...props} />
          </MyProvider>
        );
      },
    },
  },
};

Same for beforeCSR, where having a “Koa context” feels more weird.

EDIT: I have decided to go with libraries.frontity. I have updated the implementation proposal to reflect this decision.

Related Feature Discussions

These Feature Discussions also make use of the new frontity.config.js file.

References

1 Like

Wow! Thanks a lot for the detailed explanation, I think everything is much clearer now, awesome work :slightly_smiling_face:.

I really like the idea of creating a minimal version of the package and start testing it and receive some feedback before releasing the v1. I’m going to try to sum up what I understood to check if it makes sense to proceed this way, although I’m aware it may change:

First we have to work on some features that are mandatory for the initial version.

  1. Expose in the Koa context:
    • The HTML template .
    • The render function (currently renderToString ).
    • The App .
  2. Add a context to beforeCSR , similar to the context passed to beforeSSR .

Once that is done, we can start working on the first beta version of the @frontity/amp package:

  1. Overwrite the HTML template.
  2. Move all the CSS to the <head> .
  3. Change the render to be renderToStaticMarkup .
  4. Create some AMP processors (the main ones we select):
    • Maybe we can implement the ones from the old package.
    • Apart from the ones we implement, community users, if they need them, can do a Pull Request to create them if we explain how to do so.
    • Make the components and hooks (the ones we consider essential) of @frontity/components and @frontity/hooks AMP-aware.

At this point, we could release the v1 of the beta version and start testing it.

  1. We should remove /amp from link in actions.source.fetch and state.source.get to point to the canonical URL with the workaround you suggested until the hooks are implemented. I think this is pretty important as it seems the most common way of implementing amp, but it doesn’t make sense until all the previous steps have been made.

We could release a new version here.

  1. I would make the themes amp-aware at this point. Although I think most of the Frontity users that want AMP are going to build their own theme, it could be a good example on how to do it amp-aware. We should have a demo for this as well, so we could use mars.frontity.org or twentytwenty.frontity.org.
  2. Setting to change the entry point of the packages of a site.
  3. Setting to change which bundles are generated (client and/or server).
  4. Allow packages to modify the settings of a site.
  5. Add frontity.config.js file for setting the previous steps in the @frontity/amp package.

We could release a new version here.

  1. Support for package priorities (add priority to amp package).
1 Like

Yeah, I think that order of action is perfect :slightly_smiling_face:

Although it seems we’re not going to be able to include any of these in this sprint, this is the initial estimation we did:

  1. Expose the HTML template, the render function and the App in libraries - 5pt
  2. Add a context to beforeCSR , similar to the context passed to beforeSSR - 5pt
  3. Overwrite the HTML template - 5pt
  4. Move all the CSS to the <head> - 5pt
  5. Change the render to be renderToStaticMarkup - 3pt.

Here we could release the first beta version, which means 18pts.

  1. Create minimum amount of AMP processors (the main ones we select) - 13pt
  2. Remove /amp from link in actions.source.fetch and state.source.get - 3pt

We could do this little by little, there would be 16pt more.

  1. Make the themes amp-aware - 21pt.
  2. Setting to change the entry point of the packages of a site - 8pt.
  3. Setting to change which bundles are generated (client and/or server) - 8pt.
  4. Allow packages to modify the settings of a site. - 13pt.
  5. Add frontity.config.js file for setting the previous steps in the @frontity/amp package - 3pt.

There would be 53pt more until we make these optimizations. At this point it could even be the v1.

  1. Support for package priorities (add priority to amp package). - 8pt.

Ok, thanks @santosguillamot!

Some updates:

  1. I’ve decided we should go with libraries.frontity because:

    • Adding ctx to the client means adding a new concept, while people already know about libraries.
    • It is consistent with state.frontity where we expose data about the current site.
  2. I forgot to add that we should also update some packages to make them AMP-aware, like the analytics packages: @frontity/comscore-analytics, @frontity/google-analytics, @frontity/google-tag-manager--analytics.

  3. I have fixed the code snippet of the section Moving CSS to the <head>.

@nicholasio.oliveira, these is a summary of what needs to be done for the AMP package:

Frontity Core

AMP package

  • Modify the HTML template

    This should also be simple, as it’d be to use the init or beforeSSR action of the package to modify libraries.frontity.template to a function like this one: https://github.com/frontity/frontity/blob/dev/packages/core/src/server/templates/amp.ts

  • Change the render to be renderToStaticMarkup

    This is also trivial. Just using the same init or beforeSSR action to modify libraries.frontity.reder to the ReactDOM’s renderToStaticMarkup.

  • Move all the CSS to the <head>

    This is the trickiest part, but we have already done so with success, like I explained in the Move CSS to the section of the Implementation Proposal above. It shouldn’t take long as it’d mean we need to replicate what we already had in the very first version of Frontity.

With those features you should be able to render AMP sucessfully with a @frontity/amp package.

Extra things

  • Make your theme AMP aware

    Of course, you need to make sure your theme is AMP aware. The @frontity/amp package can create a state.amp object that React components can check to see if an amp package is present or not.

  • Create processors to make the post content AMP aware

    We did that in the past. They are not that many to be honest. For example, this was the processor and component for SoundCloud:

    More processors here: https://github.com/wp-pwa/h2r/tree/dev/src/processors

  • Remove /amp from the internal source actions

    If the URL needs to be https://domain.com/some-post/amp, we’d also need to remove the /amp from link in actions.source.fetch and state.source.get.

    We decided we are going to hardcode it in the v1 of source because it’s not possible to do it through the AMP package, although that will be solved in the v2 of source.

    This is not required if the URL is a subdomain https://amp.domain.com/some-post or a query https://domain.com/some-post?amp=true.

    It should not be hard to do, a couple of lines of code.

1 Like

It’d be interesting to see if it’d be possible to integrate Frontity with styling libraries that require SSR steps like Material-UI (SSR instructions) after exposing these parts.