@orballo suggested it’d be great to be able to capture SSR errors to display a more beautiful error, including the logo and a custom message.
I guess we can make sure that you can wrap the function that does the SSR with a middleware like:
app.use(({ ctx, next }) => {
const oldRender = ctx.libraries.frontity.render;
ctx.libraries.frontity.render = (...args) => {
try {
return oldRender(...args);
} catch (error) {
// Return my pretty HTML.
}
};
next();
});
@luisherranz I would like to understand the use cases here a little better before starting to research it further and so that we can come up with the API that’s appropriate for the problem.
This is the partial list so far:
I think it would make sense to show in the proposal how each of them could be solved with with the API that we are going to propose.
What other features are on the roadmap will require server extensibility?
I also saw that you’ve mentioned that:
Server extensibility will also allow other changes, like for example hooking into ReactDOM’s render or changing the HTML template. Everything that Frontity uses in the SSR will be exposed in a
ctx.frontityobject and passed to all the Koa middleware.
Could you elaborate on that?
Well, we are going to expose all the things that Frontity is using for the SSR part in libraries, and the Koa middleware will be able to access libraries, so you can use it to modify the SSR.
I think it won’t be strictly necessary as we are going to expose it also for beforeSSR, like explained in the AMP package FD.
So… if you want to add a provider for example (I think you bumped into this problem before) you can do it with either:
beforeSSR action:// my-package/src/index.js
const myPackage = {
actions: {
myPackage: {
beforeSSR: ({ libraries }) => {
const OldApp = libraries.frontity.App;
libraries.frontity.App = (props) => (
<MyProvider>
<OldApp {...props} />
</MyProvider>
);
},
},
},
};
// my-package/src/app.js
export default ({ app }) => {
app.use((ctx) => {
const OldApp = ctx.libraries.frontity.App;
ctx.libraries.frontity.App = (props) => (
<MyProvider>
<OldApp {...props} />
</MyProvider>
);
});
};
The same can be true for other parts, like the template, the React render function and so on…
Interested to work on this. Do we have a preferred way of exposing the server? I like the idea of a custom server export in the settings file.
One important thing in my mind is creating custom routes/api endpoints (which should be fine if we just expose app, passing ctx, as an example, this would allow optimizing images on-demand (like next/image).
@nicholasio.oliveira that is awesome! 
I plan to publish our ideas (a draft of the Implementation Proposal) at the beginning of next week to gather feedback.
I am more inclined now to do this in a separate entry point, like app.js.
The reason is that most of the times you will want to include Node-only packages:
fs, path and so on…Even though it is already possible to divide the index.js file into two different entry points: client.js and server.js, I think it’s better to do this in its own, separate entry point.
Other than that, I think it should be pretty straightforward. As I said, I will publish a draft of the Implementation Proposal next week and we can continue the conversation 
This is only a draft of the Implementation Proposal. It is still open for comments and suggestions.
Frontity uses a Koa server to handle the HTTP requests.
Right now, the server only handles the server-side rendering of React and a few other cases:
/static folder.favicon.ico or robots.txt.Koa uses middleware functions for configuration. To hook into the Koa server and the SSR process of Frontity, we need to add server extensibility, which means packages are allowed to create middleware functions for Koa and expose the objects/functions used by Frontity in the SSR process so they can be modified.
The functions of the Koa server, like this one:
app.use(async (ctx) => {
ctx.body = "Hello World";
});
ctx
The context that middleware functions receive. It is an abstraction on top of the req/res params of Node servers.
Full documentation can be found at https://koajs.com/.
For Node-only modules I mean:
fs, path and so on…We chose Koa instead of other frameworks, like Express, because:
ctx abstraction that it is easier to understand than the req/res objects of Node.next().The combination of async/await for the next(), which means that the middleware functions have the opportunity to execute twice (from first to last and then from last to first) and the fact that the middleware can never send or “close” the request with something like req.send() but instead rely on mutations of the ctx properties, make Koa middleware especially good for extensibility.
Frontity packages need to be able to hook into the Frontity server for different reasons:
<App> to add a new <Provider> for the whole app.Cache-Control headers.ads.txt./wp-content to WordPress.Right now users are forced to add their own NodeJS server on top of Frontity if they want to modify/control the requests, but that approach is not ideal as I explain in the Existing Solution section.
For the modification/control of the SSR process, we are already adding the different parts to the libraries.frontity namespace as I explain in the Existing Solution section below.
The main goal is that packages can expose middleware functions that:
Even though this is called “Server Extensibility”, these middleware functions run only on the HTTP requests. It is out of the scope the hooking and modification of the build process:
Those will be managed with code and/or configuration in the frontity.config.js files, which are planned to be run during the build process.
Right now people that need to hook into the NodeJS server is using the function exposed in build/server.js as a middleware for a NodeJS server which runs on top.
const frontity = require("./build/server.js").default;
const http = require("http");
const server = http.createServer(frontity);
server.listen(3000);
console.log("Listening on port 3000");
That way, they can add more behavior to the NodeJS server as they please. For example, change the headers of the static assets.
const addStaticHeaders = (req, res) => {
const matched = /^\/static\//.url.match(regex);
if (matched) res.setHeader("Cache-Control", "max-age=31536000");
};
But, this is not extensible. Other users of Frontity can’t add a package that adds this functionality and configure it in their frontity.settings.js file.
Also, if people use their own framework to control these middleware functions on top of the Frontity server, they are adding unnecessary layers because Frontity is already running a server powered by Koa.
There is an ongoing process of exposing the parts involved in the SSR in libraries.frontity as part of the AMP package, which means the SSR will be able to be modified from the init and beforeSSR actions.
For example, to modify the template you could do:
import myTemplate from "./my-template";
export default {
actions: {
myPackage: {
beforeSSR: ({ libraries }) => {
libraries.frontity.template = myTemplate;
},
},
},
};
The alternative using middleware functions would be something like:
import myTemplate from "./my-template";
export default {
server: {
myPackage: {
changeTemplate: ({ ctx }) => {
ctx.libraries.frontity.template = myTemplate;
},
},
},
};
Right now, I cannot think of anything regarding SSR that would be better managed in a middleware function as opposed to the init/beforeSSR methods or vice-versa.
Also, I don’t think we should restrict access to libraries.frontity in middleware functions, so I guess we will see over time which approach works better and/or if some things can only be done with one approach or the other.
There are different options for each part.
I am exposing all of them because I don’t have strong feelings for any of the approaches. I do have a current opinion, however, which I express later in this post.
index/client/server.js entry pointsThis means we reuse the same entry points that we already have for the React part and we simply add a new property to the store. We could call it server.
// packages/my-package/src/index.js
export default {
state: {
myPackage: {
//...
},
},
actions: {
myPackage: {
//...
},
},
server: {
myPackage: {
// Add static headers for the static assets (served at publicPath).
addStaticHeaders: ({ ctx }) => {
if (ctx.href.startsWith(ctx.frontity.publicPath))
ctx.set("Cache-Control", "max-age=31536000");
},
},
},
};
Pros:
server name, which is more intuitive than app for this use.Cons:
index.js cannot be used anymore and the user needs to divide it to client.js and server.js.The previous index.js example would need to be divided in client.js and server.js, like this:
// packages/my-package/src/client.js
export default {
state: {
myPackage: {
//...
},
},
actions: {
myPackage: {
//...
},
},
};
// packages/my-package/src/server.js
import myPackage from "./client";
import serve from "koa-static"; // Node-only package.
import { get } from "koa-route"; // Node-only package.
export default {
...myPackage,
server: {
myPackage: {
// Serve the ads.txt using the /public/ads.txt file.
adsTxt: get("/ads.txt", serve("./public"))),
},
},
};
By the way, the possibility of using the client/server.js entry points is already working in Frontity, we would not need to add it as a new feature.
If we go with the same entry points option (A1), we could use the same export than the rest of the properties (default) or we could use a separate one, like server for example.
export default {
state: {
// ...
},
actions: {
// ...
},
server: {
myPackage: {
// ...
},
},
};
Pros:
state, actions and libraries.export default {
state: {
// ...
},
actions: {
// ...
},
};
export const server = {
myPackage: {
// ...
},
};
Pros:
It is isolated, which may be cleaner when refactoring to client.js and server.js is required, doing this:
// client.js
export default {
state: {
// ...
},
actions: {
// ...
},
};
// server.js
export { default } from "./client";
export const server = {
myPackage: {
// ...
},
};
Cons:
So far we have managed to export a single type for a whole package. If we use a separate export, we would need to do two type exports.
This would not be possible:
import SomePackage from "some-package/types";
import OtherPackage from "other-package/types";
export type Packages = MergePackages<MyPackage, SomePackage, OtherPackage>;
And we would have to do something like:
import SomePackage, {
Server as SomePackageServer,
} from "some-packages/types";
import OtherPackage, {
Server as OtherPackageServer,
} from "some-packages/types";
export type Packages = MergePackages<
MyPackage,
SomePackage,
SomePackageServer,
OtherPackage,
OtherPackageServer
>;
app.js entry pointThe other option would be to use a new entry point, exclusive for the middleware functions, like app.js.
// packages/my-package/src/app.js
export default {
myPackage: {
// Add static headers for the static assets (served at publicPath).
addStaticHeaders: ({ ctx }) => {
if (ctx.href.startsWith(ctx.frontity.publicPath))
ctx.set("Cache-Control", "max-age=31536000");
},
},
};
Pros:
client/server.js, they will just learn to use app.js.Cons:
server.js because that name is already been used when you need to divide index.js in both client and server code, for the case when beforeSSR contains Node-only code, so this would be confusing:server.js supposed to be the file for the server??”
We can promote the use of namespaces as we do for other store properties.
export default {
state: {
myPackage: {
//...
},
},
actions: {
myPackage: {
//...
},
},
server: {
myPackage: {
// ...
},
},
};
Pros:
Cons:
Or we can just expose app and let packages add middleware as they please.
export const server = (app) => {
app.use(({ ctx }) => {
// ...
});
app.use(({ ctx }) => {
// ...
});
};
Pros:
Cons:
app or abstract app.use()
Koa works with an app instance that has a small API:
app.use(middlewareFunction): Add middleware functions to the server.app.callback(): Return a callback function suitable for the http.createServer() method to handle a request. You may also use this callback function to mount your Koa app in a Connect/Express app.app.listen(): Starts the server.app.env: The same value than process.env.NODE_ENV.app.proxy: Setting to trust proxy headers.app.subdomainOffset: Setting to ignore subdomains.app.keys: Set signed cookie keys.app.context: The contect ctx prototype. Useful to add things that can be accessed later in the middleware functions, like ctx.libraries.app.silent: Turn off error output to stderr.app.on(): Listen to events. For example listen to "error".From those, the main one packages will need to use 99% of the time is app.use().
These are not required at all:
app.callback()app.listen()app.envapp.proxy (this on will be turned on by default)And these may be useful at some point, but it is going to be quite rare.
app.subdomainOffsetapp.keysapp.contextapp.silentapp.on()app
We could expose the app itself and let packages use app.use() explicitly.
This could be the way to do so if we use the same entry points (A1) and namespaces (B1):
export default {
// ...
server: {
myPackage: {
myMiddlewares: (app) => {
// Configure app here:
app.silent = true;
app.context.db = new Database();
app.use((ctx) => {
// ...
});
},
},
},
};
And this could be the way to do so if we use a separate entry point (A2) and no namespaces (B2):
// packages/my-package/src/app.js
export default (app) => {
// Configure app here:
app.silent = true;
app.context.db = new Database();
app.use((ctx) => {
// ...
});
app.use((ctx) => {
// ...
});
};
Pros:
app and app.use
Cons:
We cannot pass ctx as part of an object, like this:
app.use(({ ctx }) => {
// ...
});
So if want to add more things, like the store, we would need to do so inside ctx.
app.use((ctx) => {
const { state, libraries } = ctx;
// ...
});
Destructuring is not straightforward when you need to mutate a prop of the context, like ctx.status or ctx.body, so this could be a bit confusing.
app.use(({ state, libraries, ...ctx }) => {
ctx.status = 200;
ctx.body = "Hi mum!";
});
As opposed to:
app.use(({ ctx, state, libraries }) => {
ctx.status = 200;
ctx.body = "Hi mum!";
});
app.use
We can accept funtions where we abstract app.use() away and find an alternative way to expose the app for the uses cases where packages want to hook in the other APIs/settings.
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx, next }) => {
// This is `app.use(ctx, next) => ...`
// but abstracted within the store.
},
},
},
};
Pros:
ctx in the first parameter, and maybe will simplify things if we ever add DevTools for this part of Frontity.Cons:
app.To expose the app, I think we have two options.
app in the ctx
We could simply expose app inside the middleware functions:
export default {
state: {},
actions: {},
server: {
myPackage:
myMiddleware: ({ ctx, app }) => {
// Configure app here:
app.silent = true;
app.on("error", () => {
// Do something...
});
},
},
},
};
I am not sure that app settings can be configured inside middleware functions, so if we choose this option we would have to check that out.
app in the package creationPackage modules are usually an object, but they can also be a function. That means we have the opportunity to pass app in that call:
export default ({ app }) => {
// Configure app here:
app.silent = true;
app.on("error", () => {
// Do something...
});
return {
state: {},
actions: {},
server: {},
};
};
app.use(ctx, next) or pass an object like ({ ctx, next })
app.use(ctx, next) APIWe can keep the exact same API than app.use() is using:
app.use((ctx, next) => {
const { state, libraries } = ctx;
// ...
});
This is required if we don’t abstract app.use (C1).
This would mean the store would be part of the ctx object.
export default {
// ...
server: {
myPackage: {
myMiddleware: (ctx) => {
// Access state, actions, libraries or even server.
const { state, server } = ctx;
if (state.frontity.name === "my-site") {
// Delete the middleware from other package...
delete server.otherPackage.someMiddleware;
}
},
},
},
};
Cons:
ctx.status or ctx.body, so this could be a bit confusing. This was explained previously in the C1 section.
({ ctx, next })
Or we can pass an object that contains ctx, next and maybe other things, like the store.
app.use(({ ctx, next, state, libraries }) => {
// ...
});
If we use namespaces, this would be:
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx, state, server }) => {
// Access state, actions, libraries and now even server.
if (state.frontity.name === "one-of-my-sites") {
delete server.otherPackage.someMiddleware;
ctx.status = 200;
ctx.body = "Hi mum!";
}
},
},
},
};
Cons:
There are a lot of packages for Koa packages and sometimes they are passed directly to the app.use() function, like this:
const ratelimit = require("koa-ratelimit");
export default {
// ...
server: {
myPackage: {
rateLimiter: ratelimit(options),
},
},
};
These packages expect to receive (ctx, next), so they won’t work with ({ ctx, next }) directly.
The way to use them would be this:
export default {
// ...
server: {
myPackage: {
rateLimiter: ({ ctx, next }) => ratelimit(options)(ctx, next),
},
},
};
Finally, we could expose both APIs because they are compatible with each other.
The trick is to do add a reference to the ctx in the ctx itself. Something like this:
const middlewareWrapper = (middlewareFunction) => {
app.use((ctx, next) =>
middlewareFunction(
{
ctx, // Pass ctx.
next, // Pass next.
...store, // Pass state, actions, libraries...
...ctx, // And also spread the ctx itself to copy the original API.
},
next // Pass next as second argument to copy the original API.
)
);
};
I don’t know if genearting a new ctx object is necessary, so this is just an example. I have assumed it is because the reference to the next object needs to be kept in the function’s closure. It will depend on how Koa builds the ctx object that passes to the middleware functions.
Then, these two options are possible:
app.use(({ ctx, next, state, libraries }) => {
// ...
});
app.use(ratelimit(options));
If we use namespaces, this would be:
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx, state }) => {
// Do stuff with ctx, state and so on...
},
rateLimiter: ratelimit({
// Add koa packages directly.
}),
},
},
};
Pros:
Cons:
ctx repeated: ctx.ctx, ctx.body, ctx.ctx.body and so on…get with serve, this will work:export default {
// ...
server: {
myPackage: {
serveSomeStatics: get("/public/:file", serve("./public")),
},
},
};
But if you want to use get for your own logic, you may end up back to the app.use(ctx, next) API, because that is the standard in Koa and therefore what get accepts in its callback:
export default {
// ...
server: {
robots: {
serveRobotsTxT: get("/robots.txt", async (ctx, next) => {
// Serve the robots.txt found in the root.
if (await promisify(exists)("./robots.txt")) {
await serve("./")(ctx, next);
} else {
// Output a default robots.txt.
ctx.type = "text/plain";
ctx.body = "User-agent: *\nAllow: /";
}
}),
},
},
};
So if you need access to the store, you need to access it through ctx.
export default {
// ...
server: {
robots: {
serveRobotsTxT: get("/robots.txt", async (ctx, next) => {
ctx.type = "text/plain";
if (await promisify(exists)("./robots.txt")) {
// Serve the robots.txt found in the root.
await serve("./")(ctx, next);
} else if (ctx.state.robots.txt) {
// Output the robots.txt that is stored in the state.
ctx.body = ctx.state.robots.txt;
} else {
// Output the default robots.txt.
ctx.body = "User-agent: *\nAllow: /";
}
}),
},
},
};
libraries or ctx
We have two options when it comes to enabling middleware functions to share utils with each other.
libraries
Expose utils in libraries, like the way we do so in the React side.
// Other Package.
export default {
// ...
libraries: {
otherPackage: {
processHeaders: (headers) => {
// Do stuff...
},
},
},
};
And consume them in the middleware functions.
// My Package.
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx, libraries }) => {
// Use utilities from other packages.
libraries.otherPackage.processHeaders(ctx.headers);
},
},
},
};
Pros:
ctx
Expose utils in ctx. This requires one of the methods to expose app to be able to access app.conext.
// Other Package.
export default ({ app }) => {
app.context.processHeaders: (headers) => {
// Do stuff...
};
};
Consume them in the middleware functions.
// My Package.
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx }) => {
// Use utilities from other packages.
ctx.processHeaders(ctx.headers);
},
},
},
};
Pros:
Cons:
libraries which are also exposed to the React app anyway, so that would mean we have two ways to do the same thing.Before writing the whole Implementation Proposal Draft I was more inclined towards using a separate entry point, like app.js, and no namespaces.
Now, after seeing the whole picture together, I think it makes more sense to keep this as close to the React app as possible, and fulfill these design principles:
So I would go with:
default export (because of 1.)
app.use() (because of 3.)
app in the package creation (because of 3.)
(ctx, next) and ({ ctx, state }) APIs (because of 1. and 3.)
libraries (because of 1.)
This approach still has an inconsistency with another API: the beforeSSR action when it needs to use ctx.
Right now that API is:
export default {
actions: {
myPackage: {
beforeSSR: ({ state }) => ({ ctx }) => {
// ...
},
},
},
server: {
myPackage: {
someMiddleware: ({ state, ctx }) => {
// ...
},
},
},
};
To make it consistent with the middleware functions, we would have to migrate it to:
export default {
actions: {
myPackage: {
beforeSSR: ({ state, ctx }) => {
// ...
},
},
server: {
myPackage: {
someMiddleware: ({ state, ctx }) => {
// ...
},
},
},
},
};
Thankfully, this can be done with backward compatibility.
Another option would be to deprecate this beforeSSR option altogether and move the logic that needs to deal with ctx to middleware functions. After all, we passed ctx to beforeSSR as an easy fix because we didn’t have server extensibility yet.
All feedback is welcomed so please go ahead and share yours
Once we make a decision about the final Implementation Proposal, I will finish it adding:
@SantosGuillamot I have used user stories because that is what we still have in the template, but if you want to turn them into regular goals let me know 
I updated yesterday this template. Did you use that one or there is another template in another place?
I think that with the user stories is clear, so we can keep them in this case. The purpose of changing it was to make it easier to write the Implementation Proposal and make the functionalities clearer.
Btw, I loved the explanation of the different possibilities, it makes it much easier to understand. I am not used to work with this, but overall it seems what you suggest makes sense, and it is based on the design principles. I just have a couple of comments:
app.js, as we’re already using server.js, would be pretty confusing. Apart from that, if I would have to use Node-only libraries, it makes sense to me to add them only in the server.js file, although we can’t use index.js anymore.ctx repeated. But it’s true that D2 reduces extensibility and D1 is not straightforward. I never work with this kind of things so I have no idea what is better from a developer experience, so I trust your opinion here as well .Oh, I started on Monday so I used the old copy then 
I am going to rewrite the goals more as “features” to see how they would look like.
EDIT: Done.
EDIT: No need to read, we are going to start a new Koa application for each request. You can jump here: Server Extensibility
I’m working on the Final Implementation Proposal for this but I stumbled upon a problem that I wanted to share.
We need to dynamically populate the middleware of the application depending on the packages of each site because the Frontity server can handle requests for multiple sites and sites can use different packages.
We need to find a way to isolate each request from each other. I’ve been reading Koa’s code and it doesn’t seem like that is the case, so we could have leakage from the requests to one site to the requests of another.
I’ve made a video to explain it a bit better: https://www.loom.com/share/206b60c83319451c96d32b75b2675e70
I’ll keep thinking about it but any help/feedback is welcomed
Well, maybe it was not that hard to solve. I think we can use a second compose passing an array created inside the first middleware. Each compose then would start the dispatch(i) from 0 and each array is unique to each request. It will go a bit like this:
On app creation:
const app = new Koa();
app.use(init); // same than app.middleware = [init]`
On each request:
const init = (ctx, next) => {
const isolatedMiddleware = [pkgA, pkgB, SSR];
ctx.app.middleware[1] = compose(isolatedMiddleware);
next();
};
dispatch(0): the init middleware.init creates and populates an internal array: isolatedMiddleware.init adds it as second item of app.middleware using compose().init runs next().dispatch(1): compose(isolatedMiddleware).compose runs dispatch(0): pkgA.
pkgA runs next().compose runs dispatch(1): pkgB.pkgB runs next().compose runs dispatch(2): SSR.SSR runs next().compose runs next().dispatch(2).app.middleware to replace the item 1 with its own internal array.It is also explained here: https://www.loom.com/share/32418c44d5e9487c9aff342e21a46e96
It is probably not the best approach so open to suggestions but at least we have one working approach
I will continue with the rest of the IP.
EDIT: By the way, another option would be to create a new Koa application for each request. I’ve been trying to avoid that to make the application as performant as possible but it is true that Koa itself is incredibly lightweight. So that could be a much simpler option.
Ok, after a bit more thought it is clear to me that we need to go with the new Koa app per request because there are other things that could be leaked from one site to another, not only middleware, like event handlers or options:
app.silent = false;
app.on("error", (err) => {
log.error("server error", err);
});
So we need an isolated app instance per request.
And after reading Koa’s code I am not worried about that performance cost anymore because it will be orders of magnitude smaller than the real bottleneck of this type of application, which is the data fetching.
All good then, sorry for the trouble 
Nice one Luis!
In the past I’ve used the same approach with multiple Koa instances per website. In that case where multiple brands but same technique. But as you said the perf bottleneck wasn’t the multiple Koa instances but the data layer .
Nice to hear. Thanks Cris
This is the Final Implementation Proposal.
The Final Implementation will be written once all the tasks are completed and the feature is released.
Frontity uses a Koa server to handle the HTTP requests.
Right now, the server only handles the server-side rendering of React and a few other cases:
/static folder.favicon.ico or robots.txt.Koa uses middleware functions for configuration. To hook into the Koa server and the SSR process of Frontity, we need to add server extensibility, which means packages are allowed to create middleware functions for Koa and expose the objects/functions used by Frontity in the SSR process so they can be modified.
The functions of the Koa server, like this one:
app.use(async (ctx) => {
ctx.body = "Hello World";
});
ctx
The context that middleware functions receive. It is an abstraction on top of the req/res params of Node servers.
Full documentation can be found at https://koajs.com/.
For Node-only modules I mean:
fs, path, and so on…We chose Koa instead of other frameworks, like Express, because:
ctx abstraction that it is easier to understand than the req/res objects of Node.await next().The usage of async/await for the next function makes Koa middleware especially interesting for extensibility because it means that the middleware functions have the opportunity to execute twice (from first to last and then from last to first) and that the middleware won’t “close” the request with something like req.send() but instead rely on mutations of the ctx properties like ctx.body = "...".
Frontity packages need to be able to hook into the Frontity server for different reasons:
<App> to add a new <Provider> for the whole app.Cache-Control headers.ads.txt./wp-content to WordPress.fetch requests./frontity/cache/purge.state within the HTML file and serve it using a different endpoint, like /some-post/?frontity_state=true.Right now users are forced to add their own NodeJS server on top of Frontity if they want to modify/control the requests, but that approach is not ideal as I explain in the Existing Solution section.
For the modification/control of the SSR process, we are already adding the different parts to the libraries.frontity namespace as I explain in the Existing Solution section below.
The main goal is that packages can expose middleware functions that:
state (frontity.settings.js).Even though this is called “Server Extensibility”, these middleware functions run only on HTTP requests. It is out of the scope the hooking and modification of the build process:
Those will be managed with code and/or configuration in the frontity.config.js files, which are planned to be run during the build process.
Right now people that need to hook into the NodeJS server are using the function exposed in build/server.js as a middleware for a NodeJS server that runs on top.
const frontity = require("./build/server.js").default;
const http = require("http");
const server = http.createServer(frontity);
server.listen(3000);
console.log("Listening on port 3000");
That way, they can add more behavior to the NodeJS server as they please. For example, change the headers of the static assets.
const addStaticHeaders = (req, res) => {
const matched = /^\/static\//.url.match(regex);
if (matched) res.setHeader("Cache-Control", "max-age=31536000");
};
But, this is not extensible. Other users of Frontity can’t add a package that adds this functionality and configure it in their frontity.settings.js file.
Also, if people use their own framework to control these middleware functions on top of the Frontity server, they are adding unnecessary layers because Frontity is already running a server powered by Koa.
There is an ongoing process of exposing the parts involved in the SSR in libraries.frontity as part of the AMP package, which means the SSR will be able to be modified from the init, beforeSSR and afterSSR actions.
For example, to modify the HTML template you could do:
import myTemplate from "./my-template";
export default {
actions: {
myPackage: {
beforeSSR: ({ libraries }) => {
libraries.frontity.template = myTemplate;
},
},
},
};
The alternative middleware function would be something like:
import myTemplate from "./my-template";
export default {
server: {
myPackage: {
changeTemplate: ({ libraries }) => {
libraries.frontity.template = myTemplate;
},
},
},
};
I haven’t found any SSR modification that would be better managed in a middleware function as opposed to the init/beforeSSR/afterSSR methods or vice-versa.
Despite that, I think we should still expose ctx to the actions because they run at different times and only for React’s SSR, so they may be simpler to use in some cases.
Once we finish this feature, the execution order of React’s SSR will be:
next()).init actions.beforeSSR actions.afterSSR actions.state snapshot.next()).So, for example, if you want to modify the Cache-Control headers based on the content of each URL, the afterSSR action is probably the best one because it only runs for React’s SSR:
export default {
actions: {
myPackage: {
afterSSR: ({ ctx, state }) => {
const data = state.source.get(state.router.link);
// Check if this is a post.
if (data.isPost) {
// Check how old it is, in days.
const post = state.source.post[data.id];
const now = new Date();
const postDate = new Date(post.date);
const postAge = (now - postDate) / (1000 * 60 * 60 * 24);
if (postAge > 365) {
// If the post is older than a year, cache it during a month.
ctx.set("Cache-Control", `max-age=${60 * 60 * 24 * 30}`);
} else if (postAge > 30) {
// If post is older than a month, cache it during a week.
ctx.set("Cache-Control", `max-age=${60 * 60 * 24 * 7}`);
} else {
// If none of the above, cache it during an hour.
ctx.set("Cache-Control", `max-age=${60 * 60}`);
}
}
},
},
},
};
It is true that we could also use a middleware function, although it is less straightforward:
export default {
server: {
myPackage: {
changeCacheControl: async ({ ctx, state, next }) => {
// Wait until the SSR has finished.
await next();
// Check that we are in React's SSR because we don't want to
// change the Cache-Control headers for other responses.
if (!!state.router.link) {
// Change the Cache-Control header. Same logic as above.
// ...
}
},
},
},
};
We have decided to go with the following options.
index/client/server.js entry pointsapp.usefn(ctx, next) and fn({ ctx, next }) signaturesapp in the ctxlibrariesThis means we will reuse the same entry points and export that we already have for the React part and we will simply add the property server to the store. We will also use namespaces.
// packages/my-package/src/index.js
export default {
state: {
myPackage: {
//...
},
},
actions: {
myPackage: {
//...
},
},
server: {
myPackage: {
// Change static headers for the static assets (served at publicPath).
addStaticHeaders: ({ ctx, next }) => {
if (ctx.href.startsWith(ctx.frontity.publicPath))
ctx.set("Cache-Control", "max-age=31536000");
next();
},
},
},
};
When Node-only libraries are needed, index.js users will need to divide their entry points into client.js and server.js, like this:
// packages/my-package/src/client.js
export default {
state: {
myPackage: {
//...
},
},
actions: {
myPackage: {
//...
},
},
};
// packages/my-package/src/server.js
import myPackage from "./client";
import serve from "koa-static"; // Node-only package.
import { get } from "koa-route"; // Node-only package.
export default {
...myPackage,
server: {
myPackage: {
// Serve the ads.txt using the /public/ads.txt file.
adsTxt: get("/ads.txt", serve("./public"))),
},
},
};
The possibility of using the client/server.js entry points is already working in Frontity, we don’t need to add it as a new feature.
We need to create a new Koa() application on each request to make sure that packages from different sites are not leaked between requests.
I didn’t notice this before, but app is already part of the context in ctx.app, so we don’t need to do anything for that one either:
export default {
server: {
myPackage:
myMiddleware: ({ ctx }) => {
// Configure app here:
ctx.app.silent = true;
ctx.app.on("error", () => {
// Do something...
});
},
},
},
};
The middleware functions will accept both the fn(ctx, next) and fn({ ctx, next }) signatures. In the second case, the object received in the first argument will contain:
ctxnextstatelibrariesactionsrootsserverSo we need to add those properties to the ctx. Something like this:
Object.assign(ctx, {
ctx,
...store,
});
export default {
server: {
myPackage:
myMiddleware: ({ ctx, state, libraries, actions, roots, server }) => {
// ...
},
},
},
};
Adding next is a bit trickier because it needs to be different for each middleware call.
I’ve been reading Koa’s code and it is very simple and straightforward. The middleware functions are run one after another, so replacing ctx.next within a wrapper before calling the middleware seems to work fine.
const wrapper = (middleware) => (ctx, next) => {
ctx.next = next;
return middleware(ctx, next);
};
These options will be possible:
export default {
// ...
server: {
myPackage: {
someMiddleware: ({ ctx, state, libraries, next }) => {
// Some logic...
},
rateLimiter: ratelimit({
/* options... */
}),
serveSomeStatics: get("/public/:file", serve("./public")),
serveRobotsTxT: get("/robots.txt", async ({ ctx, next, state }) => {
ctx.type = "text/plain";
if (await exists("./robots.txt")) {
// Serve the robots.txt found in the root.
return serve("./")(ctx, next);
} else if (!!state.robots.txt) {
// Use info defined in the settings.
ctx.body = state.robots.txt;
} else {
// Output a default robots.txt.
ctx.body = "User-agent: *\nAllow: /";
}
}),
},
},
};
We will expose utils using libraries, as we do in the React app.
// Other Package.
export default {
// ...
libraries: {
otherPackage: {
processHeaders: (headers) => {
// Do stuff...
},
},
},
};
And consume them in the middleware functions.
// My Package.
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx, libraries }) => {
// Use utilities from other packages.
libraries.otherPackage.processHeaders(ctx.headers);
},
},
},
};
This middleware functions API is inconsistent with how the ctx is passed to the init/beforeSSR/afterSSR actions.
Right now the server actions API is:
export default {
actions: {
myPackage: {
beforeSSR: ({ state }) => ({ ctx }) => {
// ...
},
},
},
};
But we want the new middleware API to be:
export default {
server: {
myPackage: {
someMiddleware: ({ state, ctx }) => {
// ...
},
},
},
};
So we have to change the server actions API to this:
export default {
actions: {
myPackage: {
beforeSSR: ({ state, ctx }) => {
// ...
},
},
},
};
We need to do this with backward compatibility.
server property of their default export.fn(ctx, next) and fn({ ctx, next }) signatures must be valid.server and roots).Apart from the normal unit/e2e test, we must make sure that there is no leakage between sites with different packages.
There are no dependencies and this work could start right away.
Things that are mentioned in the IP but don’t require a task because they are already working:
src/client.js and src/server.js entry points to be able to import Node-only libraries.ctx.app.List of tasks:
Koa application per request.server) is properly merged in the creation of the store.ctx with the ctx itself and the store.middleware array with the correct packages.ctx.next.Koa application per requestTo make sure that we don’t leak middleware and app configuration from one request to another, my proposal is to create a new Koa application in each request.
Right now this is our @frontity/core/src/server export:
const server = ({ packages }) => {
// Create new app.
const app = new Koa();
// Configure the app...
app.use(/* ... */);
// Return the req/res function.
return app.callback();
};
We should switch to something like this:
const server = ({ packages }) => (req, res) => {
// Create a new app for each request.
const app = new Koa();
// Configure the app...
app.use(/* ... */);
// Return the final response.
return app.callback()(req, res);
};
Dependencies: none.
Relevant code:
server) is properly merged in the creation of the storeI think this should work almost out of the box, but we have to make sure that the server middleware is populated in the store.
This is where the merge is happening right now. At least we need to add the server prop to the initialization:
let config: Package = {
roots: {},
state: {},
actions: {},
libraries: {},
server: {},
};
Also, we need to make sure that if a package wants to overwrite the middleware of another package, it can do so by exporting the new version with the same namespace and name, in the same way that people can overwrite state and actions now.
// frontity.settings.js
const settings = {
name: "my-site",
packages: ["some-package", "my-custom-package"],
};
// some-package
export default {
server: {
somePackage: {
someMiddleware: ({ state, ctx }) => {
// Original implementation.
},
},
},
};
// my-custom-package
export default {
server: {
somePackage: {
someMiddleware: ({ ctx }) => {
// My custom overwrite!
},
},
},
};
Again, that should work out of the box.
Dependencies: None
Relevant code:
ctx with the ctx itself and the storeIn the initialization middleware, we should get the settings, create the store and populate the ctx before we run the middleware.
Something like this:
const server = ({ packages }) => (req, res) => {
// Create a new app for each request.
const app = new Koa();
// Initialization.
app.use((ctx, next) => {
// ...
const settings = getSettings(/* ... */);
const store = createStore(settings, packages /* ... */);
// Populate the `ctx`.
Object.assign(ctx, {
// Populate the context with itself, for `fn({ ctx })`.
ctx,
// Populate the context with the store, for `fn({ state... })`.
...store,
});
});
// Return the final response.
return app.callback()(req, res);
};
Dependencies:
None.
Relevant code:
middleware array with the correct packagesIn the initialization middleware, we should add the middleware exposed by the packages to the app.middleware array, using app.use().
The core middleware that needs to run after the package middleware (like the React SSR, the statics folder and so on) should be added dynamically after the package middleware to make sure it runs in the last place.
const server = ({ packages }) => (req, res) => {
// Create a new app for each request.
const app = new Koa();
// Initialization.
app.use((ctx, next) => {
// ...
const settings = getSettings(/* ... */);
const store = createStore(settings, packages /* ... */);
Object.assign(ctx, { ctx, ...store });
// Add package middleware.
const middleware = getMiddlwareArray(store.server);
middleware.forEach(app.use);
// Add core middleware.
app.use(staticFolder);
app.use(robotsTxt);
app.use(serverSideRendering);
// ...
});
return app.callback()(req, res);
};
It’s worth noting here that by extracting the middleware from the store, packages won’t be able to overwrite middleware from other packages on-the-fly. We could solve this by using a wrapper that retains the reference to the namespace and the name of the middleware (similar to target and key in proxies) but I don’t think it’s worth the complexity.
If a package really needs to overwrite the middleware of another package on the fly, they can search for the middleware function in the app.middleware array.
export default {
server: {
somePackage: {
someMiddleware: ({ app, server }) => {
const fnRef = server.someNamespace.otherMiddleware;
const index = app.middleware.findIndex(fnRef);
app.middleware[index] = () => {
/* New logic */
};
},
},
},
};
Dependencies:
server) is properly merged in the creation of the store.Relevant code:
ctx.next
The next step would be to create a wrapper to populate ctx.next so the middleware can work with the signature fn({ ctx, next }).
I’ve been reading the Koa’s code and I think this simple wrapper should be enough:
const wrapper = (fn) => (ctx, next) => {
ctx.next = next;
return fn(ctx, next);
};
Dependencies:
middleware array with the correct packages.Relevant code:
Finally, we should change the API of the server actions from this:
export default {
actions: {
myPackage: {
beforeSSR: () => ({ ctx }) => {
// ...
},
},
},
};
To this:
export default {
actions: {
myPackage: {
beforeSSR: ({ ctx }) => {
// ...
},
},
},
};
We need to do so while maintaining backward compatibility. I’ve done a video to explain one possible solution:
We should expose converToAction and the raw action in actions.namespace.actionName.raw for example and use those two to call the init/beforeSSR/afterSSR actions like this:
converToAction(beforeSSR.raw, { ...store, ctx })();
By the way, right now we are not passing ctx to the init actions, we should fix that as well.
Dependencies: None.
Relevant code:
This will require a whole new section in the docs, inside the Core Concepts.
We need to find a way to allow packages to “skip React SSR” so they can return other responses for URLs that should not be managed by React.
My current idea is that, if a package middleware populates ctx.body, it means that the response is not managed by React and we should skip the React SSR.
I’ll keep thinking about it but any feedback is welcomed
All feedback is welcomed so please go ahead and share yours
@nicholasio.oliveira: it would be especially great to hear your impressions about both the IP and the list of tasks.
If I am not mistaken @cristianbote is currently refactoring the core middleware. Cristian, could you please leave a message with the status of that refactoring? Thanks
Hi @luisherranz, great IP! 
This will make Frontity amazingly extensible.
Order of middleware execution.
While reading through the details noticed that there is no guarantee of the order of middleware execution. Do you think this is important? Would this be something that folks would expect?
Escape hatch by exposing only the app.
While having a server property it’s pretty explicit and it makes a clear separation between what part of the app it’s involving, have you thought about having this kind of api? This practically will be called after the beforeSSR.
export default {
server: {
myPackage: ({ app }) => {
// Sample of a middleware code
app.use((ctx, next) => {
const store = ctx.store;
// At this point the `store` is available and it can do anything it wants with it.
});
},
},
};
Open Questions: We need to find a way to allow packages to “skip React SSR” so they can return other responses for URLs that should not be managed by React.
If we would allow the api at point #2 this will turn into simply handle the response on the server. Like this: https://codesandbox.io/s/bold-feistel-4og51?file=/index.js
Example:
export default {
server: {
myPackage: ({ app }) => {
// Custom `/api` endpoint
app.use((ctx, next) => {
if (ctx.url.startsWith('/api')) {
// Serialize the state, or other API responses, auth
ctx.body = JSON.stringify(ctx.store.state, null, 2);
return;
}
await next();
});
},
},
};
If I am not mistaken @cristianbote is currently refactoring the core middleware. Cristian, could you please leave a message with the status of that refactoring? Thanks
Yup! I am done with it. The changes can be seen here: https://github.com/frontity/frontity/pull/687/files#diff-2aa5d096a9b58e6ba83f934e3f72bcff6ab4f27ca5cfe05dde74fe087ab53df5R93-R100.
At this point the store and settings are exposed on ctx.state but it’s not something that my implementation relies on so we can redefine those at will.
I am confident that allowing and exposing a function instead of a dictionary with middlewares, will empower the developers to extend at will the server as they wish.
What do you think @luisherranz? Did something slipped through my mind?
Our plans for this were to add priorities to packages. There is a small description on the AMP Implementation Proposal: AMP package - #10 by luisherranz
What I am wondering lately is at which level we should add priorities:
For some of the things, like Html2React processors and Source handlers, we have defined them at a function level. It’s more complex, much also more versatile.
WordPress also has them at a function level for actions (add_action) and filters (add_filter).
We should start a conversation about this
The code you mention will work because app is exposed in ctx.
However, I think our recommendation should be to use a normal Frontity middleware, as described in this IP, and await for next() when you need to run something after the SSR.
You are right. Most of the time, when the rendering should be skipped, it would be enough to not call next().
I wonder if there will be sometimes when skipping the React SSR AND keep executing the rest of the middleware (from other packages) will be necessary.
I guess it’s better to wait until we see more real usage of this
By the way, the API you describe is not required for this. "Not calling next()" is possible with the normal Frontity middleware functions described in this IP.
Awesome, thanks!
Yes, we will have to change that when we work in this implementation.
If you read my IP draft, that was one of the options explained there (section C). It included the pros and cons of not only that approach but also the alternatives: Server Extensibility - #13 by luisherranz.
It’s not too late to change some parts of the IP, so if you have a strong opinion about that specific part or any other, or you want to point out additional pros/cons of each approach that were not reflected in that draft, please do so
Awesome IP @luisherranz 
I’m sure there’s a reason that I missed here, but why do we want to support both signatures?
By the way, good points @cristianbote I also read the Final Implementation first and didn’t realize that priorities were discussed elsewhere and exposing the app was in the draft IP 
I have just realized there is another important reason to use namespaces that I forgot to add to the Proposal Implementation Draft (section B1): The DevTools.
If we use namespaces, we will have much better information for the server DevTools in the future, because after we run each middleware we can report it to the DevTools, the same way we will do with actions:
server.somePackage.some
state.xxx.counter: 1 → 2actions.zzz.toggle(true).
state.zzz.other: false → trueserver.otherPackage.some
Because one is “the Frontity way” and the other is needed to reuse Koa middleware. It is explained in the draft, section D3.
import ratelimit from "koa-ratelimit";
export default {
// ...
server: {
myPackage: {
myMiddleware: ({ ctx, state }) => {
// Do stuff with ctx, state and so on...
},
rateLimiter: ratelimit({
// Add koa packages directly.
}),
},
},
};