Repositories & Projects Organization

Ok, I’ve been figuring out how to organize our codebase and looking at how other developers do this. Here is a bunch of thoughts I had.

Projects

  • frontity (monorepo)
    In this repository, we should add all the officially maintained projects, except for wp plugins (only JavaScript projects). The repo will include:

    • core
    • sources (wp-org & wp-com)
    • routers (1d-router, 3d-router, etc.)
    • settings (file-settings, etc.)
  • wp-org plugins
    These plugins will be some utilities for the WordPress.org REST API. It could be a monorepo as well but I don’t know if it is better in this case. I prefer to start developing each plugin in its own repository.

  • wp-org themes
    The same as before but with WordPress themes.

  • examples & tutorials
    These examples are intended to show how to use frontity in a project, so it makes sense to maintain a different repository for each example, right?

  • frontity.org webpage
    The repository for the organization’s webpage.

Why to use a monorepo?

Extracted from Babel docs

Pros:

  • Single lint, build, test and release process.
  • Single place to report issues.
  • Easier to setup a development environment.
  • Easy to coordinate changes across modules.
  • Tests across modules are run together which finds bugs that touch multiple modules easier.

Cons:

How can we do it?

Currently, we can implement a monorepo in two different ways:

  • Using Lerna - https://lernajs.io/
    This is the tool most people are using. It seems that Lerna can be used with semantic-release and conventional-changelog and that would be awesome! Here is a tutorial on how to do that.

  • Using yarn workspaces and multi-semantic-release
    We could adopt this implementation and wait for semantic-release to add support to monorepos in the meantime. Here is an issue to follow their progress on that.

People using a monorepo

Without Lerna:

With Lerna:

People not using a monorepo

1 Like

Great job! :muscle: :frontity:

I have a couple of notes/questions.

Maybe in this case is also useful to have a monorepo. I think that in the case we have a Frontity plugin that includes other minor plugins, it could be useful to have them all together? However, I don’t know how managing a monorepo of PHP modules might differ from managing one of JS modules.

I think the number of examples might be quite large (different integrations, routers, starter themes, plugins…) and it might result in a bit of a mess to have them all in their individual repos.

But I don’t know if it would be better to have a monorepo with all the examples or maybe include them within the related modules in an examples directory (it might be where developers expect to find them).

@David, what’s the difference between lerna and semantic release and yarn workspaces and multi-semantic release? I took a quick glance at the lerna tutorial you’ve posted and it seems that each package keeps its own version! So it may be the best option, right?

wp-org plugins: yes, maybe they should be in one repo. There’s going to be shared code between the Frontity plugin and the individual ones.

examples: yep, examples are usually found in the main repo if they are simple. Like this: https://github.com/zeit/next.js/tree/canary/examples
If they are not so simple maybe we’ll need additional repos. We’ll see :slight_smile:

Yes, @luisherranz, I think so. The two options work exactly the same. Even Lerna with semantic-release bumps the version of those packages that don’t have any change but depend on updated packages!

multi-semantic-release, on the other hand, is a hacky implementation over semantic-release and, as his author said, it “may not be fundamentally stable enough for important production use” and “it may break or get out-of-date in future versions of semantic-release”.

Uhm, I’ve taken a look at the tutorial and I haven’t seen where semantic-release is being used. It seems Lerna is responsible of doing the “semantic release” but I guess Lerna can be configured in the same way.

Perfect, let’s use Lerna with conventional commits support as explained in the post you shared:


We still need to decide if these packages will have their own repo or not:

  • Official npm packages
    Like html2react, gutenberg-to-react, 1d-router…
  • Official extensions
    Like wp-org-comments, adsense-ads, wp-com-source…
  • Official theme starters
    Simple themes that can be cloned so people have a starting point. Like Gatsby starters.
  • Usage examples
    How to use Frontity in some common scenarios. Like Next examples.
  • The frontity.org theme
    The frontity theme powering the frontity.org website.
  • The Frontity WP Plugin and all its child plugins
    Both the official Frontity WP Plugin and the related plugins we want to release separately.

My vote goes for: every npm package goes into frontity/frontity except starter themes:

frontity/frontity
├── examples // usage examples 
└── packages/
    ├── frontity // frontity - cli and entry point to other packages
    ├── core // @frontity/core - main package with server & webpack...
    ├── html2react // @frontity/html2react
    ├── wp-org-comments // @frontity/wp-org-comments
    ├── ...
    └── wp-org-source // @frontity/wp-org-source

The WP plugins have their own monorepo. I’m not sure if it should be called wp-org-plugins instead of wp-plugins because I don’t know if wp-com plugins are different. Maybe they are.

frontity/wp-org-plugins
└── packages/
    ├── frontity // The official Frontity plugin
    ├── html-purifier
    ├── url-discovery
    ├── expose-custom-fields
    ├── ...
    └── content-attachment-ids

I’d use a different repo for each starter theme and a different repo for our frontity.org theme.

frontity/frontity-org-theme

I completely agree with your proposal, @luisherranz. I would probably name the frontity package as frontity-cli though, or do you think simply frontity is the best choice?

I was about to write about this. When users use npx instead of installing the package as global, the name of the package is used.

I’ve been taking a look at Gatsby and they have both gatsby and gastby-cli. When users use npx gatsby they are installing gatsby. In that package, bin points to gatsby-cli:

This means users have to wait until the the full gatsby package is installed without any benefit, and it has a lot of dependencies: babel, webpack… a lot of stuff not needed by the cli.

  • npx gatsby-cli takes 4 seconds.
  • npx gatsby takes 25 seconds.

Subsequent executions take the same amount of time.


I’d prefer that we use npx frontity because it’s easier to remember. People in the npm world is used to xxx-cli packages but in the WP world they are not.

My proposal is to use frontity as our cli and @frontity/core as the webpack/babel package.

I also have another proposal: use frontity to interface other libraries.

But it’s just an interface, the frontity package will only have the cli code so it installs as fast as possible when users do npx frontity.

The only thing I don’t like about this frontity and @frontity/core configuration is that it is harder to explain/understand when just looking at the code in github.

Things we need:

  • eslint
  • tslint and tsconfig
  • travis (maybe travis can do the releases automatically?)
  • a first release with its tags

Can we publish dist-tags with lerna? (like alpha, beta… and so on)

Yes, we can. What I’m not sure about is if Lerna can track changes in a specific branch and publish them automatically with a specific dist-tag.

By the way, maybe we could use Travis to do that job. I mean, if the name of the branch is next then execute npm run release:next, or something like that.

I’ve just noticed I published the 1.0.0 version of frontity in npm :sweat:: https://www.npmjs.com/package/frontity

That means we need to start from 1.0.0 or work with an alpha/beta until we have the 1.0.0 (1.0.1!) version.

Oops! :see_no_evil:

Maybe we could contact NPM and try to convince them to remove it. There is no one using it!

It’s ours!

@David how is the behaviour between packages that have as dependencies another package of the repo? Do they use the version published in npm or do they use the version from packages in the repo?

They use the other packages in the monorepo as if they were published to NPM, I mean, you import them the same way, you need to have them defined as a dependency in package.json and with a semver that matches the version in the monorepo.

I’m not sure if this is what you asked… :thinking:

I forgot mentioning that before start developing, you need to execute lerna bootstrap to connect the packages. I’m going to add this command in the postinstall script of the package.json.

@David how will a folder /examples work regarding installing npm pacakges from /packages, versions and so on…?