The Source API

Source API

State

Let’s start by explaining how the state data is used and then how that data is requested and stored. The state works with two main concepts: paths and entities.

The state is designed so that you can know which entities correspond to which path, and then access the data of these entities in a simple way.

NOTE: for the data to exist, it will be necessary to request them previously using the fetch action.

example

// Accessing path's data
const data = source.data["/categories/nature"]

// `data` value:
({
  
  // Type & id of the entity
  type: "category",
  id: 2,
  
  // Type & id of the listed entities
  // (when the path is an archive or similar)
  items: [{ type: 'post', id: 60, path }, ...],   // all fetched entities
  pages: [[{ type: 'post', id: 60, path }, ...]], // fetched entities per page
  totalItems: // total number of entities
  totalPages: // total number of pages

  // Booleans to easily identify the type of path
  // inside the WP hierarchy
  isArchive: true,
  isTaxonomy: true,
  isCategory: true,

})

// Accessing the entity's properties
const entity = source[d.type][d.id]

// `entity` value:
({
  id: 2,
  count: 23,
  description: "Beautiful nature locations",
  link: "https://myblog.com/category/nature/",
  name: "Nature",
  slug: "nature",
  taxonomy: "category",
})

The information to distinguish each type of path is based on the WP Template Hierarchy and is as follows:

  • archives: isArchive
    • taxonomy: isTaxonomy (with type, id properties)
      • category: isCategory
      • tag: isTag
      • deal: isDeal
    • author: isAuthor (with id property)
    • postTypeArchive: isPostTypeArchive (with type property)
      • post: isHome, isPostArchive (isFrontPage optional)
      • product: isProductArchive
    • date: isDate (with date)
  • postTypes: isPostType (with type, id properties)
    • post: isPost
    • page: isPage (isFrontPage optional)
    • product: isProduct
    • media: isMedia, isAttachment
  • 404: is404

State API

source.data[path]

Access data related to a path. The path parameter could be also a name that identifies custom lists. See source.add for more info.

source.data["/the-beauties-of-gullfoss"]
source.data["/category/nature"]
source.data["gallery-post-60"]

source.taxonomy[taxonomy]

Access taxonomy entities. taxonomy can be "category", "tag" or any custom taxonomy slug. That value is stored in the type attribute of the source.data[path] object.

source.taxonomy["category"]
source.taxonomy["tag"]
source.taxonomy["deal"] // custom taxonomy

source[taxonomy][id]

Access category, tag, or custom taxonomy’s entities.

source.category[2]
source.tag[13]
source.deal[3]

source.type[type]

Access post types. type can be "post", "page", "attachment" or any custom post type.

source.type["post"]
source.type["page"]
source.type["product"] // custom post type

source[type][id]

Access posts, pages, attachments or custom post type’s entities.

source.post[60]
source.page[7]
source.product[36]

source.author[id]

Access author entities.

source.author[4]

Actions

source.fetch(path, page)

Action that retrieves entities using a path or a name. fetch works using the corresponding fetch function defined with the add action. This action doesn’t return anything but ensures that the received data are correctly populated in the state.

arguments

  • path: string that contains a path or a custom list’s name
  • page: page number (default is 1)

example

source.fetch("/category/nature")
source.fetch("/category/nature", 2)
source.fetch("/the-beauties-of-gullfoss")
source.fetch("custom-list")

There are some URL patterns handled by default. Those are:

postArchive: '/(?s=:search)'
category:    '/category/:slug(/?s=:search)'
tag:         '/tag/:slug(/?s=:search)'
author:      '/author/:name(?=:search)'
date:        '/:year/:month?/:day?(?s=:search)'
postOrPage:  '/:slug'
attachment:  '/:year/:postSlug/:attachmentSlug'

source.add(pattern, callback)

Action that attaches a function to handle a specified name or route pattern, using a string or a regexp. That function should request data from the WP REST API and populate the store with it. For that purpose, you may use the populate action.

arguments

  • pattern: string or regexp

  • callback: async function that will be executed if path matches pattern when calling source.fetch(path). Its arguments are:

    • config: Overmind config object ({ state, actions, effects }).
    • payload: object with the following attributes:
      • name: the path or name that has matched the pattern
      • params: params extracted from name after the match
      • page: page number passed when calling source.fetch(path, page)

example

source.add(
  "/post/:slug",
  async ({ state, actions, effects }, { name, params }) => {
    // Get a post
    const { body: post } = await effects.api.get(
      {
        endpoint: "/wp/v2/posts",
        params: {
          slug: params.slug
        }
      }
    );

    // Populate data response in the store
    actions.source.populate(name, post);
  }
)

source.populate(path, entities, page)

Action that stores entities in the state so you can find them using the API specified in the State API section. Also, it sets all info related to paths for each entity (based on the entity’s link). This action is intended to be used after request entities from the REST API (using api.get effect).

arguments

  • path: path or name
  • entities: array or a single entity, from a WP REST API request
  • page: the requested page number

example

source.populate('/category/nature', posts, 1)

// That action should populate all the state below
source.data['/category/nature']
source.data["/categories/nature"].page[1]
source.data["/categories/nature"].items
source.category[2]
source.data['/the-beauties-of-gullfoss']
source.post[60]
source.data["/author/luis"]
source.author[4]
source.attachment[15]
source.attachment[16]

NOTE: when using a name instead of a path, any property to identify that name should be added explicitly:

source.populate("gallery-1", attachments, 1)

// That action should populate all the state below
source.data["gallery-1"]
source.data["gallery-1"].page[1]
source.data["gallery-1"].items
source.attachment[15]
source.attachment[16]
// ...

// This should be added explicitly
source.data["gallery-1"].isGallery = true

Effects

api.get({ endpoint, params })

Request entity to the WordPress REST API.

arguments

  • endpoint: name of the endpoint if is a /wp/v2 endpoint (e.g. posts), or the full path of other REST endpoints (e.g. /frontity/v1/discovery)
  • params: any parameter that will be included in the query params.

For more info, visit the WP REST API reference.

example

// Get posts from categories 2, 3 and 4
api.get({ endpoint: "posts", params: { _embed: true, categories: '2,3,4' } });

// Get the page 14
api.get({ endpoint: "pages", params: { _embed: true, include: '14' } });

// Other endpoints: 
api.get({ 
  endpoint: "/frontity/v1/discovery",
  params: { path: "/the-beauties-of-gullfoss" }
});

Settings

All the options that can be configured in the package settings, like if it is WordPress.org or WordPress.com, the REST base route, etc.

1 Like

Two doubts I have:


If we want to populate the attributes

totalItems
totalPages

for a list, the action source.populate is ignoring that attributes, as the entities argument doesn’t contain them.

source.populate(path, entities, page)

How would it be better to pass those attributes?

// response = { headers, body: entities }
source.populate(path, response, page)

// other options
const entities = response.body
const totalItems = response.headers["X-WP-Total"]
const totalPages = response.headers["X-WP-TotalPages"]

source.populate(path, entities, page)

source.data[path].totalItems = totalItems;
source.data[path].totalPages = totalPages;


Perhaps, since path and entity are two clearly differentiated concepts, we could leave clearer the separation with

const { type, id } = source.path["/category/nature"]
source.entity[type][id]

even changing path to name so that both things are included

const { type, id } = source.name["/category/nature"]
source.entity[type][id]

const { items, isGallery } = source.name["gallery-1"]
const entities = items.map(({ type, id }) => source.entity[type][id])

would this change worth it?

I hope that with all this I can start programming. :smile::rocket:

@luisherranz, @orballo, let me know what you think!

1 Like

Amazing work @David!!

I think I’d leave things like totalItems, totalPages, isCategory and so on to the source.fetch instead of the populate. We can include fetchedItems and fetchedPages, even isFetching like we had in the old one.

We also need to decide what is an item, what is an entity and if we need both.

Oh and we need to decide if we call it name or path.

I don’t have a strong opinion about this right now. Are entities: taxonomies, authors and post types?

Sure!! :clap::clap:

First of all, great work! :muscle:

I have some doubts:

  • In the case of custom-list, do we define what entities belong to it in a source.add() function?
  • Does the developer need to define all source.add() functions or are some already part of wp-source code? If yes, can a developer override those paths with another add()?
  • I find source.add() name a bit confusing… maybe is better to use something like register or handle?

I like source.path["/category/nature"], source.name["gallery-one"] and source.entity[type][id].

If I remember correctly, in the previous Frontity items was used only in the router, any data stored would be entities. Maybe we should use only entities in wp-source.

I like source.register :+1:

That’s a good point: how do we order all the source.registers? Maybe we need priorities or some clever way to sort them.

Hmm… the thing is that path is just a name. It just happens to start with "/" but otherwise, there’s no difference at all.

We are going to change type for taxonomy to match the API.

What about if we allow either a name or an object in the functions that accept a name and a page? Like source.fetch, source.data and router.set?

source.fetch("/category/nature");
source.fetch({ name: "/category/nature", page: 2 });

We also need to decide if we call it name or path.

@David, what do you think?

Uhm, I like it!

About how to call that property, I’m not really sure. We could name it path as it will be the most usual kind of value, but there will be other cases where path won’t fit, e.g. custom lists.

I like the idea of having the exact same API for source.fetch , source.data and router.set, so maybe path is the best option:

actions.source.fetch("/category/nature");
state.source.data("/category/nature");
actions.router.set("/category/nature");

or

actions.source.fetch({ path: "/category/nature", page: 2 });
state.source.data({ path: "/category/nature", page: 2 });
actions.router.set({ path: "/category/nature", page: 2 });

so simple!

1 Like

Ok, you have convinced me :grin:

Ok, we have launched the beta! :tada:

Now it’s time to rethink some aspects of the Source API that need to be fixed, like 404 support for archive pages.

At this moment, when you use source.data(path), that path must be the pathname of the URL without the page (i.e. without /page/2 or similar). That is because pages are stored right now in the same data object, inside the property pages (data object is loosely explained here).

The problem is is404 isn’t set for each page but for the whole data object and this won’t let you know which page is actually a 404.

I’ll post some options I’m working on soon.

We another problem with the links because in the API they include the wordpress domain: https://test.frontity.io/wp-json/wp/v2/posts

When devs use that .link property to populate their links, it shows the wrong url:

It works because we are ignoring that domain in actions.router.set(), but they should be fixed.

Possible solutions:

  1. Change the WP domain for the state.frontity.url domain.
  2. Remove the domain so they are paths: /my-link.

I’ve opened a bug:

I see. Uhm, I prefer to transform links into paths as we are using paths more often than links. And also populate function wouldn’t need to check any property outside state.source (this is the object it receives as argument).


About 404's, one thing we could do is to move is404 property inside each page.

Right now, is404 property is being set by fetch action. In order to be consistent, if archive handlers must set this property for not-found pages, then fetch action should not set this property at all, and so, only handlers should be responsible to set this value.

Other option would be to create a data object for each path with pagination (/page/2).

Great. I have updated the issue.

Regarding the 404 problem I think it would be a good approach to have the very same API for all source.get source.fetch and router.set. So yes, I would do as separate data entry for each page.

That’s one of the options I was thinking. :thinking:

If we decide to maintain the same API as you mentioned before, the way you access archive pages would be:

const page1 = state.source.get("/category/nature")
const page2 = state.source.get({ path: "/category/nature", page: 2 })

As we are currently using dataMap to store the data objects, we would need to transform the second get call parameters to a string, something like "/category/nature/page/2", in order to create a reference to that page (i.e. source.dataMap["/category/nature/page/2"]).

The structure of the data object would end up like this:

const page1 = {
  taxonomy: "category",
  id: 7,

  isArchive: true,
  isTaxonomy: true,
  isCategory: true,

  items: [{ type, id, link }],

  is404: false
}

Ok, some considerations:

  1. We need a final specification for the pathOrObj API, used in source.fetch, source.get and router.set. Something like:

    • You can pass a string with the path or an object with a path and page.
    • If the path has a domain, it will be stripped out.
    • If the path has a page, it will be assigned to the page variable and it will be stripped out.
    • If path has a page but page is present, page will prevail.
    • If page is not present either in path or page, it will be 1.

With this API, there 3 ways to access page 2:

const page2 = state.source.get("/category/nature/page/2")
const page2 = state.source.get({ path: "/category/nature/page/2" })
const page2 = state.source.get({ path: "/category/nature", page: 2 })
  1. The pathOrObj to path & page converters should live in the source packages as a library. Routers (or other packages) should be agnostic to the final urls.

  2. I agree with you, the source.data keys could be "/category/nature/page/2". isFetching and isReady start making sense again, don’t they?

And a question:

  1. How’s the API to access all fetched pages of a category going to be?

I find passing a full string to the path param a bit confusing. If we can pass "/category/nature/page/2" as a single param, maybe it doesn’t make sense to have the object version of that api.

The problem is that sometimes developers have the full url (for example, from a post.link) and sometimes they have the path and the page (for example, from router.path and router.page).

If we don’t allow both, they have to convert them, which is bothersome. Sometimes is even difficult to know what they have, for example the Link component can receive both full link or path and it doesn’t know which one it get.