Infinite Scroll hooks

orballo · 2 July 2020 23:54

I don’t think the interface shown to the user is to complex, though I don’t like the logic behind. I’d not play with the isLimit logic, I’d add an isError logic that prevents the infinite scroll, and it’s set until a new fetch works.

luisherranz · 3 July 2020 07:30

OK, so you would make it two separate things:

isLimit and button the get next page.
isError and button to try again.

Is that right?

orballo · 3 July 2020 07:42

Yes, for the user it would be like you described. But internally the fetchNext function will be the one in charge of reseting isError.

orballo · 20 August 2020 02:02

The final changes implemented to the infinite-scroll hook with the intention of handling fetch errors are the following:

useArchiveInfiniteScroll and usePostTypeInfiniteScroll now return a new boolean called isError which can be used to know if the last fetch inside the infinite scroll hook has failed.
In addition, now the useArchiveInfiniteScroll and usePostTypeInfiniteScroll accept the fetchInViewOptions and routeInViewOptions that will be passed to the IntersectionObserver used by useInView.

And example of using the hooks with the new changes could be:

const Archive: React.FC = () => {
  const { state, libraries } = useConnect<Packages>();
  const current = state.source.get(state.router.link);

  const { pages, isFetching, isError, fetchNext } = useArchiveInfiniteScroll({
    fetchInViewOptions: {
      rootMargin: "400px 0px",
      triggerOnce: true,
    },
    routeInViewOptions: {
      rootMargin: "-80% 0% -19.9999% 0%",
    },
  });

  return (
    <div>
      {pages.map(({ Wrapper, key, link }) => {
        const { page } = libraries.source.parse(link);
        const data = state.source.get(link);
        return (
          <Wrapper key={key}>
            <div>
              Page {page}
              <ul>
                {data.items.map((item) => (
                  <li key={item.id}>{item.link}</li>
                ))}
              </ul>
            </div>
          </Wrapper>
        );
      })}
      {isFetching && <div>Fetching</div>}
      {isError && (
        <div>
          <button onClick={fetchNext}>
            Try Again
          </button>
        </div>
      )}
    </div>
  );
};

export default connect(Archive, { injectProps: false });

luisherranz · 20 August 2020 06:57

Thanks for the update @orballo.

So I guess that if the user wants to use a limit, the fetchNext action will be used for both isLimit and isError, right?

const Archive = () => {
  const {
    pages,
    isFetching,
    isError,
    isLimit,
    fetchNext,
  } = useArchiveInfiniteScroll({
    limit: 3,
  });

  return (
    <div>
      {pages.map(({ Wrapper, key, link }) => (
        <Wrapper key={key}>
          <ArchivePage link={link} />
        </Wrapper>
      ))}
      {isFetching && <div>Fetching</div>}
      {isLimit && <button onClick={fetchNext}>Next Page</button>}
      {isError && <button onClick={fetchNext}>Try Again</button>}
    </div>
  );
};

export default connect(Archive);

luisherranz · 20 August 2020 08:07

I’ve published a new beta version with this change. The packages updated were:

@frontity/hooks@2.1.0-infinite-scroll-beta.2
@frontity/html2react@1.3.4-infinite-scroll-beta.1
@frontity/wp-source@1.8.2-infinite-scroll-beta.0

Please update your version and let us know how it goes

orballo · 20 August 2020 09:26

That is correct, as long as the last item couldn’t be fetched, the fetchNext action will try to fetch it again, and will never try to get the next item until the last one is fetched.

luisherranz · 21 August 2020 07:58

I’ve been testing this and the only packages required are:

@frontity/hooks@2.1.0-infinite-scroll-beta.2

@frontity/tiny-router@1.3.0-infinite-scroll-beta.0.

@frontity/wp-source@1.8.2-infinite-scroll-beta.0

I’ve updated the opening post with this info as well.

I’ve also created a CodeSandbox in case you want to test it out in the browser:

SantosGuillamot · 26 August 2020 09:58

I tried to implement this in frontity.org and it works for both the archives and the single post. However, while using the UsePostTypeInfiniteScroll in the posts, it just works if I navigate first to the homepage. If I access the post directly (SSR), it doesn’t. It seems the problem is that we’re using the postsPage setting, so if we go directly to the post, it’s trying to load the link "/", and that’s not correct. I’ve check that if I change this line of code it works nice in our case:

return state.source.postsPage || "/";

I’ve also checked and it seems it happens the same if you use the subdirectory setting, so I guess the solution could be something like this (in case you have both of them defined):

return state.source.subdirectory + state.source.postsPage || "/";

The problem here is that, although it works, TypeScript is complaining because properties subdirectory and postsPage don’t exist. This part I don’t know how to solve it.

Moreover, changing that line as I said only works if you defined the subdirectory with an opening slash (/blog). If we remove it (blog) or add and ending slash (/blog/), it wouldn’t work. So we have to take this into account as well.

Let me know what you think. If it can be solved easily just modifying that line I can open a Pull Request directly. If it’s going to be more difficult I’ll open an issue to keep track of it.

orballo · 26 August 2020 12:29

I think we talked about this once already? Maybe related to something else or was it related to infinite scroll?

I’m not familiar with those settings. If it is safe to use them, I can implement that change easily.

SantosGuillamot · 26 August 2020 15:14

Yes, it was this same problem but we didn’t write down our findings.

Maybe @David can help here as he implemented them.

luisherranz · 28 September 2020 15:54

Thanks for the last PR @orballo!

I’ve just release a new beta version of @frontity/hooks: 2.1.0-infinite-scroll-beta.3

@orballo could you please update your project and see if everything is fine?

After that, I will take a deep look to see if we don’t forget anything. I think there may be some tests still missing, right? When everything is ok we will merge the infinite-scroll-beta branch with dev and we will release the hook

orballo · 28 September 2020 16:01

@SantosGuillamot Could you please also check that the bug you mentioned is working properly now? Thanks!

SantosGuillamot · 29 September 2020 06:56

It works great now for me

orballo · 29 September 2020 13:53

My issue has been solved too

luisherranz · 9 December 2020 15:51

I have released a new version of the required packages to test the infinite scroll hook:

@frontity/hooks@2.1.0-infinite-scroll-beta.4
@frontity/tiny-router@1.3.2-infinite-scroll-beta.0
@frontity/wp-source@1.10.1-infinite-scroll-beta.0

David · 22 February 2021 11:12

Hey! Some updates here:

The PR (#652) where we’ve been preparing the infinite scroll hooks to be officialy released is almost done. I’ll be working on writing the Final Implementation post in the meantime. Just so you know, the API will remain the same.

Keep tuned!

David · 24 February 2021 17:43

Pull Request

Requirements

Feature:

@frontity/hooks (^2.2.0)

Dependencies:

@frontity/tiny-router (to be filled once the latest version is released)
@frontity/wp-source (to be filled once the latest version is released)

Functionalities

Three React hooks were implemented. Two for general use:

useArchiveInfiniteScroll
usePostTypeInfiniteScroll

And a basic one for implementing custom infinite scroll hooks (used internally
by the previous two hooks):

useInfiniteScroll

NOTE: useInfiniteScroll is not intented to be used directly by theme
developers unless they are creating their own infinite scroll logic.

The main idea behind these hooks is that they return a list of Wrapper
components, one for each entity listed while scrolling, that handle both the
route updating and fetching of the next entity.

Also, the following action was added to the @frontity/router API and the
@frontity/tiny-router package.

actions.router.updateState - Function to update the content of the browser
history state for the current link.

Infinite scroll hooks

`useInfiniteScroll`

This is the core hook with the basic logic to build an infinite scroll hook.

It basically receives two links, currentLink and nextLink, and returns two
React refs that should be attached to react elements. The hook uses useInView
internally to track the visibility of those elements and trigger an
actions.router.set to update the current link or an actions.source.fetch to
fetch the next entity. You can pass options for these useInView hooks as well,
using the fetchInViewOptions and the routeInViewOptions params.

useInfiniteScroll also keeps a record of the fetched & ready entities in the
browser history state, in order to restore the list when you go back and
forward while navigating. That record is accessible from the browser history
state under the infiniteScroll.links array.

Note: the history state is also accessible from the Frontity state, in
state.router.state.

It was designed to be used inside a Wrapper component that would wrap the
entity pointed by currentLink.

Parameters

It requires an object with the following props:

Name	Type	Default	Required	Description
`currentLink`	string	-	yes	The current link that should be used to start the infinite scroll.
`nextLink`	string	-	no	The next link that should be fetched and loaded once the user scrolls down.
`fetchInViewOptions`	IntersectionOptions	-	no	The intersection observer options for fetching.
`routeInViewOptions`	IntersectionOptions	-	no	The intersection observer options for routing.

NOTE: The IntersectionOptions type refers to the type of the the parameters
received by the useInView hook.

Return value

Name	Type	Description
`supported`	boolean	Boolean indicating if the Intersection Observer is supported or not by the browser.
`routeRef`	React.Ref	The ref that should be attached to the element used to trigger `actions.router.set`.
`fetchRef`	React.Ref	The ref that should be attached to the element used to trigger `actions.source.fetch`.
`routeInView`	boolean	Boolean that indicates when the element used to trigger `actions.router.set` is in the screen.
`fetchInView`	boolean	Boolean that indicates when the element used to trigger `actions.source.fetch` is in the screen.

Usage

Note: this is just an example to illustrate how the useInfiniteScroll works.
For better examples, see the useArchiveInfiniteScroll and the
usePostTypeInfiniteScroll implementation.

import { useConnect, connect, css } from "frontity";
import useInfiniteScroll from "../use-infinite-scroll";
import { isArchive, isError } from "@frontity/source";

export const wrapperGenerator = ({
  link,
  fetchInViewOptions,
  routeInViewOptions,
}) => {
  const Wrapper = ({ children }) => {
    const { state } = useConnect();

    const current = state.source.get(link);
    const next =
      isArchive(current) && current.next
        ? state.source.get(current.next)
        : null;

    const { supported, fetchRef, routeRef } = useInfiniteScroll({
      currentLink: link,
      nextLink: next?.link,
      fetchInViewOptions,
      routeInViewOptions,
    });

    if (!current.isReady || isError(current)) return null;
    if (!supported) return children;

    const container = css`
      position: relative;
    `;

    const fetcher = css`
      position: absolute;
      width: 100%;
      bottom: 0;
    `;

    return (
      <div css={container} ref={routeRef}>
        {children}
        {<div css={fetcher} ref={fetchRef} />}
      </div>
    );
  };

  return connect(Wrapper);
};

`useArchiveInfiniteScroll`

This hook implements the logic needed to include infinite scroll in archives
(i.e. categories, tags, the posts archive, etc.).

The hook receives options to set a limit of pages shown automatically, to
disable it, and also settings for the intersection observers that are passed to
the useInfiniteScroll hooks used internally.

useArchiveInfiniteScroll is designed to be used inside an Archive component.
That component would render all the archive pages from the pages returned by the
hook.

Apart from that list, it returns a set of boolean values to know if the next
pages is being fetched, if the limit has been reached or if the next page
returned an error, and a function to fetch the next page manually.

Parameters

It accepts an optional object with the following props:

Name	Type	Default	Required	Description
`active`	boolean	`true`	no	A boolean indicating if this hook should be active or not. It can be useful in situations where users want to share the same component for different types of Archives, but avoid doing infinite scroll in some of them.
`limit`	number	`infinite`	no	The number of pages that the hook should load automatically before switching to manual fetching.
`fetchInViewOptions`	IntersectionOptions	-	no	The intersection observer options for fetching.
`routeInViewOptions`	IntersectionOptions	-	no	The intersection observer options for routing.

NOTE: The IntersectionOptions type refers to the type of the the parameters
received by the useInView hook.

Return value

An object with the following properties:

Name	Type	Description
`pages`	Array of page props	An array of the existing pages. Users should iterate over this array in their own layout. The content of each element of this array is explained below.
`isFetching`	boolean	If it’s fetching the next page. Useful to add a loader.
`isLimit`	boolean	If it has reached the limit of pages and it should switch to manual mode.
`isError`	boolean	If the next page returned an error. Useful to try again.
`fetchNext`	function	A function that fetches the next page. Useful when the limit has been reached (`isLimit === true`) and the user pushes a button to get the next page or when there has been an error fetching the last page and the user wants to retry.

Each element of the pages array has the following structure:

Name	Type	Description
`key`	string	A unique key to be used in the iteration.
`link`	string	The link of this page.
`isLast`	boolean	If this page is the last page. Useful to add separators between pages, but avoid adding it for the last one.
`Wrapper`	React.FC	The Wrapper component that should wrap the real `Archive` component.

Usage

import { connect, useConnect } from "frontity";
import { useArchiveInfiniteScroll } from "@frontity/hooks";
import ArchivePage from "./archive-page";

/**
 * Simple component showing the usage of the `useArchiveInfiniteScroll` hook.
 *
 * @example
 * ```
 * // In the Theme component:
 * <Switch>
 *   {...}
 *   <Archive when={data.isArchive} />
 * </Switch>
 * ```
 */
const Archive = () => {
  // Get the list of pages from the hook.
  const {
    pages,
    isFetching,
    isLimit,
    isError,
    fetchNext,
  } = useArchiveInfiniteScroll({ limit: 3 });

  return (
    <>
      {pages.map(({ Wrapper, key, link, isLast }) => (
        <Wrapper key={key}>
          <ArchivePage link={link} />
          {!isLast && <PageSeparator />}
        </Wrapper>
      ))}

      {isFetching && <div>Loading more...</div>}

      {(isLimit || isError) && (
        <button onClick={fetchNext}>
          {isError ? "Something failed - Retry" : "Load More"}
        </button>
      )}
    </>
  );
};

export default connect(Archive);

`usePostTypeInfiniteScroll`

Hook that implements the logic needed to include infinite scroll in a post type
view (i.e. posts, pages, galleries, etc.).

This hook is more complex than the previous one, as it works getting the post
type entities from the specified archive and thus it doesn’t fetch the next post
but the next page of posts.

It recevies an archive and a fallback prop ―both links―, to specify the
source of the post entities. If none of them is specified,
state.source.postsPage is used. When the penultimate post of the first page is
rendered, the next page of the archive is fetched. A list of the fetched pages
is stored in the browser history state along with the list of posts.

The limit prop in this case stands for the number of posts being shown, not
the number of fetched pages. In the same way, the fetchNext shows the next
post, and only fetches the next page of posts if needed.

Parameters

It accepts an optional object with the following props:

Name	Type	Default	Required	Description
`active`	boolean	`true`	no	A boolean indicating if this hook should be active or not. It can be useful in situations where users want to share the same component for different types of Archives, but avoid doing infinite scroll in some of them.
`limit`	number	`infinite`	no	The number of pages that the hook should load automatically before switching to manual fetching.
`archive`	string	-	no	The archive that should be used to get the next posts. If none is present, the previous link is used. If the previous link is not an archive, the homepage is used.
`fallback`	string	-	no	The archive that should be used if the `archive` option is not present and the previous link is not an archive.
`fetchInViewOptions`	IntersectionOptions	-	no	The intersection observer options for fetching.
`routeInViewOptions`	IntersectionOptions	-	no	The intersection observer options for routing.

NOTE: The IntersectionOptions type refers to the type of the the parameters
received by the useInView hook.

Return value

The output of these hooks is pretty similar to the previous one’s:

Name	Type	Description
`posts`	Array of post props	An array of the existing posts. Users should iterate over this array in their own layout. The content of each element of this array is explained below.
`isFetching`	boolean	If it’s fetching the next post. Useful to add a loader.
`isLimit`	boolean	If it has reached the limit of posts and it should switch to manual mode.
`isError`	boolean	If the next page returned an error. Useful to try again.
`fetchNext`	function	A function that fetches the next post. Useful when the limit has been reached (`isLimit === true`) and the user pushes a button to get the next post or when there has been an error fetching the last post and the user wants to retry.

Each element of the posts array has the following structure:

Name	Type	Description
`key`	string	A unique key to be used in the iteration.
`link`	string	The link of this page.
`isLast`	boolean	If this post is the last post. Useful to add separators between posts, but avoid adding it for the last one.
`Wrapper`	React.FC	The Wrapper component that should wrap the real `Post` component.

Usage

import { connect, useConnect } from "frontity";
import { usePostTypeInfiniteScroll } from "@frontity/hooks";
import PostTypeEntity from "./post-type-entity";

/**
 * Simple component showing the usage of the `usePostTypeInfiniteScroll` hook.
 *
 * @example
 * ```
 * // In the Theme component:
 * <Switch>
 *   {...}
 *   <PostType when={data.isPostType} />
 * </Switch>
 * ```
 */
const PostType = () => {
  // Get the list of posts from the hook.
  const {
    posts,
    isFetching,
    isLimit,
    isError,
    fetchNext,
  } = usePostTypeInfiniteScroll({ limit: 5 });

  return (
    <>
      {posts.map(({ Wrapper, key, link, isLast }) => (
        <Wrapper key={key}>
          <PostTypeEntity link={link} />
          {!isLast && <PostSeparator />}
        </Wrapper>
      ))}

      {isFetching && <div>Loading more...</div>}

      {(isLimit || isError) && (
        <button onClick={fetchNext}>
          {isError ? "Something failed - Retry" : "Load More"}
        </button>
      )}
    </>
  );
};

export default connect(PostType);

More things added

`actions.source.updateState`

Action that replaces the value of state.router.state with the give object. The
same object is stored in the browser history state using the
history.replaceState()
function.

Parameters

Name	Type	Default	Required	Description
`historyState`	object	-	yes	The object to set as the history state.

Out of Scope

I was out of the scope of this PR to implement a way to let developers to change
the logic that usePostTypeInfiniteScroll uses to get the next post.

Right now, that logic is the following:

If the post is the first post rendered and it’s not included in the first page
of the archive, the next post is the first post of the archive.
For any other case, get the index where the post appears in the fetched pages.
The next post will be the one with index + 1.

API changes

Backward compatible changes

Instead of having to import each hook from its module, hooks can be imported now
from the package root:

import { useInView, useInfiniteScroll } from "@frontity/hooks";

They can still be imported directly from each module:

import useInView from "@frontity/hooks/use-in-view";
import useInfiniteScroll from "@frontity/hooks/use-infnite-scroll";

Breaking changes

No breaking changes.

Deprecated APIs

No deprecated APIs.

Technical explanation

Work in progress.

luisherranz · 1 March 2021 08:33

I said so in the PR, but I’ll repeat it here.

I don’t have words to say how amazing this feature is guys. This implementation is simply brilliant. And the test coverage is outstanding.

It is the best infinite-scroll feature a frontend framework could dream of:

100% of abstraction of the infinite scroll logic.
100% of control of the layout to the developer.

Congratulations to both @orballo and @david for your amazing work

BRAVO!

luisherranz · 1 March 2021 17:01

I’ve just read the FI. Great work, thanks @david

I’ve added isLimit and fetchNext to the examples to show how to use them, can you review them and let me know if they are correct?

Also, the logic behind the "archive" and "fallback" is a bit difficult to understand so I did this drawing:

Can you review it and confirm that is correct? (link to the excalidraw)

Thanks!