Navigate paginated websites

How can I navigate paginated Websites?

Paginated sites are those with page numbers and Previous/Next page navigation links near the bottom of the page. There are often other pagination elements, such as arrows.

The Iterate Pages step often works right out of the box by selecting an element that navigates to the next page, such as a Next or arrow link, and selecting Iterate Pages in the element panel. If it works, then great. You are all set.

During its first iteration, it will do nothing (since it's the first page). Its second and all other iterations will select the next page element. It will repeat until the target element is either no longer found or doesn't cause the page to change.

However, pagination elements can behave very differently from each other, so sometimes you have to troubleshoot in order to get it to work.

A common pattern of pagination interface design is that once you navigate to Page 2, a Previousbutton displays. There is now a new element on the page, and this can cause the CSS path assigned to the Iterate Pagesstep to become incorrect. In fact, it can sometimes actually point to the last page button instead of the next page button. In practice, this means that it will first go to Page 2, then back to Page 1.

An example:

Page 1:

1,2,3,4,5,6,last,next

CSS path: UL > LI:nth-child(8) - points to next

Page 2:

previous,1,2,3,4,5,6 last,next

CSS path: UL > LI:nth-child(8) - now points to last

The iteration step should point to a Next button or similar selectable element that navigates to the next page. There are two ways to do this: :last and :text_contains().

• :last

One way to solve the problem is to navigate to Page 2 in the robot editor and verify that the CSS path of the Iterate Pagesstep still points to the next button.

If it doesn't, modify the CSS path accordingly, which can often be as simple as adding :last to the end. This will always select the last element.

• :text_contains("Next")

Use this to match the wording on the elements which skips to the next page. Just replace the text in quotes (" ") with the text from the button/element.