What is API pagination?

A method to split large results into manageable pages using strategies such as offset, cursor, or keyset.

When should I use which strategy?

Offset for small datasets and simple UIs; cursor for infinite scrolling; keyset for large datasets with fast, indexed access; avoid heavy COUNT queries.

How can I optimize performance?

Index the fields used for sorting, cap page sizes, and prefer cursor or keyset to avoid expensive counts and full table scans.

api-pagination

Scanned

npx machina-cli add skill secondsky/claude-skills/api-pagination --openclaw

Files (1)

SKILL.md

2.1 KB

API Pagination

Implement scalable pagination strategies for handling large datasets efficiently.

Pagination Strategies

Strategy	Best For	Performance
Offset/Limit	Small datasets, simple UI	O(n)
Cursor	Infinite scroll, real-time	O(1)
Keyset	Large datasets	O(1)

Offset Pagination

app.get('/products', async (req, res) => {
  const page = parseInt(req.query.page) || 1;
  const limit = Math.min(parseInt(req.query.limit) || 20, 100);
  const offset = (page - 1) * limit;

  const [products, total] = await Promise.all([
    Product.find().skip(offset).limit(limit),
    Product.countDocuments()
  ]);

  res.json({
    data: products,
    pagination: {
      page,
      limit,
      total,
      totalPages: Math.ceil(total / limit)
    }
  });
});

Cursor Pagination

app.get('/posts', async (req, res) => {
  const limit = 20;
  const cursor = req.query.cursor;

  const query = cursor
    ? { _id: { $gt: Buffer.from(cursor, 'base64').toString() } }
    : {};

  const posts = await Post.find(query).limit(limit + 1);
  const hasMore = posts.length > limit;
  if (hasMore) posts.pop();

  res.json({
    data: posts,
    nextCursor: hasMore ? Buffer.from(posts[posts.length - 1]._id).toString('base64') : null
  });
});

Response Format

{
  "data": [...],
  "pagination": {
    "page": 2,
    "limit": 20,
    "total": 150,
    "totalPages": 8
  },
  "links": {
    "first": "/api/products?page=1",
    "prev": "/api/products?page=1",
    "next": "/api/products?page=3",
    "last": "/api/products?page=8"
  }
}

Best Practices

Set reasonable max limits (e.g., 100)
Use cursor pagination for large datasets
Index sorting fields
Avoid COUNT queries when possible
Never allow unlimited page sizes

Source

git clone https://github.com/secondsky/claude-skills/blob/main/plugins/api-pagination/skills/api-pagination/SKILL.mdView on GitHub

Overview

Implements scalable pagination strategies for large datasets using offset/limit, cursor, and keyset methods. This helps you build paginated endpoints, enable infinite scroll, and reduce database query costs.

How This Skill Works

The skill maps data access patterns to three pagination strategies and provides concrete examples for each (offset, cursor, and keyset). It emphasizes practical performance considerations like setting a max page size, indexing sorted fields, and avoiding expensive count queries to keep endpoints fast.

When to Use It

Building a REST API endpoint that returns pages of items using page and limit (offset pagination)
Implementing infinite scrolling in a frontend with cursor-based pagination
Handling very large datasets where OFFSET scans become expensive and slow
Delivering near real-time feeds with low latency using cursor or keyset pagination
Reducing database load by avoiding frequent COUNT queries and using keyset/indexed sorts

Quick Start

Step 1: Assess data size and UI needs to choose between offset, cursor, or keyset pagination
Step 2: Implement the endpoint with a capped page size and the appropriate query logic
Step 3: Return a consistent shape (data plus pagination or nextCursor/links) and monitor performance

Best Practices

Set reasonable max limits (e.g., 100) and cap page sizes
Use cursor pagination for large datasets to achieve O(1) navigation
Index the fields used for sorting and filtering to speed queries
Avoid COUNT queries when possible to improve performance
Never allow unlimited page sizes; provide clear totalPages or nextCursor

Example Use Cases

An e commerce product listing implemented with offset pagination using page and limit parameters
A social feed or blog with infinite scrolling powered by cursor-based pagination
A massive user directory paginated via keyset pagination on indexed _id or createdAt fields
A dashboards API that omits heavy total counts to improve response time while paginating
A REST response that includes data, a pagination block, and navigational links (first, prev, next, last)

Frequently Asked Questions

Add this skill to your agents