DynamoDB example with paging/infinite scroll?

[rdyar]

I am using Dynamodb and react-query and it works great as long as I have a small set of data and want all of it at once.

But if I want to do a limit on what gets returned from dynamo with an Load more button I am not sure how to do it and can't find any examples.

It would more or less be the useInfiniteQuery I think - though dynamo does not work with paging the way other db's do. I also think maybe it should be a mutation to add to the original query but that doesn't sound right.

Any examples that anyone can share, or tips to point me in the right direction?

[TkDodo]

You should probably share how dynamo works with paging. Just mentioning that it works differently will not be enough to get anyone to help unless they have experience with that specific database ...

[rdyar]

Dynamo uses LastEvaluatedKey to get the next batch, which I think will work. I just need to pass that to my api.

I have the initial set loading, but what is being passed to my api doesn't make sense to me. I don't understand the example where it does:

useInfiniteQuery('projects', fetchProjects, {
     getNextPageParam: (lastPage, pages) => lastPage.nextCursor,
   })

I have changed that to

useInfiniteQuery('history', fetchHistory, {
    getNextPageParam: (lastPage, pages) => lastPage.LastEvaluatedKey,
  })

so that it will pass the LastEvaluated Key to my api but instead when I log what is passed in it is { queryKey: [ 'history' ], signal: {} } which I assume is coming from react-query. That is what I get with the initial load when it should really be empty I think.

My fetchHistory is using axios like this:

 const fetchHistory = (Last = '123') =>
    axios({
      url: '/api/getHistory',
      method: 'post',
      data: { Last },
    })

console log in the api of the data is:
req.body in api:>> { Last: { queryKey: [ 'history' ], signal: {} } }

I'm still new to fetching data and working with it, fairly sure I am just misunderstanding the example and what lastPage, pages means.

here is a console log of the data in the app:

[TkDodo]

the result of getNextPageParam will be passed as pageParam to the queryFn as part of the queryFunctionContext. If your fetchHistory function only takes one arg, you need to unwrap it:

useInfiniteQuery(
  ['history'],
  ({ pageParam }) => fetchHistory(pageParam),
  {
    getNextPageParam: (lastPage, pages) => lastPage.LastEvaluatedKey,
  }
)

[rdyar]

yes! just figured that out a minute ago, I think I am almost there.

[rdyar]

I got it working, thanks for your help!

Here is what my api code is (NextJS):

import { DynamoDB } from '@aws-sdk/client-dynamodb' // ES6 import
import { DynamoDBDocument } from '@aws-sdk/lib-dynamodb'
// Full DynamoDB Client
const client = new DynamoDB({
  credentials: {
    accessKeyId: process.env.ACCESS_KEY,
    secretAccessKey: process.env.SECRET_KEY,
  },
  region: process.env.REGION,
})
const ddbDocClient = DynamoDBDocument.from(client)

export default async function handler(req, res) {
  console.log('req.body in api:>> ', req.body)
  const startKey = req.body.startKey
  const params = {
    TableName: process.env.TABLE_NAME,
    KeyConditionExpression: 'typeTag = :typeTag',
    ExpressionAttributeValues: {
      ':typeTag': 'history',
    },
    ScanIndexForward: false,
    ReturnConsumedCapacity: 'TOTAL',
    Limit: 50,
  }
  if (startKey) {
    params['ExclusiveStartKey'] = startKey
  }
  console.log('params in api', params)
  let data
  try {
    data = await ddbDocClient.query(params)
    console.log('data :>> ', data)
  } catch (e) {
    console.error('error in api', e.message)
  }

  return res.status(200).json(data)
}

and then I call it like:

const fetchHistory = ({ pageParam = '' }) =>
    axios({
      url: '/api/getHistory',
      method: 'post',
      data: { startKey: pageParam },
    })
  const {
    data,
    error,
    isLoading,
    isError,
    fetchNextPage,
    hasNextPage,
    isFetching,
    isFetchingNextPage,
    status,
  } = useInfiniteQuery('history', fetchHistory, {
    getNextPageParam: (lastPage) => lastPage.data.LastEvaluatedKey,
  })

as a newb trying to understand this: getNextPageParam: (lastPage, pages) => lastPage.LastEvaluatedKey, was a little difficult, I eventually did it as:

getNextPageParam: (lastPage, pages) => {
 console.log(lastPage)
 console.log(pages)

which helped me to see that I needed to use lastPage.data.LastEvaluatedKey not lastPage.LastEvaluatedKey