Delete old data automatically

[schlagmichdoch]

Another option that I would love is the ability for umami to auto-clear data after a week/month/other timeframe and only keep the recent amount of time. I primarily use umami for its real-time/short-term analytics and have no need for anything longer than that, therefore, would like to save space/database overhead in the long run with something like this.

Originally posted by @ncovercash in #292

As this request got buried in another issue that was already merged I'd like to come back to this and open a separate issue for this.

I too am only interested in data from the last 2 month.

I'd appreciate a possibility to automatically truncate all data that's older than a specified time range. This could be a simple cronjob that deletes old data once a day.

[mikecao]

This should be implemented as a cron job and is outside the scope of the app. The app has no concept of jobs.

[liuwenzhuang]

Maybe reset website data could specify time range, like 1 week age、1 month ago、1 year ago...

[FHU-yezi]

This should be implemented as a cron job and is outside the scope of the app. The app has no concept of jobs.

How about add a background worker and expose a config, letting user choose how long the data should be stored?

[schlagmichdoch]

Another idea would be to create a possibility to delete old data manually and expose that via a umami-cli or a local endpoint so admins can easily create a cronjob for that.
I imagine this to be similar to the current delete all data option (#752)

Similar issues: #599 #1277

[schlagmichdoch]

Also, please reopen this issue to discuss.

[schlagmichdoch]

Wouldn't it be similar to the reset function with the only difference that there would be a second where filter?

umami/src/queries/prisma/website.ts

Lines 158 to 190 in 1552a3f

	export async function resetWebsite(
	websiteId: string,
	): Promise<[Prisma.BatchPayload, Prisma.BatchPayload, Website]> {
	const { client, transaction } = prisma;
	const cloudMode = !!process.env.cloudMode;
	return transaction([
	client.eventData.deleteMany({
	where: { websiteId },
	}),
	client.sessionData.deleteMany({
	where: { websiteId },
	}),
	client.websiteEvent.deleteMany({
	where: { websiteId },
	}),
	client.session.deleteMany({
	where: { websiteId },
	}),
	client.website.update({
	where: { id: websiteId },
	data: {
	resetAt: new Date(),
	},
	}),
	]).then(async data => {
	if (cloudMode) {
	await redis.client.set(`website:${websiteId}`, data[3]);
	}
	return data;
	});
	}

Something like:

export async function resetWebsiteOlderTwoMonths(
  websiteId: string,
): Promise<[Prisma.BatchPayload, Prisma.BatchPayload, Website]> {
  const { client, transaction } = prisma;
  const cloudMode = !!process.env.cloudMode;

  const now = new Date();
  const twoMonthsAgo = new Date(now);
  twoMonthsAgo.setMonth(now.getMonth() - 2);

  return transaction([
    client.eventData.deleteMany({
      where: { 
        id: websiteId,
        lte: twoMonthsAgo
      },
    }),
    client.sessionData.deleteMany({
      where: { 
        id: websiteId,
        lte: twoMonthsAgo
      },
    }),
    client.websiteEvent.deleteMany({
      where: { 
        id: websiteId,
        lte: twoMonthsAgo
      },
    }),
    client.session.deleteMany({
      where: { 
        id: websiteId,
        lte: twoMonthsAgo
      },
    }),
    client.website.update({
      where: { id: websiteId },
      data: {
        resetAt: now,
      },
    }),
  ]).then(async data => {
    if (cloudMode) {
      await redis.client.set(`website:${websiteId}`, data[3]);
    }

    return data;
  });
}

If your open to such a option in the settings + cli endpoint I'd be happy to contribute a PR.