Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions db/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@
},
"dependencies": {
"@filecoin-station/spark-evaluate": "^1.2.0",
"pg": "^8.13.3",
"pg": "^8.14.0",
"postgrator": "^8.0.0"
},
"standard": {
"env": [
"mocha"
]
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @Goddhi

44 changes: 29 additions & 15 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 24 additions & 5 deletions stats/bin/migrate.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,28 @@
import {
getPgPools,
migrateEvaluateDB,
migrateStatsDB
migrateStatsDB,
getPgPools
} from '@filecoin-station/spark-stats-db'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please undo this change to keep the diff clean

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, noted

const pgPools = await getPgPools()
await migrateStatsDB(pgPools.stats)
await migrateEvaluateDB(pgPools.evaluate)
const {
DATABASE_URL,
EVALUATE_DB_URL
} = process.env
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const {
DATABASE_URL,
EVALUATE_DB_URL
} = process.env

These constants seem not to be used anywhere. Let's delete them.


try {
console.log('Running migrations for stats database...')
const pgPools = await getPgPools()

// @ts-ignore - PgPoolStats actually does have a query method at runtime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try to add correct types to migrate functions rather then using @ts-ignore. Using it should be avoided.

await migrateStatsDB(pgPools.stats)

// @ts-ignore - Similarly for evaluate
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise, let's please not use @ts-ignore

await migrateEvaluateDB(pgPools.evaluate)

await pgPools.end()

console.log('All migrations completed successfully')
} catch (error) {
console.error('Migration failed:', error)
process.exit(1)
}
15 changes: 9 additions & 6 deletions stats/bin/spark-stats.js
Original file line number Diff line number Diff line change
@@ -1,23 +1,26 @@
import '../lib/instrument.js'
import { createApp } from '../lib/app.js'
import { getPgPools } from '@filecoin-station/spark-stats-db'

const {
PORT = '8080',
HOST = '127.0.0.1',
SPARK_API_BASE_URL = 'https://api.filspark.com/',
REQUEST_LOGGING = 'true'
REQUEST_LOGGING = 'true',
DATABASE_URL,
EVALUATE_DB_URL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add default values for both env variables:

Suggested change
DATABASE_URL,
EVALUATE_DB_URL
DATABASE_URL = 'postgres://localhost:5432/spark_stats',
EVALUATE_DB_URL = 'postgres://localhost:5432/spark_evaluate'

} = process.env

const pgPools = await getPgPools()

const app = await createApp({
SPARK_API_BASE_URL,
pgPools,
DATABASE_URL,
EVALUATE_DB_URL,
logger: {
level: ['1', 'true'].includes(REQUEST_LOGGING) ? 'info' : 'error'
}
})

console.log('Starting the http server on host %j port %s', HOST, PORT)
const baseUrl = app.listen({ port: Number(PORT), host: HOST })
console.log(baseUrl)
await app.listen({ port: Number(PORT), host: HOST })
console.log(`Server listening at ${HOST}:${PORT}`)

33 changes: 27 additions & 6 deletions stats/lib/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import * as Sentry from '@sentry/node'
import Fastify from 'fastify'
import cors from '@fastify/cors'
import urlData from '@fastify/url-data'
import fastifyPostgres from '@fastify/postgres'

import { addRoutes } from './routes.js'
import { addPlatformRoutes } from './platform-routes.js'
Expand All @@ -12,18 +13,38 @@ import { addPlatformRoutes } from './platform-routes.js'
/**
* @param {object} args
* @param {string} args.SPARK_API_BASE_URL
* @param {import('@filecoin-station/spark-stats-db').PgPools} args.pgPools
* @param {Fastify.FastifyLoggerOptions} args.logger
* @returns
* @param {string} args.DATABASE_URL - Connection string for stats database
* @param {string} args.EVALUATE_DB_URL - Connection string for evaluate database
* @param {import('fastify').FastifyLoggerOptions} args.logger
* @returns {Promise<import('fastify').FastifyInstance>}
*/
export const createApp = ({
export const createApp = async ({
SPARK_API_BASE_URL,
pgPools,
DATABASE_URL,
EVALUATE_DB_URL,
logger
}) => {
const app = Fastify({ logger })
Sentry.setupFastifyErrorHandler(app)

await app.register(fastifyPostgres, {
connectionString: DATABASE_URL,
name: 'stats'
})

await app.register(fastifyPostgres, {
connectionString: EVALUATE_DB_URL,
name: 'evaluate',
})

const pgPools = {
stats: app.pg.stats,
evaluate: app.pg.evaluate,
async end() {
await app.close()
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const pgPools = {
stats: app.pg.stats,
evaluate: app.pg.evaluate,
async end() {
await app.close()
}
}

I don't see this being used anywhere. If it's not used, let's removed. YAGNI

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is being passed to addRoutes() and addPlatformRoutes() below. However, instead of passing pgPools, the route handlers should access app.pg.stats etc. directly


app.register(cors, {
origin: [
'http://localhost:3000',
Expand All @@ -40,4 +61,4 @@ export const createApp = ({
})

return app
}
}
1 change: 1 addition & 0 deletions stats/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
"dependencies": {
"@fastify/cors": "^11.0.0",
"@fastify/url-data": "^6.0.3",
"@fastify/postgres": "^5.2.0",
"@filecoin-station/spark-stats-db": "^1.0.0",
"@sentry/node": "^9.5.0",
"@sentry/profiling-node": "^9.5.0",
Expand Down