The biggest headache I have to deal is scale. The more users we have, the more machines we have to deploy, which means we need to pay more for these machines. I am committed to allowing people to use our platform for free at some capacity, like Character.AI does, but I also don't have millions and millions of dollars to burn to meet those user demands. This leaves me with three options. Sacrifice quality for speed or reduce what the free users can do on the site. Either option is going to make people unhappy. The final third option, which is the hardest one, and cannot be done immediately, is constantly optimising models to be smaller but keeping the same quality, and increasing the speed of our inference engine. This takes a lot of development work.
0 commit comments