nextjs-best-practice
- rate-limiter
collapsed:: true
packages
- rate-limit-redis - npm, https://www.npmjs.com/package/rate-limit-redis
/**
* 2. 检查频率等相关
* note:
* 1. 因为我们已经有了用户系统,所以不需要检查频率了
* 2. 这个基于 KV 的对于大陆来说,太慢了,真要做,可以使用本地的 redis,参考:rate-limit-redis - npm, https://www.npmjs.com/package/rate-limit-redis
*/
if (baseEnv.KV_REST_API_URL && baseEnv.KV_REST_API_TOKEN) {
const ip = req.headers.get("x-forwarded-for")
const ratelimit = new Ratelimit({
redis: kv,
// rate limit to 1 requests per 20 seconds
limiter: Ratelimit.slidingWindow(3, "10s"),
})
const { success, limit, reset, remaining } = await ratelimit.limit(`ratelimit_${ip}`)
if (!success) {
return NextResponse.json("您的速度太快啦,请慢点!", {
status: 429,
headers: {
"X-RateLimit-Limit": limit.toString(),
"X-RateLimit-Remaining": remaining.toString(),
"X-RateLimit-Reset": reset.toString(),
},
})
}
console.log("passed rate limiter")
}
- openai streaming
version management
- 教训:所以接第三方库真地还是谨慎,就算vercel这种公司的第三方库,也可能有bug,或者不满足需求,vercel的sdk里是一股脑塞messages给后端,但这样不但性能低,而且灵活性也不够,看chatgpt的前端实现,肯定是要维护一个conversation的,这样可以增量交互,不过chatgpt也鸡贼,它的网页访问速度那么快,除了基于edge之外,我猜就是做了暴力的rolling window,不然得数据库交互,就变慢了
- at the beginning
- after
yarn add langchain@latest
- after
yarn add ai@latest
import OpenAI from 'openai'
// nodejs + edge
const response = await new OpenAI({
apiKey: baseEnv.OPENAI_API_KEY,
timeout: 3000,
/**
*
* ref:
* - https://github.com/openai/openai-node/tree/v4#configuring-an-https-agent-eg-for-proxies
* - https://github.com/openai/openai-node/issues/85
* - https://www.npmjs.com/package/https-proxy-agent
*
* edge 环境中 不支持 http / https-proxy-agent 等库
*
* 大陆服务器需要开启 clash tun mode !
*/
httpAgent: undefined,
}).chat.completions.create({
model: modelType,
stream: true,
messages,
temperature: 0.7,
})