用Golang 处理每分钟100万个请求
🕘 Fri, 19 Jan 2018 by GolangNote
用渠道作为线程池,应对百万请求,这是 Marcio Castilho 优化的方法。核心代码:
var (
MaxWorker = os.Getenv("MAX_WORKERS")
MaxQueue = os.Getenv("MAX_QUEUE")
)
//Job represents the job to be run
type Job struct {
Payload Payload
}
// A buffered channel that we can send work requests on.
var JobQueue chan Job
// Worker represents the worker that executes the job
type Worker struct {
WorkerPool chan chan Job
JobChannel chan Job
quit chan bool
}
func NewWorker(workerPool chan chan Job) Worker {
return Worker{
WorkerPool: workerPool,
JobChannel: make(chan Job),
quit: make(chan bool)}
}
// Start method starts the run loop for the worker, listening for a quit channel in
// case we need to stop it
func (w Worker) Start() {
go func() {
for {
// register the current worker into the worker queue.
w.WorkerPool <- w.JobChannel
select {
case job := <-w.JobChannel:
// we have received a work request.
if err := job.Payload.UploadToS3(); err != nil {
log.Errorf("Error uploading to S3: %s", err.Error())
}
case <-w.quit:
// we have received a signal to stop
return
}
}
}()
}
// Stop signals the worker to stop listening for work requests.
func (w Worker) Stop() {
go func() {
w.quit <- true
}()
}
//Handler
func payloadHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != "POST" {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
// Read the body into a string for json decoding
var content = &PayloadCollection{}
err := json.NewDecoder(io.LimitReader(r.Body, MaxLength)).Decode(&content)
if err != nil {
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.WriteHeader(http.StatusBadRequest)
return
}
// Go through each payload and queue items individually to be posted to S3
for _, payload := range content.Payloads {
// let's create a job with the payload
work := Job{Payload: payload}
// Push the work onto the queue.
JobQueue <- work
}
w.WriteHeader(http.StatusOK)
}
参见他的blog https://medium.com/smsjunk/handling-1-million-requests-per-minute-with-golang-f70ac505fcaa 34
一个线程池库 https://github.com/shettyh/threadpool 31
本文网址: https://golangnote.com/topic/223.html (转载注明出处)
关于GolangNote:记录在工作中使用golang 遇到、面临的相关问题及解决方法。如果你在这里获得一些知识或信息,解决你的编程问题,请考虑捐赠给不幸的人或者你喜欢的慈善机构,除捐赠外,种植树木、志愿服务或减少排碳的行为也很有益处。如果你有任何问题可以在下面 留言
Be the first to comment!
Recent Go Articles
- Golang 把cookie 字符串解析为cookie 结构
- Golang 计算字符串中包含某个或某些字符集的个数
- 使用Golang 对文件增删写读操作备忘
- Go Modules 使用备忘
- 使用Golang 简单删除图片exif 信息
- 谷歌翻译的 golang 库推荐
- Go 1.13.2 与1.13.3 紧急更新
- golang 人脸检测识别库
- Go build 错误 “stackcheck redeclared in this block previous declaration”的解决方法
- Golang phantomjs 动态代理实现
- Golang chrome debug protocol 库推荐
- Golang 随机打乱数组/Slice
- Golang sync.WaitGroup 的 Wait 超时处理
- Golang实现简单的Socks5代理
- Golang 用snappy + Base64 简单压缩加密进行网络传输
- Golang http IPv4/IPv6 服务
- golang 全角半角相互转换
- 在自己的网站部署TLS 1.3
- Golang 实现/打印菜单树
- Golang telegram 机器人小试
Top Go Articles
- golang 当前时间、时间戳、时区的相互转化
- golang map 按value 大小排序,降序和升序
- golang regexp 正则表达式常见的用法
- golang 正确获取绝对路径的方法
- Golang 定时循环的实现
- golang 把上传文件转为byte
- Golang 时区时差处理方式
- golang 用正则包regexp 通过user-agent 识别手机浏览器
- GoLang 正则判断字符是不是中文
- golang 生成良好的唯一ID/uuid库比较
- Golang 把cookie 字符串解析为cookie 结构
- Golang 获取两个数之间的随机数
- golang 用gzip 压缩、解压缩字符串
- GoLang 计算小文件和大文件 md5 值的例子
- Golang Web 自动安装、更新Let’s Encrypt 免费证书
- go 获取硬盘的可用空间的方法
- golang 人脸检测识别库
- Golang 生成防识别的图片验证码
- Go build 错误 “stackcheck redeclared in this block previous declaration”的解决方法
- GoLang 实现python timedelta 函数功能