Skip to content

Latest commit

 

History

History
434 lines (317 loc) · 9.83 KB

12.md

File metadata and controls

434 lines (317 loc) · 9.83 KB

Build golang http server with docker

环境

  • Go: v1.10
  • Docker: v18.03
  • OS: CentOS 7.4

构建server代码

server.go

package main

import (
	"fmt"
	"net/http"
	"log"
)

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Println("path", r.URL.Path)
	fmt.Fprintf(w, "Hello there!")
}

func main() {
	http.HandleFunc("/", hello) 
	err := http.ListenAndServe(":9090", nil)
	if err != nil {
		log.Fatal("ListenAndServe: ", err)
	}
}

打包镜像

dockerfile:

FROM golang:1.10
RUN mkdir /app 
ADD . /app/ 
WORKDIR /app
RUN go build -o main . 
CMD ["/app/main"]

编译镜像：

docker build -t go-server:test .

测试

docker run --rm -p 9099:9099 go-server:test

访问 localhost:9099/tttt

输出：

path /tttt

依赖管理

由于国内下载 go 依赖很多不方便，这里使用 dep 做依赖管理，构建镜像时不需要每次下载依赖。

安装日志库 dep ensure --add github.com/sirupsen/logrus

修改代码：

package main

import (
	"net/http"

	log "github.com/sirupsen/logrus"
)

func hello(w http.ResponseWriter, r *http.Request) {
	log.Info("path", r.URL.Path)
	log.Info(w, "Hello there!")
}

func main() {
	http.HandleFunc("/", hello)
	err := http.ListenAndServe(":9099", nil)
	if err != nil {
		log.Fatal("ListenAndServe: ", err)
	}
}

构建包含dep的基础golang镜像

下载最新的 dep release

给下载好的dep-linux-amd64文件增加执行权限

dockerfile:

FROM golang:1.10

# Set go bin which doesn't appear to be set already.
ENV GOBIN /go/bin
# GO dep
COPY dep-linux-amd64 /go/bin/dep

构建镜像 docker build -t mygolang:1.10 .

构建服务镜像

dockerfile:

FROM mygolang:1.10

# Build directories
RUN mkdir /app
RUN mkdir /go/src/app
ADD . /go/src/app
WORKDIR /go/src/app

# Install dependencies
RUN dep ensure

# Build my app
RUN go build -o /app/main .
CMD ["/app/main"]

构建镜像：

docker build -t golang-server:test .

测试镜像

运行：

docker run --rm golang-server:test

测试：

curl 127.0.0.1:9099

输出：

"level":"info","msg":"path/","time":"2018-11-07T05:18:18Z"}

Go和Nodejs和Python3并发测试

  • Go v1.10
  • Nodejs v8.11
  • Python v3.4

用abtest简单做一下测试，都写一个返回"hello there" 的 http server。

服务器 centos7 4c 16g

Nodejs 代码：

 const http = require("http");
 
 http.createServer(function (request, response) {
     response.writeHead(200, {'Content-Type': 'text/plain'});
     response.end('Hello there\n');
 }).listen(9092)

Go 代码：

package main

import (
	"fmt"
	"log"
	"net/http"
	"runtime"
)

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello there!")
}

func main() {
	runtime.set
	http.HandleFunc("/", hello)
	err := http.ListenAndServe(":9099", nil)
	if err != nil {
		log.Fatal("ListenAndServe: ", err)
	}
}

10w 请求 1k 并发测试：

Nodejs:

ab -c 1000 -n 100000 http://127.0.0.1:9092/
...
Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   10.652 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      11300000 bytes
HTML transferred:       1200000 bytes
Requests per second:    9388.20 [#/sec] (mean)
Time per request:       106.517 [ms] (mean)
Time per request:       0.107 [ms] (mean, across all concurrent requests)
Transfer rate:          1036.00 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   42 299.6      0    7016
Processing:    12   56  12.6     54     662
Waiting:        0   56  12.6     54     662
Total:         29   98 300.8     54    7068

Percentage of the requests served within a certain time (ms)
  50%     54
  66%     59
  75%     61
  80%     62
  90%     67
  95%     73
  98%   1055
  99%   1070
 100%   7068 (longest request)

Go:

ab -c 1000 -n 100000 http://127.0.0.1:9099/
...
Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   5.950 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      12900000 bytes
HTML transferred:       1200000 bytes
Requests per second:    16807.72 [#/sec] (mean)
Time per request:       59.496 [ms] (mean)
Time per request:       0.059 [ms] (mean, across all concurrent requests)
Transfer rate:          2117.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   27   5.9     27      45
Processing:     8   32   6.8     32      62
Waiting:        0   23   6.9     23      53
Total:         29   59   6.0     58      92

Percentage of the requests served within a certain time (ms)
  50%     58
  66%     61
  75%     63
  80%     64
  90%     67
  95%     69
  98%     73
  99%     76
 100%     92 (longest request)

Go 只用一个核：

runtime.GOMAXPROCS(1)

ab -c 1000 -n 100000 http://127.0.0.1:9099/
...
Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   6.420 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      12900000 bytes
HTML transferred:       1200000 bytes
Requests per second:    15577.49 [#/sec] (mean)
Time per request:       64.195 [ms] (mean)
Time per request:       0.064 [ms] (mean, across all concurrent requests)
Transfer rate:          1962.40 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   2.2      0      32
Processing:     4   63  24.0     75     111
Waiting:        0   63  24.0     75     105
Total:          4   64  23.7     75     111

Percentage of the requests served within a certain time (ms)
  50%     75
  66%     76
  75%     77
  80%     78
  90%     80
  95%     82
  98%     84
  99%     86
 100%    111 (longest request)

之后再增加到1w并发的时候，结果差的比较明显。这也只是个参考，并不够严谨。

Python3：


from http.server import BaseHTTPRequestHandler, HTTPServer

class testHTTPServer_RequestHandler(BaseHTTPRequestHandler):

  def do_GET(self):
    self.send_response(200)

    self.send_header('Content-type','text/html')
    self.end_headers()

    message = "Hello world!"
    self.wfile.write(bytes(message, "utf8"))
  def log_message(self, format, *args):
      return

def run():
  print('starting server...')

  server_address = ('127.0.0.1', 4477)
  httpd = HTTPServer(server_address, testHTTPServer_RequestHandler)
  print('running server...')
  httpd.serve_forever()


run()

ab -c 1000 -n 100000 http://127.0.0.1:4477/ 压测期间server半途直接报错不止。。

更新： Go1.11 module

环境: go 1.10 -> go 1.11.5

Go module 基础

可以通过环境变量 GO111MODULE 开启或关闭module特性，它有三个可选值：off、on、auto，默认值是 auto。go modules 下载的包在 GOPATH/pkg/mod, 安装的可执行文件仍在 GOPATH/bin

  • GO111MODULE=off 无模块支持，go 会从 GOPATH 和 vendor 文件夹寻找包。
  • GO111MODULE=on 模块支持，go 会忽略 GOPATH 和 vendor 文件夹，只根据 go.mod 下载依赖。
  • GO111MODULE=auto 在 $GOPATH/src 外面且根目录有 go.mod 文件时，开启模块支持。

go mod 使用

download    download modules to local cache
edit        edit go.mod from tools or scripts
graph       print module requirement graph
init        initialize new module in current directory
tidy        add missing and remove unused modules
vendor      make vendored copy of dependencies
verify      verify dependencies have expected content
why         explain why packages or modules are needed

直接运行 go build 命令也会整理并更新go.mod文件

go build -mod=vendor 则是使用项目根目录的vendor目录 go build -mod=readonly 任何会导致依赖关系变动的情况都将导致build失败

将项目从Go1.10 vendor 迁移到 Go1.11 module

由于需要编译成docker镜像，在项目中vendor目录存放依赖，这样编译的时候不需要下载，对编译环境要求低。坏处就是如果更新了依赖每次都要使用go mod vendor命令同步到vendor目录。(注意原来来的glide godep等的vendor目录不能当作go module的vendor用，需要删掉重新同步)

将项目从GOPATH目录拷贝出去,删除vendor目录,main.go 增加声明

package main // import "bi-material-collector"

执行 go mod init

增加缺失的包，移除没用的包 go mod tidy

Dockerfile:

FROM golang:1.11.5

# Build directories
RUN mkdir /app
ADD . /app
WORKDIR /app

# Build use vendor modules
RUN go build -mod=vendor -o /app/main .
CMD ["/app/main"]