PlayWithDocker-应用程序容器化和微服务编排

引用

• https://training.play-with-docker.com/microservice-orchestration/

教程说明

本教程使用Docker compose编排服务
教程示例服务仓库 Link Extractor 链接信息提取
git clone https://github.com/ibnesayeed/linkextractor.git
包含三个服务:

• PHP编写的运行在Apache服务上的web程序,用于输入链接和展示提取信息
• python(和ruby)编写的API服务,提供链接解析的JSON响应
• Redis缓存,用于保存已解析链接的信息的缓存
  服务架构如下图
  
  本教程的最初来源 https://ws-dl.blogspot.com/2017/12/2017-12-03-introducing-docker.html

第0步: 链接提取脚本

git clone https://github.com/ibnesayeed/linkextractor.git
cd linkextractor
#切换分支
git checkout step0
#查看目录
tree
.
├── README.md
└── linkextractor.py
#执行下脚本
$ cat linkextractor.py
#!/usr/bin/env python

import sys
import requests
from bs4 import BeautifulSoup

res = requests.get(sys.argv[-1])
soup = BeautifulSoup(res.text, "html.parser")
for link in soup.find_all("a"):
    print(link.get("href"))
$ chmod +x ./linkextractor.py
$ ./linkextractor.py https://training.play-with-docker.com/microservice-orchestration/
Traceback (most recent call last):
  File "./linkextractor.py", line 5, in <module>
    from bs4 import BeautifulSoup
ImportError: No module named bs4

可以发现,这个脚本并不能顺利的执行:

• 脚本是可执行文件吗?
• Python环境是否安装?
• 可以安装软件到这个机器吗?
• pip软件是否有安装?
• 依赖模块request和bs4是否有安装?
  这一些列的问题决定了能否正常运行这个脚本程序,这正是Docker容器的用处,下面步骤封装这个脚本使其可以简单执行.

第1步: 容器化链接提取脚本

切换分支,构建容器

$ git checkout step1

$ tree
.
├── Dockerfile
├── README.md
└── linkextractor.py

0 directories, 3 files

$ cat Dockerfile
FROM python:3
LABEL maintainer="Sawood Alam <@ibnesayeed>"

RUN pip install beautifulsoup4
RUN pip install requests

WORKDIR /app
COPY linkextractor.py /app/
RUN chmod a+x linkextractor.py

ENTRYPOINT ["./linkextractor.py"]

这个Dockerfile文件:

• python:3基础镜像构建
• LABEL maintainer设定维护者
• RUN运行安装依赖模块beautifulsoup4,requests
• WORKDIR设定构建的工作目录
• COPY复制本机脚本到镜像目录/app/
• 运行设置脚本权限可执行
• ENTRYPOINT设定启动入口指令./linkextractor.py

现在开始构建镜像,运行容器

$ docker image build -t linkextractor:step1 .

$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
linkextractor step1 679c38f239dc 6 seconds ago 943MB
python 3 b55669b4130e 33 hours ago 933MB

$ docker container run -it --rm linkextractor:step1 https://training.play-with-docker.com/microservice-orchestration/
/
/about/
https://docs.docker.com/compose/
/beginner-linux
https://github.com/ibnesayeed/linkextractor
https://odu.edu/compsci
https://www.odu.edu/
https://www.youtube.com/watch?v=Y_X0F2FgYm8
https://www.slideshare.net/ibnesayeed/introducing-docker-application-containerization-service-orchestration
https://ws-dl.blogspot.com/2017/12/2017-12-03-introducing-docker.html
#stage-setup
#step-0-basic-link-extractor-script
#step-1-containerized-link-extractor-script
#step-2-link-extractor-module-with-full-uri-and-anchor-text
#step-3-link-extractor-api-service
#step-4-link-extractor-api-and-web-front-end-services
#step-5-redis-service-for-caching
#step-6-swap-python-api-service-with-ruby
#conclusions
http://example.com/
/
/
/
https://redis.io/commands/monitor
/
/swarm-stack-intro
/swarm-stack-intro
//www.slideshare.net/ibnesayeed/introducing-docker-application-containerization-service-orchestration
//www.slideshare.net/ibnesayeed
https://twitter.com/intent/tweet?text=Application Containerization and Microservice Orchestration&url=https://training.play-with-docker.com/microservice-orchestration/&via=docker&related=docker
https://facebook.com/sharer.php?u=https://training.play-with-docker.com/microservice-orchestration/
https://plus.google.com/share?url=https://training.play-with-docker.com/microservice-orchestration/
http://www.linkedin.com/shareArticle?mini=true&url=https://training.play-with-docker.com/microservice-orchestration/&title=Play%20with%20Docker%20Classroom&source=https://training.play-with-docker.com
https://www.docker.com/legal/docker-terms-service
https://www.docker.com
https://www.facebook.com/docker.run
https://twitter.com/docker
https://www.github.com/play-with-docker/play-with-docker.github.io

第2步: 修改脚本使其能输出完整的URL路径

$ git checkout step2
$ cat linkextractor.py

linkextractor.py

#!/usr/bin/env python

import sys
import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin

def extract_links(url):
    res = requests.get(url)
    soup = BeautifulSoup(res.text, "html.parser")
    base = url
    # TODO: Update base if a <base> element is present with the href attribute
    links = []
    for link in soup.find_all("a"):
        links.append({
            "text": " ".join(link.text.split()) or "[IMG]",
            "href": urljoin(base, link.get("href"))
        })
    return links

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("\nUsage:\n\t{} <URL>\n".format(sys.argv[0]))
        sys.exit(1)
    for link in extract_links(sys.argv[-1]):
        print("[{}]({})".format(link["text"], link["href"]))

构建镜像,再次运行

$ docker image build -t linkextractor:step2 .
Sending build context to Docker daemon 107.5kB
Step 1/8 : FROM python:3
 ---> b55669b4130e
Step 2/8 : LABEL maintainer="Sawood Alam <@ibnesayeed>"
 ---> Using cache
 ---> f7b887bf5047
Step 3/8 : RUN pip install beautifulsoup4
 ---> Using cache
 ---> 222608bb81da
Step 4/8 : RUN pip install requests
 ---> Using cache
 ---> cdf5a25a6028
Step 5/8 : WORKDIR /app
 ---> Using cache
 ---> 39d206b932a7
Step 6/8 : COPY linkextractor.py /app/
 ---> 42a1a7aae8fa
Step 7/8 : RUN chmod a+x linkextractor.py
 ---> Running in a59a288da86e
Removing intermediate container a59a288da86e
 ---> 89527192fc48
Step 8/8 : ENTRYPOINT ["./linkextractor.py"]
 ---> Running in 573392b0fc70
Removing intermediate container 573392b0fc70
 ---> b2816ba933b7
Successfully built b2816ba933b7
Successfully tagged linkextractor:step2

$ docker container run -it --rm linkextractor:step2 https://training.play-with-docker.com/microservice-orchestration/
[Play with Docker classroom](https://training.play-with-docker.com/)
[About](https://training.play-with-docker.com/about/)
[Docker Compose](https://docs.docker.com/compose/)
[Docker for Beginners](https://training.play-with-docker.com/beginner-linux)
[Link Extractor](https://github.com/ibnesayeed/linkextractor)
[Computer Science Department](https://odu.edu/compsci)
[Old Dominion University](https://www.odu.edu/)
[video recording](https://www.youtube.com/watch?v=Y_X0F2FgYm8)
[presentation slides](https://www.slideshare.net/ibnesayeed/introducing-docker-application-containerization-service-orchestration)
[a blog post](https://ws-dl.blogspot.com/2017/12/2017-12-03-introducing-docker.html)
[Stage Setup](https://training.play-with-docker.com/microservice-orchestration/#stage-setup)
[Step 0: Basic Link Extractor Script](https://training.play-with-docker.com/microservice-orchestration/#step-0-basic-link-extractor-script)
[Step 1: Containerized Link Extractor Script](https://training.play-with-docker.com/microservice-orchestration/#step-1-containerized-link-extractor-script)
[Step 2: Link Extractor Module with Full URI and Anchor Text](https://training.play-with-docker.com/microservice-orchestration/#step-2-link-extractor-module-with-full-uri-and-anchor-text)
[Step 3: Link Extractor API Service](https://training.play-with-docker.com/microservice-orchestration/#step-3-link-extractor-api-service)
[Step 4: Link Extractor API and Web Front End Services](https://training.play-with-docker.com/microservice-orchestration/#step-4-link-extractor-api-and-web-front-end-services)
[Step 5: Redis Service for Caching](https://training.play-with-docker.com/microservice-orchestration/#step-5-redis-service-for-caching)
[Step 6: Swap Python API Service with Ruby](https://training.play-with-docker.com/microservice-orchestration/#step-6-swap-python-api-service-with-ruby)
[Conclusions](https://training.play-with-docker.com/microservice-orchestration/#conclusions)
[example.com](http://example.com/)
[click to open the Link Extractor](https://training.play-with-docker.com/)
[clicking here](https://training.play-with-docker.com/)
[clicking the Link Extractor](https://training.play-with-docker.com/)
[monitor](https://redis.io/commands/monitor)
[clicking the Link Extractor](https://training.play-with-docker.com/)
[Docker Swarm Cluster](https://training.play-with-docker.com/swarm-stack-intro)
[Docker Swarm Cluster](https://training.play-with-docker.com/swarm-stack-intro)
[Introducing Docker - Application Containerization & Service Orchestration](https://www.slideshare.net/ibnesayeed/introducing-docker-application-containerization-service-orchestration)
[Sawood Alam](https://www.slideshare.net/ibnesayeed)
[[IMG]](https://twitter.com/intent/tweet?text=Application Containerization and Microservice Orchestration&url=https://training.play-with-docker.com/microservice-orchestration/&via=docker&related=docker)
[[IMG]](https://facebook.com/sharer.php?u=https://training.play-with-docker.com/microservice-orchestration/)
[[IMG]](https://plus.google.com/share?url=https://training.play-with-docker.com/microservice-orchestration/)
[[IMG]](http://www.linkedin.com/shareArticle?mini=true&url=https://training.play-with-docker.com/microservice-orchestration/&title=Play%20with%20Docker%20Classroom&source=https://training.play-with-docker.com)
[here](https://www.docker.com/legal/docker-terms-service)
[[IMG]](https://www.docker.com)
[[IMG]](https://www.facebook.com/docker.run)
[[IMG]](https://twitter.com/docker)
[[IMG]](https://www.github.com/play-with-docker/play-with-docker.github.io)

第3步: 链接提取API服务

通过以下步骤构建一个WEB访问的容器服务:

• 添加一个服务脚本main.py提供WEB访问的链接提取功能
• 在Dockerfile中应用main.py脚本
• 将需要安装的依赖模块移动到requirements.txt文件中
• 映射端口,在容器外部访问服务

$ git checkout step3
$ tree
.
├── Dockerfile
├── README.md
├── linkextractor.py
├── main.py
└── requirements.txt

0 directories, 5 files

$ cat requirements.txt
beautifulsoup4
flask
requests

$ cat Dockerfile
FROM python:3
LABEL maintainer="Sawood Alam <@ibnesayeed>"

WORKDIR /app
COPY requirements.txt /app/
RUN pip install -r requirements.txt

COPY *.py /app/
RUN chmod a+x *.py

CMD ["./main.py"]

• main.py

#!/usr/bin/env python

from flask import Flask
from flask import request
from flask import jsonify
from linkextractor import extract_links

app = Flask(__name__)

@app.route("/")
def index():
    return "Usage: http://<hostname>[:<prt>]/api/<url>"

@app.route("/api/<path:url>")
def api(url):
    qs = request.query_string.decode("utf-8")
    if qs != "":
        url += "?" + qs
    links = extract_links(url)
    return jsonify(links)

app.run(host="0.0.0.0")

创建新的镜像,运行容器,测试链接

$ docker image build -t linkextractor:step3 .
...
$ docker container run -d -p 5000:5000 --name=linkextractor linkextractor:step3
46136d5339df0a3a69e4ab53fde02dbf1c6970d07cef17d9fb40e50702af7c2f
$ curl -i http://localhost:5000/api/http://example.com/
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 79
Server: Werkzeug/1.0.1 Python/3.8.2
Date: Sat, 18 Apr 2020 11:08:20 GMT

[{"href":"https://www.iana.org/domains/example","text":"More information..."}]

-d:--detach,后台运行容器

查看容器日志

$ docker container logs linkextractor
 * Serving Flask app "main" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
172.17.0.1 - - [18/Apr/2020 11:08:06] "GET /api/https://training.play-with-docker.com HTTP/1.1" 200 -
172.17.0.1 - - [18/Apr/2020 11:08:20] "GET /api/http://example.com/ HTTP/1.1" 200 -

$ docker container rm -f linkextractor
linkextractor

到此,成功的部署了一个API 服务,监听端口5000,下一步创建一个有界面的WEB前端服务

第4步: 链接提取API和WEB前端服务

查看第4步的文件

$ git checkout step4
Branch ‘step4‘ set up to track remote branch ‘step4‘ from ‘origin‘.
Switched to a new branch ‘step4‘

$ tree
.
├── README.md
├── api
│ ├── Dockerfile
│ ├── linkextractor.py
│ ├── main.py
│ └── requirements.txt
├── docker-compose.yml
└── www
    └── index.php

2 directories, 7 files

这一步的变化:

• 将上一步的API服务文件移动到了api文件夹下
• 一个PHP编写的前端程序放在./www目录下,用于和JSON API服务通信
• PHP应用会挂载到官方镜像php:7-apache下,便于开发阶段部署
• 环境变量API_ENDPOINT用于设置JSON API服务地址
• docker-compose.yaml用于构建包含多个容器组件的服务部署
  这里Docker compose可以通过名称服务让两个容器在一个网路内通信

$ cat docker-compose.yml

version: ‘3‘

services:
  api:
    image: linkextractor-api:step4-python
    build: ./api
    ports:
      - "5000:5000"
  web:
    image: php:7-apache
    ports:
      - "80:80"
    environment:
      - API_ENDPOINT=http://api:5000/api/
    volumes:
      - ./www:/var/www/html

这个配置文件创建了两个服务api和web
api服务使用镜像linkextractor-api:step4-python构建,暴露5000端口用于通信
web服务使用镜像php:7-apache构建,挂载卷本机./www目录到容器的/var/www/html目录(apache服务的默认web根目录).环境变量API_ENDPOINT用于提供PHP此程序访问API服务的地址

$ cat www/index.php
···
```php
<!DOCTYPE html>

<?php
  $api_endpoint = $_ENV["API_ENDPOINT"] ?: "http://localhost:5000/api/";
  $url = "";
  if(isset($_GET["url"]) && $_GET["url"] != "") {
    $url = $_GET["url"];
    $json = @file_get_contents($api_endpoint . $url);
    if($json == false) {
      $err = "Something is wrong with the URL: " . $url;
    } else {
      $links = json_decode($json, true);
      $domains = [];
      foreach($links as $link) {
        array_push($domains, parse_url($link["href"], PHP_URL_HOST));
      }
      $domainct = @array_count_values($domains);
      arsort($domainct);
    }
  }
?>
...

让我们使用docker-compose实用程序以分离模式打开这些服务:

$ docker-compose up -d --build
Creating network "linkextractor_default" with the default driver
...
Creating linkextractor_api_1 ... done
Creating linkextractor_web_1 ... done

$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
linkextractor-api step4-python 8cf9bc48ddd6 9 seconds ago 947MB
linkextractor step3 fd0f4f30a0b9 25 minutes ago 947MB
php 7-apache 470476c8d529 24 hours ago 414MB
python 3 b55669b4130e 41 hours ago 933MB

查看运行的容器:

$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d6a60ee72fc linkextractor-api:step4-python "./main.py" About a minute ago Up58 seconds 0.0.0.0:5000->5000/tcp linkextractor_api_1
d7f0c356c29e php:7-apache "docker-php-entrypoi…" About a minute ago Up58 seconds 0.0.0.0:80->80/tcp linkextractor_web_1

使用浏览器打开实例

现在修改./www/index.php文件,刷新浏览器可以看到标题和页脚都有变化

sed -i ‘s/Link Extractor/Super Link Extractor/g‘ www/index.php

还原变更

$ git reset --hard
HEAD is now at 2a3ec3e Synchronize branch step4

进行下一步前,先关闭服务

$ docker-compose down
Stopping linkextractor_api_1 ... done
Stopping linkextractor_web_1 ... done
Removing linkextractor_api_1 ... done
Removing linkextractor_web_1 ... done
Removing network linkextractor_default

第5步: 使用Redis缓存服务

切换到第五步的分支查看文件

$ git checkou step5
$ tree
.
├── README.md
├── api
│ ├── Dockerfile
│ ├── linkextractor.py
│ ├── main.py
│ └── requirements.txt
├── docker-compose.yml
└── www
    ├── Dockerfile
    └── index.php

变化之处:

• PHP web程序./www目录下添加了独立的Dockerfile文件,使用自包含来避免挂载本机活动文件
• 添加了一个Redis容器镜像
• api服务使用redis来缓存已经解析过的链接
• REDIS_URL环境变量用于配置api服务的redis连接地址

$ cat ./www/Dockerfile
FROM php:7-apache
LABEL maintainer="Sawood Alam <@ibnesayeed>"

ENV API_ENDPOINT="http://localhost:5000/api/"

COPY . /var/www/html/

$ cat api/main.py

#!/usr/bin/env python

import os
import json
import redis
from flask import Flask
from flask import request
from linkextractor import extract_links

app = Flask(__name__)
redis_conn = redis.from_url(os.getenv("REDIS_URL", "redis://localhost:6379"))

@app.route("/")
def index():
    return "Usage: http://<hostname>[:<prt>]/api/<url>"

@app.route("/api/<path:url>")
def api(url):
    qs = request.query_string.decode("utf-8")
    if qs != "":
        url += "?" + qs

    jsonlinks = redis_conn.get(url)
    if not jsonlinks:
        links = extract_links(url)
        jsonlinks = json.dumps(links, indent=2)
        redis_conn.set(url, jsonlinks)

    response = app.response_class(
        status=200,
        mimetype="application/json",
        response=jsonlinks
    )

    return response

app.run(host="0.0.0.0")

$ cat api/requirements.txt
beautifulsoup4
flask
redis
requests

[node1] (local) root@192.168.0.13 ~/linkextractor
$ cat docker-compose.yml
version: ‘3‘

services:
  api:
    image: linkextractor-api:step5-python
    build: ./api
    ports:
      - "5000:5000"
    environment:
      - REDIS_URL=redis://redis:6379
  web:
    image: linkextractor-web:step5-php
    build: ./www
    ports:
      - "80:80"
    environment:
      - API_ENDPOINT=http://api:5000/api/
  redis:
    image: redis

现在构建并启动这些服务:

$ docker-compose up -d --build
...
###
$ docker-compose down

我们已经成功地编排了三个微服务来组成我们的链接提取器应用程序. 现在, 我们有了一个应用程序堆栈, 它表示本教程介绍中所示的图中所示的体系结构. 在下一个步骤中, 我们将探索使用微服务体系结构来交换应用程序中的组件是多么容易.

第6步: 使用Ruby语言替换Python API服务

$ git checkout step6
$ tree
.
├── README.md
├── api
│ ├── Dockerfile
│ ├── Gemfile
│ └── linkextractor.rb
├── docker-compose.yml
├── logs
└── www
    ├── Dockerfile
    └── index.php

3 directories, 7 files

变化:

• api服务使用ruby实现
• API_ENDPOINT环境变量更新为指向Ruby的API服务
• 链接提取程序的缓存事件(HIT/MISS)日志将被持久化到数据卷

$ cat api/linkextractor.rb

#!/usr/bin/env ruby
# encoding: utf-8

require "sinatra"
require "open-uri"
require "uri"
require "nokogiri"
require "json"
require "redis"

set :protection, :except=>:path_traversal

redis = Redis.new(url: ENV["REDIS_URL"] || "redis://localhost:6379")

Dir.mkdir("logs") unless Dir.exist?("logs")
cache_log = File.new("logs/extraction.log", "a")

get "/" do
  "Usage: http://<hostname>[:<prt>]/api/<url>"
end

get "/api/*" do
  url = [params[‘splat‘].first, request.query_string].reject(&:empty?).join("?")
  cache_status = "HIT"
  jsonlinks = redis.get(url)
  if jsonlinks.nil?
    cache_status = "MISS"
    jsonlinks = JSON.pretty_generate(extract_links(url))
    redis.set(url, jsonlinks)
  end

  cache_log.puts "#{Time.now.to_i}\t#{cache_status}\t#{url}"

  status 200
  headers "content-type" => "application/json"
  body jsonlinks
end

def extract_links(url)
  links = []
  doc = Nokogiri::HTML(open(url))
  doc.css("a").each do |link|
    text = link.text.strip.split.join(" ")
    begin
      links.push({
        text: text.empty? ? "[IMG]" : text,
        href: URI.join(url, link["href"])
      })
    rescue
    end
  end
  links
end

$ cat api/Dockerfile
FROM ruby:2.6
LABEL maintainer="Sawood Alam <@ibnesayeed>"

ENV LANG C.UTF-8
ENV REDIS_URL="redis://localhost:6379"

WORKDIR /app
COPY Gemfile /app/
RUN bundle install

COPY linkextractor.rb /app/
RUN chmod a+x linkextractor.rb

CMD ["./linkextractor.rb", "-o", "0.0.0.0"]

$ cat docker-compose.yml
version: ‘3‘

services:
  api:
    image: linkextractor-api:step6-ruby
    build: ./api
    ports:
      - "4567:4567"
    environment:
      - REDIS_URL=redis://redis:6379
    volumes:
      - ./logs:/app/logs
  web:
    image: linkextractor-web:step6-php
    build: ./www
    ports:
      - "80:80"
    environment:
      - API_ENDPOINT=http://api:4567/api/
  redis:
    image: redis

注意:这里使用了volumes指令挂载本地日志目录./logs:/app/logs
重新构建并启动服务

$ docker-compose up -d --build

测试程序并查看服务日志:

$ docker-compose down
Stopping linkextractor_redis_1 ... done
Stopping linkextractor_api_1 ... done
Stopping linkextractor_web_1 ... done
Removing linkextractor_redis_1 ... done
Removing linkextractor_api_1 ... done
Removing linkextractor_web_1 ... done
Removing network linkextractor_default


$ cat logs/extraction.log
1587270250 MISS https://training.play-with-docker.com
1587270264 MISS http://ip172-18-0-29-bqddrkaosm4g00a1vs90-80.direct.labs.play-with-docker.com/
1587270271 HIT https://training.play-with-docker.com
1587270344 HIT https://training.play-with-docker.com
1587270385 HIT https://training.play-with-docker.com
1587270405 MISS http://example.com/

总结

这个示例演示了如下特性:

• 容器化应用程序使其能简单运行
• 编排多个服务程序运行/终止
• 使用挂载数据卷持久化数据

PlayWithDocker-应用程序容器化和微服务编排

原文：https://www.cnblogs.com/onion94/p/12731032.html