K8S中使用Ingress時的小技巧 - 網安 - 專業的網絡安全產業、社區、知識平臺

背景

在用k8s時，經常會用到ingress暴露服務。

ingress可以簡單地理解成類似nginx的反向代理，可以根據配置將流量路由到不同的后端。

在"云廠商國內虛機上的k8s集群"中使用ingress會遇到兩個小問題：

ingress配置中，必須使用域名，而不能使用ip
云廠商會檢查域名是否備案

一般有三種解決辦法：

買一個域名、給它備案、配置域名A記錄。整個過程感覺有一點點麻煩
用備案過的域名(比如 www.baidu.com )，機器綁定host來訪問
在香港、新加坡等虛機上部署服務，這樣似乎就不用備案

第二個方法已經很簡單了，自己用來測試挺好的。但是如果想要讓同事也訪問服務時，就需要讓同事也綁定host。

如果能讓"綁定host"這一步也省掉，就會更方便一點。

分析

nip.io域名是什么？

這種域名可以實現下面這種效果，x.x.x.x.nip.io域名的a記錄會被解析成x.x.x.x。

?  ~ ping 10.0.0.1.nip.io
PING 10.0.0.1.nip.io (10.0.0.1): 56 data bytes
...
?  ~ ping -nc 1 service.10.0.0.2.nip.io
PING service.10.0.0.2.nip.io (10.0.0.2): 56 data bytes
...

類似功能的域名還有sslip.io，這樣就可以省去"購買域名、配置A記錄"兩個步驟。

舉個例子，如果虛機ip是 1.2.3.4，我們就可以用 1.2.3.4.nip.io 當作ingress配置中的host字段。

因為nip.io這種域名是沒有備案的，所以訪問 http://1.2.3.4.nip.io 時，就有可能被云廠商禁止訪問。那廠商是怎么知道要封禁我這個請求呢？

怎么被封禁的呢？

猜測廠商可能是從請求的host字段拿域名，然后查詢是否有備案，如果沒有備案，就會禁止訪問。

所以如果我們用https加密，廠商就無法從http請求拿到域名。雖然可以tls握手包中獲取域名，但是廠商不一定實現。所以有可能通過https正常通信。

下面來看一看怎么在ingress中用上https。

怎么給ingress配置證書？

這里圖省事，我直接用kubernets集群中ca給kubelet簽發的證書。

因為ca是自建的、瀏覽器不信任的，所以瀏覽器訪問時會提示證書信任問題。

root@ip-172-31-14-33:~# cat /var/lib/kubelet/pki/kubelet-client-current.pem
-----BEGIN CERTIFICATE-----
MIIDKjCCAhKgAwIBAgIINVokOmwC4XUwDQYJKoZIhvcNAQELBQAwFTETMBEGA1UE
AxMKa3ViZXJuZXRlczAeFw0yMjA1MjUwMDMwMDVaFw0yMzA1MjUwMDMwMDZaMD0x
FTATBgNVBAoTDHN5c3RlbTpub2RlczEkMCIGA1UEAxMbc3lzdGVtOm5vZGU6aXAt
MTcyLTMxLTE0LTMzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAp/O4
Im6Qhskf84RAVT/4XS6bUy3NA2OCIM7ypM/ninIKRNP5F71o85i0C+z4tXDKLsDN
7dl1q1RK4Dryk6HfjSYml0eLkREM07inQZhebhjpp8Rt1L2kjkHWjAhWWhqKF071
bN+IdrHwONC5dnNXWAEnRMuROVKdpPLI5cn4R000SKLinirXGirCjjrhn+pPjQ1F
dafTF3s/xXzL+Mh1veT4CFSWifsMnWgLzrFlwl7VW/esnTGvgNlZJ0aTmgtn8fhS
CZBgqLPyNJAihcBh7f8S6NvNgd+L2cuJFeqfp0cek3KAKp8nOJMtCf+6Mr8eKe31
BVqIqBiLS/EL/SLz0wIDAQABo1YwVDAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAww
CgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAWgBTfvO/Uep8VnufK
t9AZNcKEWNolVjANBgkqhkiG9w0BAQsFAAOCAQEAfUOpcIZEiIg5OVz5Tx8J3LhX
KtH7/EAxIrQz461hjnFzM6u3B+v80b8kK6bxlKspBKdGMhfo9h9VEs0Oq1xeS9zu
INHySMOVdzjew/1SwyeyrJMTFGigwm2lPG2CBcIyccpCQKcC6ROEfivkUZnqTkYb
wN9fqBIbwfaHc3UMNrwkHSx/FiM1RN1SnockCZ6UYuqnntwnNo5lPVJNNk/2K9IK
EPNz8dirR/nAZySLQi92s+Ldw7cX9d8urUPZrW2Wzfm7eTCE5Z1jZZCEgpoWQWNz
6HJusg4eq9VVhlchQr1vhLyoI66hOVcmRUhn7klX6z6XP7b2dHoo7blpspHcMw==
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAp/O4Im6Qhskf84RAVT/4XS6bUy3NA2OCIM7ypM/ninIKRNP5
F71o85i0C+z4tXDKLsDN7dl1q1RK4Dryk6HfjSYml0eLkREM07inQZhebhjpp8Rt
1L2kjkHWjAhWWhqKF071bN+IdrHwONC5dnNXWAEnRMuROVKdpPLI5cn4R000SKLi
nirXGirCjjrhn+pPjQ1FdafTF3s/xXzL+Mh1veT4CFSWifsMnWgLzrFlwl7VW/es
nTGvgNlZJ0aTmgtn8fhSCZBgqLPyNJAihcBh7f8S6NvNgd+L2cuJFeqfp0cek3KA
Kp8nOJMtCf+6Mr8eKe31BVqIqBiLS/EL/SLz0wIDAQABAoIBAEubhRZDDd8Ppcmb
jNaT4LwaIiR05ukSn98jKsqVKZgCtKq9flJ4m0mmQc9ok6Iir3ISq+HaVoWVgcul
3dQmOBwzw4Ww2Jyqv0qEww7diA0qO+2hmQv/f5fm/a22hyEy91181zF3A9jjS9BX
7lXroLNmeYYX2j2i+oLqJRSFMrbtyHQyK5MJ30hDLYXkV3RSXGE9HZumpD5+GUCz
BQ/9eC5l2NTzw+lZ+GODj9y4xbBDITkC49jAXfAu9lbMQJxQ7NxCRRKLsm4yp8YC
EIVrGDQMmd+udvSYQjKjR3VPSb8YWqSEQ95yMtL+Uvywd4RN3MOPIhq4J6aZvg26
8JiYmrECgYEAwg5fGAEJ+BtBzujC1/82Px469JeybueB8dQ25CrOGacheuOEyT9k
5QCCPL/bKCvREp/GlL81ZbqhMc9aHRrDgOfEropavGoWLhOqDw5ZUXjU+4aJfaPF
DavzSLZjIHMJhdwilwBYPFZUnewYscNTe8xNQDSg6O1Pps3UfIeeAbsCgYEA3ZA0
tnJJJ+8jxFIpUOvtNsw53pm8tqur31Qp1lH4Gild9whUDyh4ImBbrNDTLsRwk1BK
j8vUyLuMIngRMH6FdMl+3bjEQiI7mAaby609EgtZXWEwyXES9oW/hVwLFZnC+Woj
k4Qi5YdBU8Pzrq50x3//ovNTucn+BEOVrlPVSMkCgYBv27ri6k5lzshrTW5q9Xi+
f116eirnlNkpnasacLYmwVkiLh3vp3QwMM/h1rGsgT1d3+2m9mUAQ8kBHkYSesfw
+Sg9eBD/hKNOYhVn4lyIAv+6EP4WBx3iWJi+9CtFnCoEGDV0F0XFWfoioeJGLZJk
zQpGlU+flJOSUhlGwyHIWwKBgCPEp/3cLVs5C/khmnHp5H24Mo9xGjoTNMf0+lwT
F46Bpx2+RnO8AMjr7WDUxYMDS3k8uQzFxzAwtsrJv1yo0DquXMDGl0hl5mEAkB4t
dXJ4SpD8o7ehfYI2zVhmJ5PxIrzJGb0y079iOnWfaLOGjmu2ijpwNdAEf/GIR53B
AumhAoGAf2bA4j8nV4ijKfqAk5Gsej5wwi5S4x9zyjwje1OgLqY6gwP/0UjPTlSn
Dwf4kpiasaFrvOoD+O/uP0OnbZ6FRzXpfgpfQ/DaoKbWSb91w7mlXb4SGZExHEPK
vCHjLSie3H43C25oep8DVzJyxSnKciK1m3e7vUP8CANKBPakFIA=
-----END RSA PRIVATE KEY-----

跟著kubesphere創建tls類型的secret^[1]文檔在kubesphere控制臺操作"保密字典"，最終創建如下Secret資源

kind: Secret
apiVersion: v1
metadata:
  name: kubelet-cert
  namespace: dongtai
  annotations:
    kubesphere.io/creator: admin
    kubesphere.io/description: kubelet證書，包括公鑰和私鑰
data:
  tls.crt: >-
    TUlJREtqQ0NBaEtnQXdJQkFnSUlOVm9rT213QzRYVXdEUVlKS29aSWh2Y05BUUVMQlFBd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pBZUZ3MHlNakExTWpVd01ETXdNRFZhRncweU16QTFNalV3TURNd01EWmFNRDB4CkZUQVRCZ05WQkFvVERITjVjM1JsYlRwdWIyUmxjekVrTUNJR0ExVUVBeE1iYzNsemRHVnRPbTV2WkdVNmFYQXQKTVRjeUxUTXhMVEUwTFRNek1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBcC9PNApJbTZRaHNrZjg0UkFWVC80WFM2YlV5M05BMk9DSU03eXBNL25pbklLUk5QNUY3MW84NWkwQyt6NHRYREtMc0ROCjdkbDFxMVJLNERyeWs2SGZqU1ltbDBlTGtSRU0wN2luUVpoZWJoanBwOFJ0MUwya2prSFdqQWhXV2hxS0YwNzEKYk4rSWRySHdPTkM1ZG5OWFdBRW5STXVST1ZLZHBQTEk1Y240UjAwMFNLTGluaXJYR2lyQ2pqcmhuK3BQalExRgpkYWZURjNzL3hYekwrTWgxdmVUNENGU1dpZnNNbldnTHpyRmx3bDdWVy9lc25UR3ZnTmxaSjBhVG1ndG44ZmhTCkNaQmdxTFB5TkpBaWhjQmg3ZjhTNk52TmdkK0wyY3VKRmVxZnAwY2VrM0tBS3A4bk9KTXRDZis2TXI4ZUtlMzEKQlZxSXFCaUxTL0VML1NMejB3SURBUUFCbzFZd1ZEQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0V3WURWUjBsQkF3dwpDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVdnQlRmdk8vVWVwOFZudWZLCnQ5QVpOY0tFV05vbFZqQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFmVU9wY0laRWlJZzVPVno1VHg4SjNMaFgKS3RINy9FQXhJclF6NDYxaGpuRnpNNnUzQit2ODBiOGtLNmJ4bEtzcEJLZEdNaGZvOWg5VkVzME9xMXhlUzl6dQpJTkh5U01PVmR6amV3LzFTd3lleXJKTVRGR2lnd20ybFBHMkNCY0l5Y2NwQ1FLY0M2Uk9FZml2a1VabnFUa1liCndOOWZxQklid2ZhSGMzVU1OcndrSFN4L0ZpTTFSTjFTbm9ja0NaNlVZdXFubnR3bk5vNWxQVkpOTmsvMks5SUsKRVBOejhkaXJSL25BWnlTTFFpOTJzK0xkdzdjWDlkOHVyVVBaclcyV3pmbTdlVENFNVoxalpaQ0VncG9XUVdOego2SEp1c2c0ZXE5VlZobGNoUXIxdmhMeW9JNjZoT1ZjbVJVaG43a2xYNno2WFA3YjJkSG9vN2JscHNwSGNNdz09
  tls.key: >-
    LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb2dJQkFBS0NBUUVBcC9PNEltNlFoc2tmODRSQVZULzRYUzZiVXkzTkEyT0NJTTd5cE0vbmluSUtSTlA1CkY3MW84NWkwQyt6NHRYREtMc0RON2RsMXExUks0RHJ5azZIZmpTWW1sMGVMa1JFTTA3aW5RWmhlYmhqcHA4UnQKMUwya2prSFdqQWhXV2hxS0YwNzFiTitJZHJId09OQzVkbk5YV0FFblJNdVJPVktkcFBMSTVjbjRSMDAwU0tMaQpuaXJYR2lyQ2pqcmhuK3BQalExRmRhZlRGM3MveFh6TCtNaDF2ZVQ0Q0ZTV2lmc01uV2dMenJGbHdsN1ZXL2VzCm5UR3ZnTmxaSjBhVG1ndG44ZmhTQ1pCZ3FMUHlOSkFpaGNCaDdmOFM2TnZOZ2QrTDJjdUpGZXFmcDBjZWszS0EKS3A4bk9KTXRDZis2TXI4ZUtlMzFCVnFJcUJpTFMvRUwvU0x6MHdJREFRQUJBb0lCQUV1YmhSWkREZDhQcGNtYgpqTmFUNEx3YUlpUjA1dWtTbjk4aktzcVZLWmdDdEtxOWZsSjRtMG1tUWM5b2s2SWlyM0lTcStIYVZvV1ZnY3VsCjNkUW1PQnd6dzRXdzJKeXF2MHFFd3c3ZGlBMHFPKzJobVF2L2Y1Zm0vYTIyaHlFeTkxMTgxekYzQTlqalM5QlgKN2xYcm9MTm1lWVlYMmoyaStvTHFKUlNGTXJidHlIUXlLNU1KMzBoRExZWGtWM1JTWEdFOUhadW1wRDUrR1VDegpCUS85ZUM1bDJOVHp3K2xaK0dPRGo5eTR4YkJESVRrQzQ5akFYZkF1OWxiTVFKeFE3TnhDUlJLTHNtNHlwOFlDCkVJVnJHRFFNbWQrdWR2U1lRaktqUjNWUFNiOFlXcVNFUTk1eU10TCtVdnl3ZDRSTjNNT1BJaHE0SjZhWnZnMjYKOEppWW1yRUNnWUVBd2c1ZkdBRUorQnRCenVqQzEvODJQeDQ2OUpleWJ1ZUI4ZFEyNUNyT0dhY2hldU9FeVQ5awo1UUNDUEwvYktDdlJFcC9HbEw4MVpicWhNYzlhSFJyRGdPZkVyb3BhdkdvV0xoT3FEdzVaVVhqVSs0YUpmYVBGCkRhdnpTTFpqSUhNSmhkd2lsd0JZUEZaVW5ld1lzY05UZTh4TlFEU2c2TzFQcHMzVWZJZWVBYnNDZ1lFQTNaQTAKdG5KSkorOGp4RklwVU92dE5zdzUzcG04dHF1cjMxUXAxbEg0R2lsZDl3aFVEeWg0SW1CYnJORFRMc1J3azFCSwpqOHZVeUx1TUluZ1JNSDZGZE1sKzNiakVRaUk3bUFhYnk2MDlFZ3RaWFdFd3lYRVM5b1cvaFZ3TEZabkMrV29qCms0UWk1WWRCVThQenJxNTB4My8vb3ZOVHVjbitCRU9WcmxQVlNNa0NnWUJ2MjdyaTZrNWx6c2hyVFc1cTlYaSsKZjExNmVpcm5sTmtwbmFzYWNMWW13VmtpTGgzdnAzUXdNTS9oMXJHc2dUMWQzKzJtOW1VQVE4a0JIa1lTZXNmdworU2c5ZUJEL2hLTk9ZaFZuNGx5SUF2KzZFUDRXQngzaVdKaSs5Q3RGbkNvRUdEVjBGMFhGV2ZvaW9lSkdMWkprCnpRcEdsVStmbEpPU1VobEd3eUhJV3dLQmdDUEVwLzNjTFZzNUMva2htbkhwNUgyNE1vOXhHam9UTk1mMCtsd1QKRjQ2QnB4MitSbk84QU1qcjdXRFV4WU1EUzNrOHVRekZ4ekF3dHNySnYxeW8wRHF1WE1ER2wwaGw1bUVBa0I0dApkWEo0U3BEOG83ZWhmWUkyelZobUo1UHhJcnpKR2IweTA3OWlPbldmYUxPR2ptdTJpanB3TmRBRWYvR0lSNTNCCkF1bWhBb0dBZjJiQTRqOG5WNGlqS2ZxQWs1R3NlajV3d2k1UzR4OXp5andqZTFPZ0xxWTZnd1AvMFVqUFRsU24KRHdmNGtwaWFzYUZydk9vRCtPL3VQME9uYlo2RlJ6WHBmZ3BmUS9EYW9LYldTYjkxdzdtbFhiNFNHWkV4SEVQSwp2Q0hqTFNpZTNINDNDMjVvZXA4RFZ6Snl4U25LY2lLMW0zZTd2VVA4Q0FOS0JQYWtGSUE9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0t
type: kubernetes.io/tls

在kubesphere控制臺界面上操作"應用路由"，最終創建如下Ingress資源

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: dongtai-anyone
  namespace: dongtai
  annotations:
    kubesphere.io/creator: admin
    kubesphere.io/description: 不用綁定host
spec:
  tls:
    - hosts:
        - 5x.x.x.x.nip.io
      secretName: kubelet-cert
  rules:
    - host: 5x.x.x.x.nip.io
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: dongtai-web-pub-svc
                port:
                  number: 8000

上面的Ingress資源創建后，就能提供https服務了。

總結

nip.io域名既可以省去"購買域名、配置A記錄"兩個步驟，也可以避免"綁定host"步驟。
需要注意的是，雖然https可以用來加密通信，但是因為tls握手包中仍然會有域名信息，所以有可能被檢查。