Skip to content

Commit 1923d79

Browse files
committed
πŸ“ doc: updated setup documentation
1 parent f0cb1fa commit 1923d79

File tree

7 files changed

+125
-78
lines changed

7 files changed

+125
-78
lines changed

β€ŽMakefileβ€Ž

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -167,10 +167,10 @@ check-helm-docs:
167167
./hack/verify-helm-docs
168168

169169
helm-manifest:
170-
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.31.1 > deploy/osc-ccm-manifest-v1.31.yml
171-
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.32.1 > deploy/osc-ccm-manifest-v1.32.yml
172-
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.33.1 > deploy/osc-ccm-manifest-v1.33.yml
173-
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.34.1 > deploy/osc-ccm-manifest-v1.34.yml
170+
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.31.3 > deploy/osc-ccm-manifest-v1.31.yml
171+
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.32.3 > deploy/osc-ccm-manifest-v1.32.yml
172+
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.33.3 > deploy/osc-ccm-manifest-v1.33.yml
173+
@helm template test ./deploy/k8s-osc-ccm/ --set image.tag=v1.34.3 > deploy/osc-ccm-manifest-v1.34.yml
174174

175175
check-helm-manifest:
176176
./hack/verify-helm-manifest.sh

β€Ždeploy/README.mdβ€Ž

Lines changed: 101 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,133 @@
1-
This documentation explains how to deploy Outscale Cloud Controller Manager.
1+
# πŸš€ Deploying the Outscale Cloud Controller Manager (CCM)
22

3-
# Prerequisites
3+
This documentation explains how to deploy Outscale Cloud Controller Manager (CCM).
4+
5+
## βœ… Requirements
46

57
You will need a Kubernetes cluster on the 3DS Outscale cloud.
68

9+
### Controller Manager and Kubelet configuration
10+
11+
When running Kubernetes in the cloud, the `--cloud-provider external` flag is required on the following components:
12+
* `kube-controller-manager`
13+
* `kubelet`
14+
* `kube-apiserver` (up to v1.33)
15+
16+
The flag has been removed from `kube-apiserver` in v1.33.
17+
18+
The configuration of this flag depends on the bootstrapping tool you use to deploy your cluster.
19+
When using Cluster-API, the required configuration is:
20+
21+
```yaml
22+
clusterConfiguration:
23+
apiServer:
24+
extraArgs:
25+
cloud-provider: "external"
26+
controllerManager:
27+
extraArgs:
28+
cloud-provider: "external"
29+
[...]
30+
initConfiguration:
31+
nodeRegistration:
32+
kubeletExtraArgs:
33+
cloud-provider: external
34+
[...]
35+
joinConfiguration:
36+
nodeRegistration:
37+
kubeletExtraArgs:
38+
cloud-provider: external
39+
```
40+
41+
More details can be found in the [Cloud Controller Manager Administration](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/#running-cloud-controller-manager) documentation.
42+
43+
### Nodes
44+
45+
Nodes should have a `spec.providerID` set with the following structure `osc:///<subregion>/<VM ID>`
46+
(for compatibility purposes, `aws:///<subregion>/<VM ID>` is also supported).
47+
48+
### Networking
49+
50+
The CCM needs to access the [metadata server](https://docs.outscale.com/en/userguide/Accessing-the-Metadata-and-User-Data-of-an-Instance.html) in order to get information about nodes.
51+
52+
Access to `169.254.169.254/32` using TCP on port 80 (http) must be allowed from the control-plane nodes.
53+
54+
### Configuring Cloud Credentials
55+
56+
The CCM needs to access the Outscale API.
57+
58+
It is recommended to use a specific [Access Key](https://docs.outscale.com/en/userguide/About-Access-Keys.html) and create an [EIM user](https://docs.outscale.com/en/userguide/About-EIM-Users.html) with limited access. Check the [EIM policy example](eim-policy.example.json) to apply to such EIM user.
59+
60+
## βš™οΈ Installation
61+
762
> Each major Kubernetes release requires a specific version of the CCM. You will need to install the CCM version tailored for your Kubernetes version.
863

9-
# Tagging
64+
### Create Secret
65+
66+
Create a secret with your cloud credentials:
67+
```shell
68+
kubectl create secret generic osc-secret \
69+
--from-literal=access_key=$OSC_ACCESS_KEY --from-literal=secret_key=$OSC_SECRET_KEY \
70+
-n kube-system
71+
```
72+
73+
### Deploy the Cloud Controller Manager
74+
75+
The CCM is deployed as a daemon set on control-plane nodes.
1076

11-
## Cluster Resource Tagging
77+
You can either deploy using a simple manifest:
78+
```shell
79+
kube_version=`kubectl get nodes --no-headers -o custom-columns=VERSION:.status.nodeInfo.kubeletVersion|cut -d . -f 1,2|head -1`
80+
kubectl apply -f deploy/osc-ccm-manifest-$kube_version.yml
81+
```
82+
83+
Or, you can use Helm:
84+
```shell
85+
helm upgrade --install --wait --wait-for-jobs k8s-osc-ccm oci://registry-1.docker.io/outscalehelm/osc-cloud-controller-manager --set oscSecretName=osc-secret --set image.tag=[the version compatible with your Kubernetes version]
86+
```
87+
88+
More [helm options are available](../docs/helm.md)
89+
90+
## πŸ”– Tagging
91+
92+
### Resource Tagging
1293

1394
The CCM expects resources to be tagged as being part of the cluster.
95+
1496
This includes:
15-
- [Subnets](https://docs.outscale.com/en/userguide/About-VPCs.html)
16-
- [Instances](https://docs.outscale.com/en/userguide/About-Instances.html)
17-
- [Security Groups](https://docs.outscale.com/en/userguide/About-Security-Groups-(Concepts).html)
97+
- [Subnets](https://docs.outscale.com/en/userguide/About-Nets.html)
98+
- [VMs](https://docs.outscale.com/en/userguide/About-VMs.html)
99+
- [Security Groups](https://docs.outscale.com/en/userguide/About-Security-Groups.html)
18100

19101
The tag key must be `OscK8sClusterID/[cluster-id]` (`[cluster-id]` being the ID of a cluster) and tag value can be one of the following values:
20102
- `shared`: resource is shared between multiple clusters, and should not be destroyed,
21103
- `owned`: the resource is considered owned and managed by the cluster.
22104

23105
The CCM will fetch the `OscK8sClusterID` tag of the node it is running on and will expect to find the other resources with matching tag keys.
24106

25-
The Cluster API Provider for Outscale (CAPOSC) sets the `OscK8sClusterID` tag, no need to do anything.
107+
When using Cluster API Provider for Outscale (CAPOSC), the tag is automatically set, no additional steps are required.
26108

27-
## Instances Tagging
109+
### VM Tagging
28110

29-
The CCM is usually able to find instances.
111+
The CCM is usually able to find VM instances using the `spec.providerID` value.
30112

31-
In some rare cases, the CCM needs a `OscK8sNodeName` tag on the VM, with the node name as a value.
113+
In some rare cases, the CCM will need a `OscK8sNodeName` tag on the VM, with the node name as a value.
32114

33-
The Cluster API Provider for Outscale (CAPOSC) sets the tag, no need to do anything.
115+
When using Cluster API Provider for Outscale (CAPOSC), the tag is automatically set, no additional steps are required.
34116

35-
# Creating load-balancers
117+
## πŸš€ Creating load-balancers
36118

37-
## Subnets
119+
### Subnets
38120

39121
The CCM will look for a subnet having one of the following tags:
40122
* `OscK8sRole/service.internal` is service is internal,
41123
* `OscK8sRole/service` is service is not internal or if no `OscK8sRole/service.internal` subnet is found,
42124
* `OscK8sRole/loadbalancer` if no subnet found.
43125

44-
The Cluster API Provider for Outscale (CAPOSC) automatically sets the `OscK8sRole/loadbalancer` tag to the subnet where the Kubernetes API load-balancer is configured.
126+
When using Cluster API Provider for Outscale (CAPOSC), the tags are automatically set, no additional steps are required.
45127

46-
## Security Groups
128+
### Security Groups
47129

48-
### Ingress
130+
#### Ingress
49131

50132
By default, the service controller will automatically create a Security Group for each Load Balancer Unit (LBU) and will attach it to nodes in a VPC setup.
51133

@@ -57,7 +139,7 @@ The CCM will automatically add manage ingress rules to allow traffic to the load
57139

58140
You can set `service.Spec.LoadBalancerSourceRanges` to restrict trafic to a list of IP ranges.
59141

60-
### Load-balancer to nodes
142+
#### Load-balancer to nodes
61143

62144
The CCM will add rules to allow trafic from the load-balancer to nodes.
63145

@@ -67,48 +149,7 @@ Within node security groups, it will search for a security group having one of t
67149

68150
The Cluster API Provider for Outscale (CAPOSC) sets a `OscK8sRole/worker` tag on all worker nodes, and allows you to add custom roles if needed.
69151

70-
## Networking
71-
72-
Node controller is deployed as a daemon set on control-plane nodes and will need to access [metadata server](https://docs.outscale.com/en/userguide/Accessing-the-Metadata-and-User-Data-of-an-Instance.html) in order to get information about its node (cpu, memory, addresses, hostname).
73-
To do this, node controller need to be able to access `169.254.169.254/32` through TCP port 80 (http).
74-
75-
## Kubelet
76-
77-
Kubelet must be run with `--cloud-provider=external`, (more details in [Cloud Controller Manager Administration](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/#running-cloud-controller-manager) documentation).
78-
79-
## Configuring Cloud Credentials
80-
81-
Outscale Cloud Controller Manager needs to access the Outscale API.
82-
83-
It is recommended to use a specific [Access Key](https://docs.outscale.com/en/userguide/About-Access-Keys.html) and create an [EIM user](https://docs.outscale.com/en/userguide/About-EIM-Users.html) with limited access. Check [EIM policy example](eim-policy.example.json) to apply to such EIM user.
84-
85-
# Deploy
86-
87-
## Create Secret
88-
89-
Create a secret with your cloud credentials:
90-
```shell
91-
kubectl create secret generic osc-secret \
92-
--from-literal=access_key=$OSC_ACCESS_KEY --from-literal=secret_key=$OSC_SECRET_KEY \
93-
-n kube-system
94-
```
95-
96-
## Deploy Cloud Controller Manager
97-
98-
You can either deploy using a simple manifest:
99-
```shell
100-
kube_version=`kubectl get nodes --no-headers -o custom-columns=VERSION:.status.nodeInfo.kubeletVersion|cut -d . -f 1,2|head -1`
101-
kubectl apply -f deploy/osc-ccm-manifest-$kube_version.yml
102-
```
103-
104-
Or, you can use Helm:
105-
```shell
106-
helm upgrade --install --wait --wait-for-jobs k8s-osc-ccm oci://registry-1.docker.io/outscalehelm/osc-cloud-controller-manager --set oscSecretName=osc-secret --set image.tag=[the version compatible with your Kubernetes version]
107-
```
108-
109-
More [helm options are available](../docs/helm.md)
110-
111-
# Upgrading CCM v0.x to v1.x
152+
## ⬆️ Upgrading CCM v0.x to v1.x
112153

113154
The secret has now the same format as the CSI driver. You need to rename:
114155
* `key_id` to `access_key`,

β€Ždeploy/osc-ccm-manifest-v1.31.ymlβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ spec:
146146
serviceAccountName: cloud-controller-manager
147147
containers:
148148
- name: osc-cloud-controller-manager
149-
image: outscale/cloud-provider-osc:v1.31.1
149+
image: outscale/cloud-provider-osc:v1.31.3
150150
imagePullPolicy: IfNotPresent
151151
command:
152152
- /bin/osc-cloud-controller-manager

β€Ždeploy/osc-ccm-manifest-v1.32.ymlβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ spec:
146146
serviceAccountName: cloud-controller-manager
147147
containers:
148148
- name: osc-cloud-controller-manager
149-
image: outscale/cloud-provider-osc:v1.32.1
149+
image: outscale/cloud-provider-osc:v1.32.3
150150
imagePullPolicy: IfNotPresent
151151
command:
152152
- /bin/osc-cloud-controller-manager

β€Ždeploy/osc-ccm-manifest-v1.33.ymlβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ spec:
146146
serviceAccountName: cloud-controller-manager
147147
containers:
148148
- name: osc-cloud-controller-manager
149-
image: outscale/cloud-provider-osc:v1.33.1
149+
image: outscale/cloud-provider-osc:v1.33.3
150150
imagePullPolicy: IfNotPresent
151151
command:
152152
- /bin/osc-cloud-controller-manager

β€Ždeploy/osc-ccm-manifest-v1.34.ymlβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ spec:
146146
serviceAccountName: cloud-controller-manager
147147
containers:
148148
- name: osc-cloud-controller-manager
149-
image: outscale/cloud-provider-osc:v1.34.1
149+
image: outscale/cloud-provider-osc:v1.34.3
150150
imagePullPolicy: IfNotPresent
151151
command:
152152
- /bin/osc-cloud-controller-manager

β€Ždocs/development.mdβ€Ž

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -115,16 +115,17 @@ make build-image image-tag image-push helm_deploy test-e2e
115115
2. Update chart version (if needed) in [Chart.yaml](../deploy/k8s-osc-ccm/Chart.yaml)
116116
3. Update cloud-provider-osc version in [values.yaml](../deploy/k8s-osc-ccm/values.yaml) (listing all active container versions)
117117
4. Generate helm doc `make helm-docs`
118-
5. Update manifests `make helm-manifest`
119-
6. Commit version with `git commit -am "cloud-controller-manager vX.Y.Z"`
120-
7. Create PR and merge it to main
121-
8. Tag & push the release
118+
5. In [Makefile](Makefile), update the recommended version in the `helm-manifest` target
119+
6. Update manifests `make helm-manifest`
120+
7. Commit version with `git commit -am "cloud-controller-manager vX.Y.Z"`
121+
8. Create PR and merge it to main
122+
9. Tag & push the release
122123
```shell
123124
export HELM_VERSION=vX.Y.Z-helm
124125
git tag -a $HELM_VERSION -m "πŸ”– Helm $HELM_VERSION"
125126
git push origin $HELM_VERSION
126127
```
127-
11. Publish the release on Github
128+
10. Publish the release on Github
128129

129130
## Container release
130131

@@ -136,18 +137,23 @@ Z is the version of the CCM, is increased at each release, and is the same for e
136137

137138
1. In [CHANGELOG.md](CHANGELOG.md), add a new version for every release branch
138139
2. In [README.md](README.md), update the recommended version for each Kubernetes release
139-
3. Create PR and merge it to main
140-
4. Merge main into each release branch
140+
3. In [Makefile](Makefile), update the recommended version in the `helm-manifest` target
141+
4. Update the manifests
142+
```shell
143+
make helm-manifest
144+
```
145+
5. Create PR and merge it to main
146+
6. Merge main into each release branch
141147
```shell
142148
VERSION=1.3X && git co kubernetes-$VERSION && git merge --no-edit main && git push origin kubernetes-$VERSION
143149
```
144-
5. Check that CI is OK on every release branch
145-
6. Tag release for each release branch
150+
7. Check that CI is OK on every release branch
151+
8. Tag release for each release branch
146152
```shell
147153
git co kubernetes-X.Y
148154
git pull --rebase
149155
export VERSION=vX.Y.Z
150156
git tag -a $VERSION -m "πŸ”– CCM $VERSION"
151157
git push origin $VERSION
152158
```
153-
7. Publish the releases on Github
159+
9. Publish the releases on Github

0 commit comments

Comments
Β (0)