Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions inventory/reference/group_vars/all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,14 @@ validation:
# Add other services if more should be checked.
check_fireedge_ui: true

# TEST CASE NAME: Networking subsystem validation
# DESCRIPTION: Instantiates a VM at each OpenNebula vNET, checks external connectivity
# from the test VM
# OUTPUT: HTML document at /tmp/cloud_verification_report.html
networks_verification: true
network:
ext_host: 'google.com'

# TEST CASE NAME: Storage Benchmark
# DESCRIPTION: Instantiates a VM for running storage benchmark from it. The VM needs to
# have public internet access to install dependencies. The VM's connected network can
Expand Down
5 changes: 5 additions & 0 deletions playbooks/networks-verification.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
- hosts: "{{ frontend_group | d('frontend') }}"
roles:
- role: validation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for this playbook, the validation role is already invoked in the main validation.yml playbook

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The basic idea is to run those optional networkings tests in scope of generic validation framework. So those tests are part of the validation role

47 changes: 47 additions & 0 deletions playbooks/templates/appendix-cloud-report.j2
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,53 @@
</table>
</div>


{% if validation_vars['networks_verification'] == true and network_verification_results | length > 0 %}

<div class="table_component" role="region" tabindex="0">

<h2 class="western">Virtual Networks validation results</h2>
<table width="656" cellpadding="3" cellspacing="0" style="page-break-before: auto; page-break-after: auto">
<col width="281"/>
<col width="200"/>
<col width="260"/>
<tr valign="top">

<td width="300" bgcolor="#666666" style="background: #666666; border: 1.00pt solid #000000; padding: 0.07in"><p align="left" style="page-break-inside: auto; orphans: 0; widows: 0; border: none; padding: 0in; background: transparent; page-break-after: auto">
<font color="#ffffff"><font size="3" style="font-size: 12pt"><b>Network name</b></font></font></p>
</td>
<td width="90" bgcolor="#666666" style="background: #666666; border: 1.00pt solid #000000; padding: 0.07in"><p align="left" style="page-break-inside: auto; orphans: 0; widows: 0; border: none; padding: 0in; background: transparent; page-break-after: auto">
<font color="#ffffff"><font size="3" style="font-size: 12pt"><b>Result</b></font></font></p>
</td>
<td width="260" bgcolor="#666666" style="background: #666666; border: 1.00pt solid #000000; padding: 0.07in"><p align="left" style="page-break-inside: auto; orphans: 0; widows: 0; border: none; padding: 0in; background: transparent; page-break-after: auto">
<font color="#ffffff"><font size="3" style="font-size: 12pt"><b>Comments</b></font></font></p>
</td>
</tr>
{% for k, v in network_verification_results.items() %}
<tr valign="top">
<td width="300" height="21" style="background: transparent; border: 1.00pt solid #000000; padding: 0.07in"><p align="left" style="page-break-inside: auto; orphans: 0; widows: 0; border: none; padding: 0in; background: transparent; page-break-after: auto">
<b>{{ k }}</b></p>
</td>
<td width="90" style="background: transparent; border: 1.00pt solid #000000; padding: 0.07in"><p align="left" style="page-break-inside: auto; orphans: 0; widows: 0; border: none; padding: 0in; background: transparent; page-break-after: auto">
{% for test, value in v.items() %}
<b>{{ test }}</b>: {{ value }}</p>
{% endfor %}
</td>
<td width="260" style="background: transparent; border: 1.00pt solid #000000; padding: 0.07in"><p align="left" style="page-break-inside: auto; orphans: 0; widows: 0; border: none; padding: 0in; background: transparent; page-break-after: auto">

</p>
</td>
</tr>
{% endfor %}

</p>
</table>
</div>

{% endif %}



{% if storage_benchmark is defined and storage_benchmark | length > 0 %}
<div class="table_component" role="region" tabindex="0">

Expand Down
3 changes: 3 additions & 0 deletions playbooks/validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
- role: storage-benchmark
when: validation.run_storage_benchmark | d(true)
- role: validation_frontend
- role: network_validation
when: validation.networks_verification | d(true)

# Run connectivity tests between virtual machines
- hosts: "{{ node_group | d('node') }},{{ frontend_group | d('frontend') }}"
Expand Down Expand Up @@ -53,6 +55,7 @@
become: false
vars:
verification_result: "{{ hostvars[groups[frontend_group | d('frontend')][0]].verification_result | default({'No verification results were found': 'N/A'})}}"
network_verification_results: "{{ hostvars[groups[frontend_group | d('frontend')][0]].network_verification_results | default({'No Network verification results were found': {'N/A':'N/A'}})}}"
date: "{{ lookup('pipe', 'date +%Y-%m-%d\\ %H:%M:%S') }}"
validation_vars: "{{ hostvars[groups[frontend_group | d('frontend')][0]].validation | default({}) }}"
tasks:
Expand Down
40 changes: 40 additions & 0 deletions roles/network_validation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
Role Name
=========

OpenNebula cloud verification role.

Requirements
------------

Ansible inventory, used for the cloud deployment should be used with this playbook

Role Variables
--------------


Example Playbook
----------------

This role validates all configured OpenNebula vNets and checks for:
- connection from VM to default GW
- DNS resolution from VM using DNS server specified for the network
- Connectivity to the external host by pinging that host

Important!!!
These network checks rely on ping for connectivity checks, thus ICMP messages should be allowed on the infrastructure level

This role should be included into the target playbook, like the following:

- hosts: "{{ frontend_group | d('frontend') }}"
roles:
- role: network_validation

License
-------

BSD

Author Information
------------------

OpenNebula team
123 changes: 123 additions & 0 deletions roles/network_validation/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---

# Gather min subset of facts for datetime
- setup:
gather_subset:
- min

# These tasks are used to test networks, reachability of the default gw, DNS and external connectivity
- name: Get all OpenNebula networks
ansible.builtin.shell: >
onevnet list -j | jq .VNET_POOL.VNET[].NAME -r
register: one_networks
failed_when: "one_networks.rc != 0"
run_once: true

- name: Print network list
debug:
msg: "Virtual Networks for testing: {{ one_networks.stdout }}"
run_once: true

# We'll use VM template from the test_vm role
#
- name: Instantiate VM at each network
ansible.builtin.shell: >
onetemplate instantiate '{{ validation.test_vm.vm.market_name }}' --nic '{{ item }}'
register: vms
failed_when: "vms.rc != 0"
loop: "{{ one_networks.stdout.splitlines() }}"
run_once: true

- name: Wait for VM come up
ansible.builtin.shell: >
onevm list --f ID={{ item.stdout.split(':')[1] | trim}} -l STAT --no-header
register: vm_state
failed_when: "vm_state.rc != 0"
until: vm_state.stdout == "runn"
retries: 10
delay: 10
loop: "{{ vms.results }}"
run_once: true

- name: Get VM IP
ansible.builtin.shell: >
onevm list --f ID={{item.stdout.split(':')[1] | trim}} -l IP --no-header
register: vm_ip
loop: "{{ vms.results }}"
run_once: true

- name: Verify ssh connection to the test VM
ansible.builtin.shell: >
ssh -i ~oneadmin/.ssh/id_rsa -o StrictHostKeyChecking=no root@{{item.stdout}} hostname
register: vm_ssh_result
until: vm_ssh_result.rc == 0
retries: 10
delay: 10
loop: "{{ vm_ip.results }}"
run_once: true

- name: Save VM IPs to list
set_fact:
vm_ips_list: "{{ vm_ip.results | map(attribute='stdout') | list }}"

- name: Install required packages
ansible.builtin.shell: |
ssh -i ~oneadmin/.ssh/id_rsa -o StrictHostKeyChecking=no root@{{item.stdout}} apk update
ssh -i ~oneadmin/.ssh/id_rsa -o StrictHostKeyChecking=no root@{{item.stdout}} apk add bind-tools jq jc
register: installation_result
loop: "{{ vm_ip.results }}"
run_once: true

- name: Check GW reachability from VM
ansible.builtin.shell: |
ssh -i ~oneadmin/.ssh/id_rsa -q -oStrictHostKeyChecking=no root@{{item.stdout}} ping -c 3 $(ip route show default | awk '/default/ {print $3}') | jc --ping --pretty
register: gw_reachability
loop: "{{ vm_ip.results }}"
run_once: true

- name: Check DNS resolvation
ansible.builtin.shell: |
ssh -i ~oneadmin/.ssh/id_rsa -q -oStrictHostKeyChecking=no root@{{item.stdout}} dig {{ validation.network.ext_host }} | jc --dig -p | jq .[].answer.[].data
register: dns_reachability
loop: "{{ vm_ip.results }}"
run_once: true

- name: Check external host reachability from the test VM
ansible.builtin.shell: |
ssh -i ~oneadmin/.ssh/id_rsa -q -oStrictHostKeyChecking=no root@{{item.stdout}} ping -c 3 {{ validation.network.ext_host }} | jc --ping --pretty
register: ext_host_reachability
loop: "{{ vm_ip.results }}"
run_once: true

- name: Save GW test results to list
set_fact:
gw_reachability_list: "{{ gw_reachability_list | default([]) + [item.stdout] }}"
loop: "{{ gw_reachability.results }}"
run_once: true

- name: Save ext host ping test results to list
set_fact:
ext_host_reachability_list: "{{ ext_host_reachability_list | default([]) + [item.stdout] }}"
loop: "{{ ext_host_reachability.results }}"
run_once: true

- name: Save DNS resolvation test results to list
set_fact:
dns_reachability_list: "{{ dns_reachability_list | default([]) + [item.stdout] }}"
loop: "{{ dns_reachability.results }}"
run_once: true

- name: Save VM IPs for report
set_fact:
network_verification_results: "{{ (network_verification_results | default({})) | combine({item.0: {'Test VM IP': item.1, '\nNetwork GW ping, packet received': item.2, '\nExternal Host ping: ': item.3, '\nExternal host DNS resolvation':item.4}}) }}"
loop: "{{ one_networks.stdout.splitlines() | zip(vm_ips_list, gw_reachability_list, ext_host_reachability_list, dns_reachability_list) | list }}"


- name: Remove test VMs
ansible.builtin.shell: >
onevm terminate '{{ item.stdout.split(':')[1] | trim }}'
register: terminated_vms
failed_when: "terminated_vms.rc != 0"
loop: "{{ vms.results }}"
run_once: true

1 change: 1 addition & 0 deletions roles/validation/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@
file: "{{ role_path }}/tasks/core_services_verification.yml"
when: validation.run_core_services


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should create a specific role for this network task and change the validation role into a different name like services. Just a suggestion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this relates to the comment above: The basic idea is to run those optional networkings tests in scope of generic validation framework. So those tests are part of the validation role originally. However not a big deal to split them out.