Skip to content

Commit a83b121

Browse files
authored
Update README documentation (#78)
* Remove redundant info and update image * Bring headings down one notch and clarify pipeline call * Get rid of mystery character * Update token macro documentation * Add documentation for new parameter
1 parent 11c55f8 commit a83b121

File tree

3 files changed

+50
-56
lines changed

3 files changed

+50
-56
lines changed

doc/README.md

Lines changed: 50 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@
66

77
This plugin collects and publishes [Robot
88
Framework](http://robotframework.org/ "Robot Framework Home")
9-
test results in Jenkins.
9+
test results in Jenkins. The plugin **doesn't** run Robot Framework, it
10+
only reports the results.
11+
1012
Preconditions: all "suggested plugins" are "installed" in Jenkins.
1113

1214
[Issue
1315
Tracker](https://issues.jenkins-ci.org/issues/?jql=project+%3D+JENKINS+AND+component+%3D+robot-plugin)
1416

15-
### Current features
17+
## Current features
1618

1719
- Test suite and case details with trend graphs
1820
- Environment variable expansion for build paths in configuration
@@ -24,17 +26,16 @@ Tracker](https://issues.jenkins-ci.org/issues/?jql=project+%3D+JENKINS+AND+compo
2426
- Trend graph of passed tests over builds on project page
2527
- Marking build failed/unstable/successful according to pass
2628
thresholds given by user
27-
- Option of evaluating only critical tests against the pass thresholds
2829
- Listview column to show overall passed/failed tests in project
2930
listing and duration trend
3031
- Configurable archiving of arbitrary Robot related artifacts per
31-
build 
32+
build
3233
- Token macros for usage with e.g. email-ext plugin
3334
- Test visibility in radiator views
3435

35-
### Configuration
36+
## Configuration
3637

37-
#### Basic configuration of testing task
38+
### Basic configuration of testing task
3839

3940
1. Configure your project
4041
2. Select: Build -\> Add build step -\> execute shell/Execute Windows
@@ -58,69 +59,61 @@ Tracker](https://issues.jenkins-ci.org/issues/?jql=project+%3D+JENKINS+AND+compo
5859
7. If your output files are named differently than default (i.e.
5960
output.xml, report.html and log.html) specify the filenames by
6061
pressing the "Advanced..." button and writing the names in relevant
61-
fields. The fields support wildcards `*` and `?`.
62+
fields. The fields support wildcards `*` and `?`.
6263
8. If you have other artifacts such as screenshots that you want to
6364
persist for viewing later in the logs you must configure them under
6465
"Advanced... -\> Other files to copy". The field accepts comma
6566
separated list of files and supports wildcards `*` and `?`.
6667
9. Set thresholds and optionally disable thresholds for critical tests
6768
only to count every test in the pass percentage.
6869

69-
#### Pipeline configuration
70+
### Pipeline configuration
7071

7172
1. Run your tests in a similar way as you would normally by using the `sh` step.
7273
2. Call the plugin with either the general `step` or directly by calling `robot` (only from
73-
version 2.0 onwards): `robot outputPath: '.', logFileName: 'log.html', outputFileName:
74-
'output.xml', reportFileName: 'report.hml', passThreshold: 100, unstableThreshold: 75.0`
74+
version 2.0 onwards):
75+
76+
```groovy
77+
robot(outputPath: ".",
78+
passThreshold: 90.0,
79+
unstableThreshold: 70.0,
80+
disableArchiveOutput: true,
81+
outputFileName: "output.xml",
82+
logFileName: 'log.html',
83+
reportFileName: 'report.html',
84+
countSkippedTests: true, // Optional, defaults to false
85+
otherFiles: 'screenshot-*.png'
86+
)
87+
```
7588
76-
#### Configuring direct links to log files
89+
**NOTE!** `onlyCritical` parameter has been deprecated and will be removed in the future. It has no effect on the results.
7790
78-
##### Version 1.3.0
91+
### Configuring direct links to log files
92+
93+
#### Version 1.3.0+
7994
8095
Links are automatically generated to whatever files are configured to
8196
"Report html" and "Log html" in the plugin configuration. Links to log
8297
and report HTMLs are now only in the summary sections in the middle of
8398
the page and the sidebar link from previous versions have been removed.
8499
85-
##### Version 1.2.3 and earlier
86-
87-
From version 1.2 it's possible to configure a direct link from both
88-
side panel and project/build page summaries to chosen file persisted with
89-
the build. If no file is configured the link won't show up. Steps to do
90-
this are as follows:
91-
92-
1. Go to project configuration
93-
2. Configure the name of the file to be linked to Log/Report link field
94-
3. Make sure that you are saving the actual file you want to link with
95-
the build by checking the filenames/masks configured in the fields
96-
behind "Advanced..." button. By default these are output.xml,
97-
report.html and log.html which are the Robot Framework default names
98-
4. Save the configuration and run a build. The links are persisted per
99-
build in order to withstand changes in file naming. Thus the links
100-
won't appear in builds that exist already.
101-
5. Links should appear in the project/build summaries and in build
102-
side panel.
103-
104-
NOTE: After configuration change etc. the link will appear also in
105-
project side panel, but for now it doesn't appear there right after the
106-
build. Conversely, the project view side panel link won't go away until
107-
after configuration change/resave in project. We're working on a
108-
solution to this.
109-
110100
![sidepanel link](images/sidepanel.png)
111101
*side panel link*
112102
113-
![summary link](images/robot_4_table.png)
103+
![summary link](images/robot_summary_table.png)
114104
*Summary link*
115105
116-
#### Configuring Robot overall pass/fail to show in the project list
106+
### Configuring Robot overall pass/fail to show in the project list
107+
108+
The list view column showing the overall pass/fail and duration trend should be visible
109+
by default in the project list. If it's not visible, you can add it by following these steps:
117110
118111
1. Go to Jenkins front page
119112
2. Select the + sign in the top tabs of the project listing to create a
120113
new view or go to
121114
[http://YOURJENKINSHOST/newView](http://yourjenkinshost/newView)
122115
3. Name your view and proceed creating a list view
123-
4. Choose project you want to include to the view. By default the
116+
4. Choose project you want to include to the view. By default, the
124117
"Robot pass/fail" column will appear to the column listing.
125118
5. Save view and select it from the top tabs in Jenkins project listing
126119
6. (Optional) If you want to have the view on by default go to "Manage
@@ -131,7 +124,7 @@ solution to this.
131124
![List view](images/robot_view_column.png)
132125
*List view column in action*
133126
134-
#### Using token macros in e-mail report
127+
### Using token macros in e-mail report
135128
136129
Prerequisites: token-macro plugin and email-ext plugin installed.
137130
@@ -142,21 +135,22 @@ Prerequisites: token-macro plugin and email-ext plugin installed.
142135
3. Now you can use the following Robot variables in your custom result
143136
email:
144137
145-
- `${ROBOT_FAILEDCASES}` - Expands to list of failed Robot cases. Each
138+
- `${ROBOT_FAILEDCASES, addErrorMessages}` - Expands to list of failed Robot cases. Each
146139
case on its own line.
147-
- `${ROBOT_PASSPERCENTAGE, onlyCritical}` - Expands to pass percentage
148-
of tests. (passed / total \* 100 %),
149-
onlyCritical - True if only critical tests should be calculated in
150-
percentage. Defaults to false.
151-
- `${ROBOT_PASSRATIO, onlyCritical}` - Expands to build result in 'passed
152-
/ total' format.
153-
onlyCritical - True if only critical tests should be calculated in
154-
percentage. Defaults to false.
140+
`addErrorMessages` is an optional parameter that can be used to include error messages in the output. Default value is `false`.
141+
- `${ROBOT_FAILED}` - Expands to count of failed cases.
142+
- `${ROBOT_PASSPERCENTAGE, countSkippedTests}` - Expands to pass percentage
143+
of tests. (passed / total \* 100 %).
144+
`countSkippedTests` is an optional parameter that can be used to include skipped tests in the total count. Default value is `false`.
145+
- `${ROBOT_PASSRATIO}` - Expands to build result in 'passed
146+
/ total' format. This total includes skipped test cases.
147+
- `${ROBOT_PASSED}` - Expands to count of passed cases.
155148
- `${ROBOT_REPORTLINK}` - If logfile link is configured in the Robot
156149
plugin this link will point to that file for the build. Else show
157150
link to Robot reports directory for the build.
151+
- `${ROBOT_TOTAL}` - Expands to total count of test cases. This total includes skipped test cases.
158152
159-
#### Displaying test numbers in build radiator views etc.
153+
### Displaying test numbers in build radiator views etc.
160154
161155
> :warning: If the Robot plugin marks the build as failure the tests will not show
162156
up. This is because only unstable and successful builds are considered
@@ -165,7 +159,7 @@ to have test results per Jenkins.
165159
>This means that in order to see the test results in other views you must
166160
set your unstable threshold so that the build never goes to failure.
167161
168-
#### Log File Not Showing Properly
162+
### Log File Not Showing Properly
169163
170164
![](images/log_fail_open.png)
171165
@@ -178,7 +172,7 @@ and [here](https://content-security-policy.com/) how to change you CSP settings
178172
but be aware that **Changing CSP settings will potentially expose you Jenkins instance for
179173
security vulnerabilities**.
180174
181-
### Robot Framework 4.x+ compatibility
175+
## Robot Framework 4.x+ compatibility
182176
183177
:heavy_exclamation_mark: As we're preparing to drop support for RF 3.x, the `onlyCritical` flag has been deprecated and no
184178
longer has any effect. It will be removed in the future, but for now it's available for the transition period.
@@ -187,15 +181,15 @@ tests in the pass percentage calculation.
187181
188182
The plugin still supports RF 3.x, but no longer takes criticality into account. If you want to use RF 3.x with criticality, please downgrade to the last major version (4.0.0)
189183
190-
### Overall Screenshots
184+
## Overall Screenshots
191185
192186
193-
##### Project view
187+
#### Project view
194188
195189
![Build page overview](images/build_page.png)
196190
197191
198-
##### Detailed build view
192+
#### Detailed build view
199193
200194
![Detailed results view](images/detailed.png)
201195

doc/images/robot_4_table.png

-28.2 KB
Binary file not shown.

doc/images/robot_summary_table.png

20.3 KB
Loading

0 commit comments

Comments
 (0)