Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
257 changes: 257 additions & 0 deletions include/rosparam_shortcuts/parameter.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
#pragma once

#include <mutex>
#include <shared_mutex>

#include <rclcpp>

namespace rosparam_shortcuts {

/*
* Thread-safe container for generic parameter values, providing callback functions.
* This class implements a tree structure of parameters which are declared recursively.
* All leafs are of type Parameter<rclcpp::Parameter> which are used for interacting with
* the rclcpp node parameter interface using the NodeConfig class further down.
*/
template <typename ParameterT>
class Parameter
{
public:
Parameter(const std::string& name);
Parameter(const std::string& name, const ParameterT& default_value);

// Fields
const std::string& name() const noexcept;
std::string description() const noexcept;
std::string additional_constraints() const noexcept;
bool isNodeParameter() const noexcept; // true if this parameter contains an rclcpp::Parameter

// Properties
Parameter<ParameterT>& describe(const std::string& description, const std::string& additional_constraints = "");
// Allow setting additional properties. i.e. read_only, dynamic_typing?
// void setProperties(...);

// Validity Status
const bool isInitialized() const noexcept; // true when created with default
const bool isValid() const noexcept; // errors empty?
const bool hasErrors() const noexcept;
const bool hasWarnings() const noexcept;
// TODO: Insert additional thread-safe access to errors/warnings

// Read/write-locked data access using std::shared_mutex
// See: https://www.youtube.com/watch?v=ozOgzlxIsdg
ReadLockedPtr<ParameterT> operator->const;
WriteLockedPtr<ParameterT> operator->;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you actually need a shared_mutex here instead of a simple one?
The current API design only locks readers while it copies the values of parameters for the method return value, right?
The concept of a shared mutex really only offers better performance with long locking times for readers and rare write updates.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you actually need a shared_mutex here instead of a simple one?

I think it has many advantages, but a simple one might work as well.

The current API design only locks readers while it copies the values of parameters for the method return value, right?

What do you mean with "only locks readers"? The ReadLockedPtr is a scoped lock that prevents write access. The read lock is shared so that multiple callers can access the value (const ref) at the same time. The WriteLockedPtr is also a scoped lock, but it's exclusive and allows write access (non-const ref).
Then there are two extra convenience operators which allow copy-get (operator()), copy-set (operator=) which would use read/write locks internally.

The concept of a shared mutex really only offers better performance with long locking times for readers and rare write updates.

I would expect that we have many many read calls and only some (declare/update callbacks) write calls.
The shared_mutex would allow many readers at the same time which would make it more efficient than a simple one as multiple threads don't need to be synchronized for reading.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect that we have many many read calls

True. That is quite a difference to ROS, where we fetch parameters only rarely via API.
Assuming you want to keep parallel access to the dynamic datastructures possible, the shared read-lock makes sense.

// optional: (useful for synchronized vector/map operations)
void applyRW(std::function<void(ParameterT& value)>);
void applyRO(std::function<void(const ParameterT& value)>);

// Copy-set value using read/write lock internally
ParameterT& operator=(const ParameterT& value);

// Copy-get value, throws exception if isInitialized is false, uses read/write lock interally
ParameterT operator() const; // or: 'operator ParameterT() const;'

// Validity Callbacks (could also use rcl_interfaces::msg::SetParametersResult like in node_parameters.h).
Copy link
Contributor Author

@henningkayser henningkayser Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There could also be something like bool testValueChange(const ParameterT& new_value) which triggers the same (chained) validity callbacks only without actually changing the value. Incoming parameter changes from other nodes can be rejected if they are not valid.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems useful. It could also mean that subsystems inside the same node could internally "test a config change" although I'm not sure that's a pattern we want to encourage. There are better ways for a node to communicate internally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd argue that a node should probably not change its own parameters internally, except for setting defaults maybe. I mean, the use case of dynamic parameters is that other nodes (e.g. user interfaces) can change the modality of the nodes subsystem.

// Callbacks are stored in ordered hash map, could also possibly return hash for deleting later on.
// Alternatively, return Parameter<ParameterT>& for method chaining
using ParameterValidityCallback = std::function<bool(const ParameterT& value, std::string& message)>;
Parameter<ParameterT>& warnIf(ParameterValidityCallback warn_if_callback);
Parameter<ParameterT>& errorIf(ParameterValidityCallback error_if_callback); // or rejectIf()/failIf()..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the idea to support these predicates in the API.

I would consider to add the textual description together with each predicate here, instead of describing them independently in describe. See my example below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So.. describe() doesn't really describe the predicates, it describes the parameter itself. I agree that we should extend the API to allow setting the predicate descriptions like you suggested below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the additional_constraints field is currently set through describe() and should pretty much describe the predicates on these parameters. That's what I meant. :-)

// Predefined generic (type-specific?) conditions, i.e. 'NO_OVERRIDE', 'EMPTY', 'NOT_NORMALIZED'
enum ParameterCondition
{
...
};
Parameter<ParameterT>& warnIf(ParameterCondition condition);
Parameter<ParameterT>& errorIf(ParameterCondition condition);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you convinced it is actually useful to have a warnIf interface at all? I would argue that requests to set parameters either have to be valid or they are invalid. Allowing warnings here might support cases where the user knows better and sets parameters that trigger warnings (and they work). The result is a useless warning in an otherwise running system.

The other way around, if the warning actually indicates a failure, it should actually be an error anyway...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably up to the user. To me it's useful to warn that some feature is disabled or if some parameter doesn't have an override even though the default is valid. Sometimes warnings are just error cases that can be handled. In the end this API would allow making warnings errors and vice-versa without having to refactor custom error handling somewhere else in the code.


// Callback for reacting to simple parameter changes (read-locked)
// NOTE: this should be called after validity callbacks so that the parameter status can be evaluated
using ParameterChangedCallback = std::function<void(const Parameter& parameter)>;
void onChanged(ParameterChangedCallback callback);

protected:
// Recursively declare parameters and populate child_params_, the actual node parameters (leafs) are
// collected and returned so that they can be maintained by NodeConfig. Should call declareChildParam()
// internally. NodeParameter is Parameter<rclcpp::Parameter> and needs to be forward declared..
virtual std::map<std::string, std::shared_ptr<NodeParameter>> declare();

// Constructor used for child parameters in the tree that share the same data, so basically all parameters
// that are neither root nor leaf.
// NOTE: not sure if this is really a good solution or if we should simply copy the data and synchronize
// using child/parent callbacks only
Parameter(const std::string& name, ParameterT& value, std::shared_mutex parent_mutex);

private:
// Initialize child parameter and call declare() function recursively
template <ChildParameterT>
void declareChildParam(const std::string, std::map<std::string, std::shared_ptr<NodeParameter>>& node_parameters);

// Internal value protected by mutex_. Alternatively use a struct ParameterValue that manages
// locked acces and use that for internal data (this would allow sharing references between child
// parameters)
ParameterT value_;
mutable std::shared_mutex mutex_;

std::map<std::string, std::shared_ptr<Parameter>> child_params_;

const std::string name_;
std::string description_;
std::string additional_constraints_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike the term additional_ because there is no simple contraints yet.
Usually this kind of information is just added to the description, but if you will support turing-complete validation predicates in the API, it makes a lot of sense to let authors explain them in text as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOL, I don't like it either, it's just the field name of the description message. Since that one only applies to basic parameter types, we could also just rename this field to type_specification, unit, or simply comment (or just leave it). The reason I added this is because it would be nice to add this as inline-comment when generating yaml dumps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name makes more sense in the message though because the other modeled constraints are listed below (ranges, etc.).
I don't think you directly model these other aspects here. I guess they are modeled in the underlying rclcpp API?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct! This field wasn't really thought through. Basically, you should be able to specify these for supported rclcpp parameters but in the context of more generic types this doesn't make a lot of sense. I'm not even sure if we really want to model the rclcpp constraints at all if they basically act the same way as the callbacks in the end.


// Store callbacks, use alternative data structurs for ordered hashes
std::map<std::string, ParameterValidityCallback> warn_if_callbacks_;
std::map<std::string, ParameterValidityCallback> error_if_callbacks_;
std::map<std::string, ParameterChangedCallback> on_changed_callbacks_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do all these key strings contain? Shouldn't these all just be lists?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hashes so that we can identify and remove them again? At least for the on_changed_callbacks_ this would be desirable. But this definitely doesn't have to be a feature for the MVP.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A much simpler approach would just use std::list then and return the iterator as a handle.
The overhead of maps for this feels unjustified.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

of course!


// Store messages of failed validity callbacks
std::vector<std::string> errors_;
std::vector<std::string> warnings_;
}

// namespace Parameter
using NodeParameter = Parameter<rclcpp::Parameter>;
std::map<std::string, std::shared_ptr<NodeParameter>>
NodeParameter::declare() override{ /* returns shared_from_this */ };

/*
* Maintains all Parameter instances and does the heavy lifting with actually declaring and interfacing with dynamic
* rclcpp Parameters. Provides additional convenience functions for validating all parameters, printing, etc...
*/
class NodeConfig
{
public:
NodeConfig(const rclcpp::Node::SharedPtr& node, const std::string base_name = "");

protected:
/*
* Declare Parameter types, internally calls Parameter<ParameterT>::declare() to get
* all actual node parameters which are then declared using the node parameter interface.
*
* This eventually calls the ros interfaces for declaring and getting the parameter.
*/
template <typename ParameterT>
Parameter<ParameterT>& declare(Parameter<ParameterT>& parameter);

/*
* Undeclare a parameter, remove all callbacks that have been registered using the declare() call
*/
template <typename ParameterT>
bool undeclare(const Parameter<ParameterT>& parameter);
bool undeclare(const std::string& name);

/*
* Return the parameter for a given name, if not found parameter is nullptr
*/
// Not possible when working with references, maybe only for actual node parameters?
// std::shared_ptr<Parameter<ParameterT>> get(const std::string& name);
// std::shared_ptr<const Parameter<ParameterT>> get(const std::string& name) const noexcept;

//
// TODO: insert utility functions: validate(), print, ostream<<, errors(), warnings()...
//

private:
const rclcpp::Node::SharedPtr node_;
const std::string base_name_;

// Class similar to the one in rosparam_shortcuts/node_parameters.h
// implements ROS calls/checks/callbacks for node_parameters_
NodeParametersInterface node_parameters_interface_;

// Store parameters, only node parameters (rclcpp::Parameters) are stored inside NodeConfig
std::map<std::string, Parameter&> parameters_;
std::map<std::string, std::shared_ptr<NodeParameter>> node_parameters_;

// Needed? keep track of registered callbacks that are required for linking SetParameters requests or
// exposing high-level update changes (invalid/valid states)
// std::string<std::string, std::string> registered_node_parameter_callbacks_ids_;
// std::string<std::string, std::string> registered_parameter_callbacks_ids_;
}

struct ExampleConfig : public NodeConfig
{
ExampleConfig(const rclcpp::Node::SharedPtr& node) : NodeConfig(node) {
// Build and declare the parameters

// declare simple parameters just like that
declare(int1_param.describe("This is the first int parameter"));
declare(int2_param.describe("This is the second int parameter"));
declare(int3_param.describe("This is the third int parameter"));

// "double_param", add some description and validity checks
declare(double_param
.describe("This is a very important double parameter", "should be greater than 0 and not 7")
.warnIf([](const double& val, std::string& message) {
message = "Value is 7, are you sure this is a good value?";
return val == 7;
})
.errorIf([](const double& val, std::string& message) {
message = "Value must not be negative!";
return val < 0;
})
);

// "string_param", use a parameter error condition "EMPTY"
declare(string_param
.describe("meh")
.warnIf([](const std::string& val, std::string& message) {
message = "string is much too long";
return val.size() > 100;
})
.errorIf(EMPTY)
);

// "double_map_param", validate keys and values, reject if not valid
declare(double_map_param
.describe("6-dof joint state")
.errorIf([](const auto& values, std::string& message) {
message = "Invalid joint state map values, expected 6-dof";
return values.size() != 6 || checkJointNamesInRobotModel(values); // from somewhere
})
);

//
// can already check for errors/warnings here...
//
}

Parameter<double> double_param{ "double_param", 0 };
Parameter<std::vector<std::string>> string_param{ "string_param", {"first", "second"} };
Parameter<std::map<std::string, double>> double_map_param{ "double_map_param"};
Parameter<int> int1_param{ "int1_param", 0 };
Parameter<int> int2_param{ "int2_param", 1 };
Parameter<int> int3_param{ "int3_param", 2 };
}

void useConfig(const rclcpp::Node::SharedPtr& node)
{
// use non-const config for initializing callbacks
// For safety, it should only be accessed as const in nodes
const ExampleConfig config(node);

// access copy of value
double value = config.double_param();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice interface for getting the data. I'm assuming this will lock a mutex to make sure when you are reading it isn't also being updated. Maybe this should throw an exception if the value has not been initialized (no default and value wasn't set). Maybe there should be a read(timeout) method for the parameters with some default value set for the timeout so you can specify the timeout if you want to per-parameter.

That might not be a good idea, we need to keep the interface simple and make it easy to follow a good pattern with it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the read is locked by a shared mutex. I don't know if a timeout is really needed at all. The only scenario I see is if that some thread locks write access and doesn't release the mutex for a long time, which would be a user error that definitely shouldn't happen. I guess that if we really wanted to protect for that, we could use an internal timeout that throws an exception if the value has been locked for too long.


// ...

// React to parameter changes (sub is a bad example, but planning pipeline, plugin etc would work well)
auto sub = rclcpp::create_subscription(config.string_param, ...); // sub managed somewhere else
config.string_param->onChanged(
[&](const std::string& value) { sub = rclcpp::create_subscription(config.value, ...); });

// ...

auto robot_state = /* get robot state copy from somewhere else */;
auto update_robot_state = [this, robot_state](const std::map<std::string, double> values) {
for (const auto& [name, val] : values)
robot_state->setVariablePosition(name, val);

// other thread-safe member function somewhere
this->updateRobotState(robot_state);
};
config.double_map_param.apply(update_robot_state);
config.double_map_param.onChanged(update_robot_state);
Comment on lines +246 to +255
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what this could enable. ROS2 is supposed to be more event driven than ros1 and this will enable more event driven behaviors. I also like that the values are passed in by value here so you don't have to worry about the issues around references and them being updated somewhere else at the same time.

Copy link
Contributor Author

@henningkayser henningkayser Apr 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right? ;) ... huh I guess I missed the reference, the apply() and onChange() calls would be read-locked so using const-ref should be fine. I think it can work both ways, copy or const-ref.

}
} // namespace rosparam_shortcuts