ids.XSD - Invalid xs:element ref="xs:restriction" in idsValue

[spaih]

Background:
Started building a Plugin for IDS-Data-Exchange and run into technical problems, as the current IDS.xsd is not fit for automatic validation in standard .XSD parser, due to technical mistakes. Did not find any corresponding issue here. Please point me when I’m wrong (2 of 2). New to the party. Programming civil engineer and CAD dev. Hi!
Description:
The ids:idsValue complex type definition includes an xs:element ref="xs:restriction" within its xs:choice group.
Problem:
xs:restriction is a schema-defining component used to derive new types by restricting existing ones (see XML Schema Part 1: Structures Second Edition, Section 2.1.3). It is not an element that should appear in an XML instance document, nor is it valid to be referenced by an xs:element for inclusion in a content model. This construct makes the schema invalid.
Impact:
The schema fails validation with standard XSD processors. It also indicates an unclear intent for representing restricted values in the instance document. As I scrolled through many issue even the intent of using it doesn’t statisfy the desired functionality. As this is the most used DataType, it is crucial to get it right!
Suggested Fix:
Introduce a own restriction class instead of Xs-restriction, with all that community requested condition possibilities: eg. MinValue, MaxValue, MinOccurs, HasToBeUnique, Pattern.

[spaih]

For a technical Fix i did the followin:

Replace <xs:element ref="xs:restriction" minOccurs="1" maxOccurs="1"/>
with <xs:element ref="ids:restriction" minOccurs="1" maxOccurs="1"/>

add

	<xs:element name="restriction">
	  <xs:complexType>
		<xs:sequence>
		  <!-- Annotation for documentation/appinfo -->
		  <xs:element name="annotation" minOccurs="0">
			<xs:complexType>
			  <xs:sequence>
				<xs:element name="documentation" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
				<xs:element name="appinfo" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
			  </xs:sequence>
			</xs:complexType>
		  </xs:element>
		  <!-- Pattern restriction -->
		  <xs:element name="pattern" minOccurs="0" maxOccurs="unbounded">
			  <xs:complexType>
				  <xs:attribute name="value" type="xs:string" use="required"/>
			  </xs:complexType>
		  </xs:element>
		  <!-- Enumeration restriction -->
		  <xs:element name="enumeration" minOccurs="0" maxOccurs="unbounded">
			  <xs:complexType>
				  <xs:attribute name="value" type="xs:string" use="required"/>
			  </xs:complexType>
		  </xs:element>
		  <!-- Length restrictions -->
		  <xs:element name="minLength" type="xs:integer" minOccurs="0"/>
		  <xs:element name="maxLength" type="xs:integer" minOccurs="0"/>
		  <!-- Value restrictions -->
		  <xs:element name="minInclusive" minOccurs="0">
			  <xs:complexType>
				  <xs:attribute name="value" type="xs:string" use="required"/>
			  </xs:complexType>
		  </xs:element>
		  <xs:element name="maxInclusive" minOccurs="0">
		      <xs:complexType>
			      <xs:attribute name="value" type="xs:string" use="required"/>
		      </xs:complexType>
		  </xs:element>
		  <xs:element name="minExclusive" minOccurs="0">
			  <xs:complexType>
				  <xs:attribute name="value" type="xs:string" use="required"/>
			  </xs:complexType>
		  </xs:element>
		  <xs:element name="maxExclusive" minOccurs="0">
			  <xs:complexType>
				  <xs:attribute name="value" type="xs:string" use="required"/>
			  </xs:complexType>
		  </xs:element>
		</xs:sequence>
		<!-- Base type attribute, similar to xs:restriction's base -->
		<xs:attribute name="base" type="xs:string" use="optional"/>
	  </xs:complexType>
	</xs:element>

So we make the restriction custom. Here we could also insert the optional much requested "HasToBeUnique" Boolean.

this works with existing .ids files when something like

          xmlContent = Regex.Replace(
                xmlContent,
                @"<xs:restriction\b[^>]*>.*?</xs:restriction>",
                m => Regex.Replace(m.Value, @"\bxs:", "ids:"),
                RegexOptions.Singleline | RegexOptions.IgnoreCase
             );

is called. That moves the xs:restriction and all it's childs to the ids namespace, that we custom-defined above.

I'm just trying to help here to get the standard following the offical XML schema rules, not because I like to complicate things.

NOTE: There were multiple Edit required to come to a solution that works for current IDS files with a standard XSD parser.

[MatthiasWeise]

@spaih Thank you very much for this workaround! I am using it as suggested for formal verification of IDS files.

However, when reviewing the definitions for max/minLength, I encountered a problem. I wonder why the proposed length restrictions differ from the definitions for min/max/In/Exclusive. Is there a reason for this?

If not, I would suggest using minLength and maxLength similarly to the other definitions. I also suggest adding length as an additional option.

  <!-- Length restrictions -->
  <xs:element name="length" minOccurs="0">
	  <xs:complexType>
		  <xs:attribute name="value" type="xs:string" use="required"/>
	  </xs:complexType>
  </xs:element>
  <xs:element name="minLength" minOccurs="0">
	  <xs:complexType>
		  <xs:attribute name="value" type="xs:string" use="required"/>
	  </xs:complexType>
  </xs:element>
  <xs:element name="maxLength" minOccurs="0">
	  <xs:complexType>
		  <xs:attribute name="value" type="xs:string" use="required"/>
	  </xs:complexType>
  </xs:element>

Just for completeness: In the XML schema there are even more options for the restriction element, but they may not be of interest (totalDigits, whiteSpace, fractDigits).

[andyward]

Regarding the original issue, I agree that the re-use of xs:restriction in the IDS schema needs review in a future release. Partly because we're making life more complex for IDS implementors using the XSD, but also because as @MatthiasWeise has picked up, it's introducing ambiguity in IDS files if people use xs:restriction elements where the intent is not specified in the IDS domain. E.g xs:assertion or xs:whitespace amongst others

e.g. a user could specify totalDigits, whiteSpace, fractDigits etc but, can we guarantee the expected behaviour across the different IDS implementations? There's no reference to these restriction types in the IDS documentation and no IDS test cases either.

It feels like IDS should have extracted the parts we needed into our schema rather than wholesale reference it - which is ultimately what is happening in @spaih's solution above -and in the schema generator in this repo itself.

It may be worth referencing #442 (comment) for details of how you can build code using the current xsd - and how the idslib audit tool handles this.

Here we could also insert the optional much requested "HasToBeUnique" Boolean.

May be worth reviewing #398 (comment) for a proposal on implementing uniqueness in IDS