Should we gracefully handle cases where servers do not support (even) shape tree auxiliary metadata?

[justinwb]

Currently we throw an exception when the client cannot discover a shapetree metadata auxiliary resource advertised by the solid server. Should clients gracefully handle these cases? Options would be:

1. Don't do any validation at all - act like a normal client
2. Use an alternative location, such as a general metadata auxiliary resource... Given the fact that our logic relies on the server providing link relations to locators, I'm not sure this would be feasible. If the server could be configured to do that, it could be configured to just do it properly in the first place.
3. Halt and catch fire

@NotNull
public String getMetadataURI() throws IOException {
    if (!this.parsedLinkHeaders.containsKey(LinkRelations.SHAPETREE_LOCATOR.getValue())) {
        log.error("The resource {} does not contain a link header of {}", this.getUri(), LinkRelations.SHAPETREE_LOCATOR.getValue());
        // TODO: Should this be gracefully handled by the client?
        throw new ShapeTreeException(500, "No Link header with relation of " + LinkRelations.SHAPETREE_LOCATOR.getValue() + " found");
    }
    String metaDataURIString = this.parsedLinkHeaders.get(LinkRelations.SHAPETREE_LOCATOR.getValue()).stream().findFirst().orElse(null);
    if (metaDataURIString != null && metaDataURIString.startsWith("/")) {
        // If the header value doesn't include scheme/host, prefix it with the scheme & host from container
        URI shapeTreeContainerURI = this.getUri();
        String portFragment;
        if (shapeTreeContainerURI.getPort() > 0) {
            portFragment = ":" + shapeTreeContainerURI.getPort();
        } else {
            portFragment = "";
        }
        metaDataURIString = shapeTreeContainerURI.getScheme() + "://" + shapeTreeContainerURI.getHost() + portFragment + metaDataURIString;
    }

    if (metaDataURIString == null) {
        throw new ShapeTreeException(500, "No Link header with relation of " + LinkRelations.SHAPETREE_LOCATOR.getValue() + " found");
    }

    return metaDataURIString;
}

[ericprud]

I think the use case for this is if some program uses the code for interception and it has to talk to multiple PODs, some which preserve aux data. For those that don't, the client program could impose its own user-configurable conventions via an extended -client-http API. The program needs to:

1. catch our current exceptions to know that some site isn't accepting or regurgitating aux data headers,
2. override HttpRemoteResource.getMetadataURI() (or, more obliquely, imitate the server functionality by extending HtttpClient.fetchIntoRemoteResource()

This might all be doable by extending an existing HttpClient.

[justinwb]

1. catch our current exceptions to know that some site isn't accepting or regurgitating aux data headers

This is probably the right place to start (and maybe where to stay). I guess it boils down to more graceful handling with a bit more informative logging. I think keeping a bit of (in-memory) cache about servers that don't support shape trees so to limit the log pollution could be useful so we don't show the same thing over and over again.