apidefinition target a k8s service by namespace and name

[asoorm]

At present, we need to specify a target_url in order to configure the upstream. https://github.com/TykTechnologies/tyk-operator/blob/master/api/v1alpha1/apidefinition_types.go#L494

This means, that a user, in order to target an internal service, needs to manually construct the target URL in the format:

http://NAME.NAMESPACE.SVC.CLUSTER.LOCAL:PORT

As an alternative, a user should be able to OPTIONALLY configure a service by namespace, name & port as the target.

... api definition stuff
proxy:
  k8s_service:                  <------ k8s prefix to make it clear that it is not part of the internal Tyk API Definition object
    namespace: default
    name: httpbin
    port:
      name: http                 <----- either this
      number: 8000           <----- or this

Internally, when the reconciler finds k8s_service has been set, the reconciler should perform the following actions:

---

Feature: Targeting internal Kubernetes services
  In order to target an internal kubernetes service
  I should be able to specify it's name, namespace and port
  And the reconciler should construct the url of the service in the API Definition resource for me

  Scenario: target_url is already set
    Given the target_url is set
    When I specify a service
    Then the validating webhook should reject the resource as invalid

  Scenario: Service does not exist
    Given the service does not exist
    When I apply the resource
      And I provide reference to the service
    Then the reconciler should return an error and requeue the request

  Scenario: Service exists, but port not specified
    Given the service exists, but the wrong or no port is specified
    When I apply the resource
    Then the reconciler should return an error and requeue the request
    
  Scenario: Service exists and port number exists
    Given the service exists and the port number is specified
    When I apply the resource
    Then the reconciler should create an api definition targeting the service
      And the target url should be of the format: http://SERVICENAME.SERVICENAMESPACE.svc.cluster.local:SERVICEPORT

  Scenario: Service exists and port name exists
    Given the service exists and the port name is specified
    When I apply the resource
    Then the reconciler lookup the portnumber in the service object
      And the reconciler should create an api definition targeting the service
      And the target url should be of the format: http://SERVICENAME.SERVICENAMESPACE.svc.cluster.local:SERVICEPORT

  Scenario: API is of TCP type
    Given the api type is tcp or tls
    When the reconciler reconstructs the URL
    Then the target URL should be of the format: tcp://SERVICENAME.SERVICENAMESPACE.svc.cluster.local:SERVICEPORT

---

Questions:

• what takes priority? Port name or port number? Validating webhook should ensure that only one or the other is selected
• How should we name the field? k8s_service? Or is service acceptable?
• In the event that the upstream is a HTTPS or TLS service, should we force the user to manually set the target URL? Or should we include an extra field protocol in addition to the name, namespace & port?
• should we call it scheme or protocol?
• what should happen if the service is deleted? Should we attach a finalizer to it, block & auto-delete the api definition resource?

@buger @excieve @gwotg @gernest

[buger]

Frankly I do not see anything bad at constructing this url. The same as with yaml config you still need to provide the same values host port and etc. Everyone working in k8s knows about this url format. Do we really solve any issue here, or just overthinking?

[gernest]

In our documentation, I quote

proxy.target_url: This defines the target URL that the request should be proxied to if it passes all checks in Tyk.

It shouldn't real matter how this value is derived(k8s/ad-hoc etc). As long as it is a valid URL resource.

This works under the assumptions that our customers will also be deploying services in the same cluster which is a very huge assumption.

IMHO: keeping a closer mapping between the resource and the resulting apidefinition will be a big win and I don't see this helping.

[asoorm]

I see your points. But I still see massive value in validation of the existence of the underlying service and port.

@buger @gernest what do you think if we took part of this rfc - the validation component. And deconstructed the target_url so that we may validate the target service & port?

[gernest]

target_url so that we may validate the target service & port

Is the goal here to make sure the target is reachable? Assume we remove the loopback targets.

On top of checking if it is a valid url. A dns lookup resolving into an IP should suffice, removing the requirement to know the structure of the url components.

@asoorm can you maybe expand more on this idea in case I missed something?

[buger]

Yes, validation can help here. E.g. if URL ends with "SVC.CLUSTER.LOCAL" actually check if such service exists.
However, I'm not sure that we should stop users from putting this URL, and at maximum send a warning to the console that services do not exist or similar. What if user uses some CI env, and he creates Tyk APIs before service is created.

[asoorm]

The canonical / k8s way is for the controller to spit an error and retry with exponential backoff until the service becomes available. The resource would still be created. But the reconciler should return an error and requeue.

[gernest]

We have settled with specifying both name and namespace of the resource
example

                target_ref:
                  name: proxy-api
                  namespace: default

This is referring to a ApiDefinition resource proxy-api in the default namespace.