feat(ws): add spec.podTemplate.ports[] to WorkspaceKind #507
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
google-oss-prow
bot
added
area/controller
google-oss-prow
bot
requested review from
andyatmiami,
kimwnasptd and
thesuperzapper
|
/ok-to-test |
workspaces/controller/config/crd/bases/kubeflow.org_workspacekinds.yaml
Outdated
Show resolved
Hide resolved
| // +kubebuilder:validation:Optional | ||
| HTTPProxy *HTTPProxy `json:"httpProxy,omitempty"` | ||
| Ports []WorkspaceKindPort `json:"ports,omitempty"` |
There was a problem hiding this comment.
Out of curiosity - what is the practical purpose of defining a WorkspaceKind with no Ports ? Why would someone want to do that ?
There was a problem hiding this comment.
This is a great question , we should rethink this 🤔
There was a problem hiding this comment.
We should require at least one port, just so that the frontend does not have to deal with the possibility of a workspace with no ports.
+kubebuilder:validation:MinItems:=1
workspaces/controller/internal/webhook/workspacekind_webhook.go
Outdated
Show resolved
Hide resolved
harshad16
force-pushed
the
port-in-wsk-podtemplate
branch
from
5170f6e to
438e485
Compare
harshad16
force-pushed
the
port-in-wsk-podtemplate
branch
from
438e485 to
3239dde
Compare
|
/ok-to-test |
|
/lgtm testing these changes on a cluster and was able to:
|
| // PortId represents a port identifier | ||
| // - this is NOT used as the Container or Service port name, but as part of the HTTP path | ||
| // - this is used to reference the port in the `imageconfig` ports.[].id | ||
| // - this is also used to reference the port in the podtemplate ports.[].portId | ||
| // |
There was a problem hiding this comment.
| // PortId represents a port identifier | |
| // - this is NOT used as the Container or Service port name, but as part of the HTTP path | |
| // - this is used to reference the port in the `imageconfig` ports.[].id | |
| // - this is also used to reference the port in the podtemplate ports.[].portId | |
| // | |
| // PortId the id of the port |
| // the id of the port | ||
| // - identifier for the port in `imageconfig` ports.[].id | ||
| // +kubebuilder:example="jupyterlab" | ||
| PortId PortId `json:"portId"` |
There was a problem hiding this comment.
Probably lets just make this id, because its kind of implied by the ports list.
harshad16
force-pushed
the
port-in-wsk-podtemplate
branch
from
3239dde to
9532f2f
Compare
- moved protocal from imageconfig.spec.ports to podtemplates.ports - included the podtemplates.ports with defaultdisplayname - add validation webhook for podtemplate.ports - update the sample workspacekind with ports reference - referencing same id for portid in imageconfig and podtemplate.ports Signed-off-by: Harshad Reddy Nalla <[email protected]>
harshad16
force-pushed
the
port-in-wsk-podtemplate
branch
from
9532f2f to
c8382d8
Compare
|
/lgtm Testing Methodology
|
| @@ -340,12 +340,9 @@ func buildServices(ws *kubefloworgv1beta1.Workspace, imageConfigValue *kubeflowo | |||
| services := make([]Service, len(imageConfigValue.Spec.Ports)) | |||
There was a problem hiding this comment.
We might want to first build a map from portId -> port config (from the outer workspace kind ports), so that we can look it up when processing each of the ports.
| @@ -340,12 +340,9 @@ func buildServices(ws *kubefloworgv1beta1.Workspace, imageConfigValue *kubeflowo | |||
| services := make([]Service, len(imageConfigValue.Spec.Ports)) | |||
| for i := range imageConfigValue.Spec.Ports { | |||
| port := imageConfigValue.Spec.Ports[i] | |||
| switch port.Protocol { //nolint:gocritic | |||
There was a problem hiding this comment.
We should keep this switch statement, but instead switch on the protocol from the outer workspace kind ports list.
| HttpPath: fmt.Sprintf("/workspace/%s/%s/%s/", ws.Namespace, ws.Name, port.Id), | ||
| } | ||
| services[i].HttpService = &HttpService{ | ||
| DisplayName: *port.DisplayName, |
There was a problem hiding this comment.
This will null pointer deref error, you should use something like ptr.Deref(port.DisplayName, default DisplayNameForThisPort)
| @@ -342,9 +348,8 @@ func NewExampleWorkspaceKind(name string) *kubefloworgv1beta1.WorkspaceKind { | |||
| Ports: []kubefloworgv1beta1.ImagePort{ | |||
| { | |||
| Id: "jupyterlab", | |||
| DisplayName: "JupyterLab", | |||
| DisplayName: ptr.To("JupyterLab"), | |||
There was a problem hiding this comment.
We should be testing with at least one of these being nil, so that we use the default.
| // +kubebuilder:validation:Optional | ||
| HTTPProxy *HTTPProxy `json:"httpProxy,omitempty"` | ||
| Ports []WorkspaceKindPort `json:"ports,omitempty"` |
There was a problem hiding this comment.
We should require at least one port, just so that the frontend does not have to deal with the possibility of a workspace with no ports.
+kubebuilder:validation:MinItems:=1
| @@ -317,9 +323,8 @@ func NewExampleWorkspaceKind1(name string) *kubefloworgv1beta1.WorkspaceKind { | |||
| Ports: []kubefloworgv1beta1.ImagePort{ | |||
| { | |||
| Id: "jupyterlab", | |||
| DisplayName: "JupyterLab", | |||
| DisplayName: ptr.To("JupyterLab"), | |||
There was a problem hiding this comment.
For one of these, we should have it empty to use the default.
| @@ -650,7 +650,7 @@ func generateStatefulSet(workspace *kubefloworgv1beta1.Workspace, workspaceKind | |||
| seenPorts[port.Port] = true | |||
|
|
|||
| // NOTE: we construct this map for use in the go string templates | |||
| containerPortsIdMap[port.Id] = port | |||
| containerPortsIdMap[string(port.Id)] = port | |||
There was a problem hiding this comment.
This implies that you need to update the type of containerPortsIdMap to map[PortId]kubefloworgv1beta1.ImagePort, note, this might mean we need to change this other places which use this map.
| @@ -516,6 +537,59 @@ func (v *WorkspaceKindValidator) validatePodTemplatePodMetadata(workspaceKind *k | |||
| return errs | |||
| } | |||
|
|
|||
| // validatePorts validates the ports in podTemplate.ports of WorkspaceKind | |||
| func validatePorts(workspaceKind *kubefloworgv1beta1.WorkspaceKind) []*field.Error { | |||
There was a problem hiding this comment.
We can remove this function and just do it in validateImageConfigValue.
- Define a map from which to look up the ports which are defined under
spec.podTemplate.portsby ID:- Note, update
validateImageConfigValueto take in this map rather than doing it each time. - The only thing we might want to validate is that HTTPProxy is only set if protocol is HTTP.
- You don't need to validate them in this loop because we already have min characters etc.
- Note, update
- Add the existence check to the existing loop over ports in
validateImageConfigValue- The only validation we need is if the port "id" exists in the previous map.
- When we encounter one which does not exist, the "path" of the error should look like
spec.podTemplate.options.imageConfig.values[<value_id>].spec.ports[<port_id>].idwith an error likemissing from spec.podTemplate.ports
| @@ -78,6 +78,9 @@ func (v *WorkspaceKindValidator) ValidateCreate(ctx context.Context, obj runtime | |||
| // validate the extra environment variables | |||
| allErrs = append(allErrs, validateExtraEnv(workspaceKind)...) | |||
|
|
|||
| // validate the ports configuration | |||
| allErrs = append(allErrs, validatePorts(workspaceKind)...) | |||
There was a problem hiding this comment.
Let's remove this and instead only validate the ports in validateImageConfigValue.
| @@ -156,6 +159,9 @@ func (v *WorkspaceKindValidator) ValidateUpdate(ctx context.Context, oldObj, new | |||
| allErrs = append(allErrs, validateExtraEnv(newWorkspaceKind)...) | |||
| } | |||
|
|
|||
| // validate the ports configuration | |||
| shouldValidatePorts := !equality.Semantic.DeepEqual(newWorkspaceKind.Spec.PodTemplate.Ports, oldWorkspaceKind.Spec.PodTemplate.Ports) | |||
There was a problem hiding this comment.
Let's structure this a little different and name it shouldValidateAllImageConfigValues:
// if the ports config changed, we need to validate all image config vaules again
var shouldValidateAllImageConfigValues bool
shouldValidateAllImageConfigValues = !equality.Semantic.DeepEqual(newWorkspaceKind.Spec.PodTemplate.Ports, oldWorkspaceKind.Spec.PodTemplate.Ports)This then should cause all imageConfig values to be true in toValidateImageConfigIds.
related: #37
This PR adds
spec.podTemplate.ports[]to workspaceKind CRD, which lets users include ports httpproxy setting for their workspaces.WorkspaceKind CRD changes
Following changes are included:
These changes would be consider while setting the routing for proper traffic controller/routing to the pods.