Skip to content

Allow KubectlBuildWrapper to work when k8s API server is behind firewall #599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Allow KubectlBuildWrapper to work when k8s API server is behind firewall #599

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d49d108
Adding changes to accomodate to situation when kubernetes API server …
swilkhu Sep 19, 2019
08f696e
Adding changes based on recommendations from reviewer(s)
swilkhu Sep 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubectlBuildWrapper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,15 @@ public class KubectlBuildWrapper extends SimpleBuildWrapper {
private final String serverUrl;
private final String credentialsId;
private final String caCertificate;
private final String https_proxy;

@DataBoundConstructor
public KubectlBuildWrapper(@Nonnull String serverUrl, @Nonnull String credentialsId,
@Nonnull String caCertificate) {
@Nonnull String caCertificate, String https_proxy) {
this.serverUrl = serverUrl;
this.credentialsId = credentialsId;
this.caCertificate = caCertificate;
this.https_proxy = https_proxy;
}

public String getServerUrl() {
Expand All @@ -89,6 +91,10 @@ public String getCaCertificate() {
return caCertificate;
}

public String getHttps_proxy() {
return https_proxy;
}

@Override
public void setUp(Context context, Run<?, ?> build, FilePath workspace, Launcher launcher, TaskListener listener, EnvVars initialEnvironment) throws IOException, InterruptedException {

Expand All @@ -99,6 +105,7 @@ public void setUp(Context context, Run<?, ?> build, FilePath workspace, Launcher
context.setDisposer(new CleanupDisposer(tempFiles));

String tlsConfig;
String kubectlBegin = "";
if (caCertificate != null && !caCertificate.isEmpty()) {
FilePath caCrtFile = workspace.createTempFile("cert-auth", "crt");
String ca = caCertificate;
Expand All @@ -113,8 +120,14 @@ public void setUp(Context context, Run<?, ?> build, FilePath workspace, Launcher
tlsConfig = " --insecure-skip-tls-verify=true";
}

if(this.https_proxy != null && !https_proxy.isEmpty()) {
kubectlBegin += "HTTPS_PROXY="+this.https_proxy+" kubectl config --kubeconfig=\"";
} else {
kubectlBegin += "kubectl config --kubeconfig=\"";
}

int status = launcher.launch()
.cmdAsSingleString("kubectl config --kubeconfig=\"" + configFile.getRemote()
.cmdAsSingleString(kubectlBegin + configFile.getRemote()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use launcher.launch().envs("HTTPS_PROXY="+this.https_proxy).cmdAsSingleString... (possibly refactor it to a method to avoid repeating it)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll see if this way works in my environment and make the change.

+ "\" set-cluster k8s --server=" + serverUrl + tlsConfig)
.join();
if (status != 0) throw new IOException("Failed to run kubectl config "+status);
Expand Down Expand Up @@ -174,18 +187,18 @@ public void setUp(Context context, Run<?, ?> build, FilePath workspace, Launcher
}

status = launcher.launch()
.cmdAsSingleString("kubectl config --kubeconfig=\"" + configFile.getRemote() + "\" set-credentials cluster-admin " + login)
.cmdAsSingleString(kubectlBegin + configFile.getRemote() + "\" set-credentials cluster-admin " + login)
.masks(false, false, false, false, false, false, true)
.join();
if (status != 0) throw new IOException("Failed to run kubectl config "+status);

status = launcher.launch()
.cmdAsSingleString("kubectl config --kubeconfig=\"" + configFile.getRemote() + "\" set-context k8s --cluster=k8s --user=cluster-admin")
.cmdAsSingleString(kubectlBegin + configFile.getRemote() + "\" set-context k8s --cluster=k8s --user=cluster-admin")
.join();
if (status != 0) throw new IOException("Failed to run kubectl config "+status);

status = launcher.launch()
.cmdAsSingleString("kubectl config --kubeconfig=\"" + configFile.getRemote() + "\" use-context k8s")
.cmdAsSingleString(kubectlBegin + configFile.getRemote() + "\" use-context k8s")
.join();
if (status != 0) throw new IOException("Failed to run kubectl config "+status);
}
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesClientProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ static KubernetesClient createClient(KubernetesCloud cloud) throws NoSuchAlgorit
String displayName = cloud.getDisplayName();
final Client c = clients.getIfPresent(displayName);
if (c == null) {
KubernetesClient client = new KubernetesFactoryAdapter(cloud.getServerUrl(), cloud.getNamespace(),
KubernetesClient client = new KubernetesFactoryAdapter(cloud.getServerUrl(), cloud.getHttps_proxy(), cloud.getNamespace(),
cloud.getServerCertificate(), cloud.getCredentialsId(), cloud.isSkipTlsVerify(),
cloud.getConnectTimeout(), cloud.getReadTimeout(), cloud.getMaxRequestsPerHost()).createClient();
clients.put(displayName, new Client(getValidity(cloud), client));
Expand Down
14 changes: 13 additions & 1 deletion src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesCloud.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ public class KubernetesCloud extends Cloud {
@Nonnull
private List<PodTemplate> templates = new ArrayList<>();
private String serverUrl;
private String https_proxy;
@CheckForNull
private String serverCertificate;

Expand Down Expand Up @@ -148,6 +149,7 @@ public KubernetesCloud(@NonNull String name, @NonNull KubernetesCloud source) {
this.defaultsProviderTemplate = source.defaultsProviderTemplate;
this.templates.addAll(source.templates);
this.serverUrl = source.serverUrl;
this.https_proxy = source.https_proxy;
this.skipTlsVerify = source.skipTlsVerify;
this.addMasterProxyEnvVars = source.addMasterProxyEnvVars;
this.namespace = source.namespace;
Expand Down Expand Up @@ -248,6 +250,15 @@ public void setServerUrl(@Nonnull String serverUrl) {
this.serverUrl = serverUrl;
}

public String getHttps_proxy() {
return https_proxy;
}

@DataBoundSetter
public void setHttps_proxy(@Nonnull String https_proxy) {
this.https_proxy = https_proxy;
}

public String getServerCertificate() {
return serverCertificate;
}
Expand Down Expand Up @@ -731,14 +742,15 @@ public FormValidation doTestConnection(@QueryParameter String name, @QueryParame
@QueryParameter String serverCertificate,
@QueryParameter boolean skipTlsVerify,
@QueryParameter String namespace,
@QueryParameter String https_proxy,
@QueryParameter int connectionTimeout,
@QueryParameter int readTimeout) throws Exception {
Jenkins.get().checkPermission(Jenkins.ADMINISTER);

if (StringUtils.isBlank(name))
return FormValidation.error("name is required");

try (KubernetesClient client = new KubernetesFactoryAdapter(serverUrl, namespace,
try (KubernetesClient client = new KubernetesFactoryAdapter(serverUrl, https_proxy, namespace,
Util.fixEmpty(serverCertificate), Util.fixEmpty(credentialsId), skipTlsVerify,
connectionTimeout, readTimeout).createClient()) {
// test listing pods
Expand Down
19 changes: 13 additions & 6 deletions src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesFactoryAdapter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,23 +71,24 @@ public class KubernetesFactoryAdapter {
private final int connectTimeout;
private final int readTimeout;
private final int maxRequestsPerHost;
private final String https_proxy;

public KubernetesFactoryAdapter(String serviceAddress, @CheckForNull String caCertData,
@CheckForNull String credentials, boolean skipTlsVerify) {
this(serviceAddress, null, caCertData, credentials, skipTlsVerify);
this(serviceAddress, null, null, caCertData, credentials, skipTlsVerify);
}

public KubernetesFactoryAdapter(String serviceAddress, String namespace, @CheckForNull String caCertData,
public KubernetesFactoryAdapter(String serviceAddress, @CheckForNull String https_proxy, String namespace, @CheckForNull String caCertData,
@CheckForNull String credentials, boolean skipTlsVerify) {
this(serviceAddress, namespace, caCertData, credentials, skipTlsVerify, DEFAULT_CONNECT_TIMEOUT, DEFAULT_READ_TIMEOUT);
this(serviceAddress, https_proxy, namespace, caCertData, credentials, skipTlsVerify, DEFAULT_CONNECT_TIMEOUT, DEFAULT_READ_TIMEOUT);
}

public KubernetesFactoryAdapter(String serviceAddress, String namespace, @CheckForNull String caCertData,
public KubernetesFactoryAdapter(String serviceAddress, @CheckForNull String https_proxy, String namespace, @CheckForNull String caCertData,
@CheckForNull String credentials, boolean skipTlsVerify, int connectTimeout, int readTimeout) {
this(serviceAddress, namespace, caCertData, credentials, skipTlsVerify, connectTimeout, readTimeout, KubernetesCloud.DEFAULT_MAX_REQUESTS_PER_HOST);
this(serviceAddress, https_proxy, namespace, caCertData, credentials, skipTlsVerify, connectTimeout, readTimeout, KubernetesCloud.DEFAULT_MAX_REQUESTS_PER_HOST);
}

public KubernetesFactoryAdapter(String serviceAddress, String namespace, @CheckForNull String caCertData,
public KubernetesFactoryAdapter(String serviceAddress, @CheckForNull String https_proxy, String namespace, @CheckForNull String caCertData,
@CheckForNull String credentials, boolean skipTlsVerify, int connectTimeout, int readTimeout, int maxRequestsPerHost) {
this.serviceAddress = serviceAddress;
this.namespace = namespace;
Expand All @@ -97,6 +98,7 @@ public KubernetesFactoryAdapter(String serviceAddress, String namespace, @CheckF
this.connectTimeout = connectTimeout;
this.readTimeout = readTimeout;
this.maxRequestsPerHost = maxRequestsPerHost;
this.https_proxy = https_proxy != null && !https_proxy.isEmpty() ? https_proxy : null;
}

private StandardCredentials getCredentials(String credentials) {
Expand Down Expand Up @@ -171,6 +173,11 @@ public KubernetesClient createClient() throws NoSuchAlgorithmException, Unrecove
builder = builder.withRequestTimeout(readTimeout * 1000).withConnectionTimeout(connectTimeout * 1000);
builder.withMaxConcurrentRequestsPerHost(maxRequestsPerHost);

if(https_proxy != null && !https_proxy.isEmpty()) {
LOGGER.info("Http proxy used is " + https_proxy);
builder.withHttpsProxy(https_proxy);
}

if (!StringUtils.isBlank(namespace)) {
builder.withNamespace(namespace);
} else if (StringUtils.isBlank(builder.getNamespace())) {
Expand Down
4 changes: 4 additions & 0 deletions src/main/resources/org/csanchez/jenkins/plugins/kubernetes/KubectlBuildWrapper/config.jelly
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
<f:textbox/>
</f:entry>

<f:entry field="https_proxy" title="${%https_proxy}">
<f:textbox/>
</f:entry>

<f:entry title="${%Certificate of certificate authority (CA)}" field="caCertificate">
<f:textarea/>
</f:entry>
Expand Down
3 changes: 3 additions & 0 deletions ...sources/org/csanchez/jenkins/plugins/kubernetes/KubectlBuildWrapper/help-https_proxy.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<div>
Use proxy when kubernetes api server is behind the firewall.
</div>
6 changes: 5 additions & 1 deletion src/main/resources/org/csanchez/jenkins/plugins/kubernetes/KubernetesCloud/config.jelly
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@
<f:textbox/>
</f:entry>

<f:entry title="${%https_proxy}" field="https_proxy">
<f:textbox/>
</f:entry>

<f:entry title="${%Kubernetes server certificate key}" field="serverCertificate">
<f:textarea/>
</f:entry>
Expand All @@ -25,7 +29,7 @@
<c:select/>
</f:entry>

<f:validateButton title="${%Test Connection}" progress="${%Testing...}" method="testConnection" with="name,serverUrl,credentialsId,serverCertificate,skipTlsVerify,namespace" />
<f:validateButton title="${%Test Connection}" progress="${%Testing...}" method="testConnection" with="name,serverUrl,credentialsId,serverCertificate,skipTlsVerify,namespace,https_proxy" />

<f:entry title="${%Jenkins URL}" field="jenkinsUrl">
<f:textbox />
Expand Down
3 changes: 3 additions & 0 deletions ...n/resources/org/csanchez/jenkins/plugins/kubernetes/KubernetesCloud/help-https_proxy.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<div>
Use proxy when kubernetes api server is behind the firewall.
</div>