Skip to content

Commit 0aafc55

Browse files
turboFeiyaooqinn
turboFei
authored and
yaooqinn
committed
[KYUUBI #304] Add Kyuubi Ctl arguments parser
![turboFei](https://badgen.net/badge/Hello/turboFei/green) [![Closes #465](https://badgen.net/badge/Preview/Closes%20%23465/blue)](https://github.com/yaooqinn/kyuubi/pull/465) ![919](https://badgen.net/badge/%2B/919/red) ![0](https://badgen.net/badge/-/0/green) ![22](https://badgen.net/badge/commits/22/yellow) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT --> <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ ``` bin/kyuubi-service <create|get|delete|list> <server|engine> --zkAddress ... --namespace ... --user ... --host ... --port ... --version Operations: - create - expose a service to a namespace, this case is rare but sometimes we may want one server to be reached in 2 or more namespaces by different user groups - get - get the service node info - delete - delete the specified serviceNode - list - list all the service nodes for a particular domain Role: - server default - engine Args: --zkAddress - one of the zk ensemble address, using kyuubi-defaults/conf if absent --namespace - the namespace, using kyuubi-defaults/conf if absent --user - --host --port --version ``` ### _How was this patch tested?_ UT Closes #465 from turboFei/KYUUBI_304_CMD. Closes #304 4bab34b [fwang12] retest pleaes 8083a12 [fwang12] to increase code converage c7e51a2 [fwang12] complete 7249cd6 [fwang12] add ut c809c27 [fwang12] enable set verbose at first bb3cbb6 [fwang12] fix ut 604820d [fwang12] validate for each action a01ac1f [fwang12] fix scala style issue 76c9b4c [fwang12] save 7139dd5 [fwang12] increase test converage 318ebce [fwang12] add more ut 72978a6 [fwang12] save 2931f93 [fwang12] address comments 10b855d [fwang12] save 420912a [fwang12] treat help as an action b27d0a6 [fwang12] treat help as action 896e20d [fwang12] add ctl module into codecov ea43d69 [fwang12] rename kyuubi-service to kyuubi-ctl 65a0e30 [fwang12] save db718b0 [fwang12] refactor 41b503e [fwang12] Add kyuubi-ctl arguments parser cb3f6a8 [fwang12] with log appender Authored-by: fwang12 <[email protected]> Signed-off-by: Kent Yao <[email protected]>
1 parent a66a905 commit 0aafc55

File tree

12 files changed

+919
-0
lines changed

12 files changed

+919
-0
lines changed

‎dev/kyuubi-codecov/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,12 @@
3838
<version>${project.version}</version>
3939
</dependency>
4040

41+
<dependency>
42+
<groupId>org.apache.kyuubi</groupId>
43+
<artifactId>kyuubi-ctl</artifactId>
44+
<version>${project.version}</version>
45+
</dependency>
46+
4147
<dependency>
4248
<groupId>org.apache.kyuubi</groupId>
4349
<artifactId>kyuubi-ha</artifactId>

‎kyuubi-assembly/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,12 @@
5757
<version>${project.version}</version>
5858
</dependency>
5959

60+
<dependency>
61+
<groupId>org.apache.kyuubi</groupId>
62+
<artifactId>kyuubi-ctl</artifactId>
63+
<version>${project.version}</version>
64+
</dependency>
65+
6066
<dependency>
6167
<groupId>org.apache.hadoop</groupId>
6268
<artifactId>hadoop-client-api</artifactId>

‎kyuubi-common/src/test/scala/org/apache/kyuubi/KyuubiFunSuite.scala‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,11 @@
1717

1818
package org.apache.kyuubi
1919

20+
import scala.collection.mutable.ArrayBuffer
21+
2022
// scalastyle:off
23+
import org.apache.log4j.{Appender, AppenderSkeleton, Level, Logger}
24+
import org.apache.log4j.spi.LoggingEvent
2125
import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach, FunSuite, Outcome}
2226
import org.scalatest.concurrent.Eventually
2327

@@ -52,4 +56,43 @@ trait KyuubiFunSuite extends FunSuite
5256
info(s"\n\n===== FINISHED $shortSuiteName: '$testName' =====\n")
5357
}
5458
}
59+
60+
/**
61+
* Adds a log appender and optionally sets a log level to the root logger or the logger with
62+
* the specified name, then executes the specified function, and in the end removes the log
63+
* appender and restores the log level if necessary.
64+
*/
65+
protected def withLogAppender(
66+
appender: Appender,
67+
loggerName: Option[String] = None,
68+
level: Option[Level] = None)(
69+
f: => Unit): Unit = {
70+
val logger = loggerName.map(Logger.getLogger).getOrElse(Logger.getRootLogger)
71+
val restoreLevel = logger.getLevel
72+
logger.addAppender(appender)
73+
if (level.isDefined) {
74+
logger.setLevel(level.get)
75+
}
76+
try f finally {
77+
logger.removeAppender(appender)
78+
if (level.isDefined) {
79+
logger.setLevel(restoreLevel)
80+
}
81+
}
82+
}
83+
84+
class LogAppender(msg: String = "", maxEvents: Int = 1000) extends AppenderSkeleton {
85+
val loggingEvents = new ArrayBuffer[LoggingEvent]()
86+
87+
override def append(loggingEvent: LoggingEvent): Unit = {
88+
if (loggingEvents.size >= maxEvents) {
89+
val loggingInfo = if (msg == "") "." else s" while logging $msg."
90+
throw new IllegalStateException(
91+
s"Number of events reached the limit of $maxEvents$loggingInfo")
92+
}
93+
loggingEvents.append(loggingEvent)
94+
}
95+
override def close(): Unit = {}
96+
override def requiresLayout(): Boolean = false
97+
}
5598
}

‎kyuubi-ctl/pom.xml‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
~ Licensed to the Apache Software Foundation (ASF) under one or more
4+
~ contributor license agreements. See the NOTICE file distributed with
5+
~ this work for additional information regarding copyright ownership.
6+
~ The ASF licenses this file to You under the Apache License, Version 2.0
7+
~ (the "License"); you may not use this file except in compliance with
8+
~ the License. You may obtain a copy of the License at
9+
~
10+
~ http://www.apache.org/licenses/LICENSE-2.0
11+
~
12+
~ Unless required by applicable law or agreed to in writing, software
13+
~ distributed under the License is distributed on an "AS IS" BASIS,
14+
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
~ See the License for the specific language governing permissions and
16+
~ limitations under the License.
17+
-->
18+
19+
<project xmlns="http://maven.apache.org/POM/4.0.0"
20+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
21+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
22+
<parent>
23+
<groupId>org.apache.kyuubi</groupId>
24+
<artifactId>kyuubi</artifactId>
25+
<version>1.2.0-SNAPSHOT</version>
26+
<relativePath>../pom.xml</relativePath>
27+
</parent>
28+
<modelVersion>4.0.0</modelVersion>
29+
30+
<artifactId>kyuubi-ctl</artifactId>
31+
<packaging>jar</packaging>
32+
<name>Kyuubi Project Control</name>
33+
34+
<dependencies>
35+
<dependency>
36+
<groupId>org.apache.kyuubi</groupId>
37+
<artifactId>kyuubi-common</artifactId>
38+
<version>${project.version}</version>
39+
</dependency>
40+
<dependency>
41+
<groupId>org.apache.kyuubi</groupId>
42+
<artifactId>kyuubi-ha</artifactId>
43+
<version>${project.version}</version>
44+
</dependency>
45+
46+
<dependency>
47+
<groupId>org.apache.hadoop</groupId>
48+
<artifactId>hadoop-client-api</artifactId>
49+
<scope>provided</scope>
50+
</dependency>
51+
52+
<dependency>
53+
<groupId>org.apache.hadoop</groupId>
54+
<artifactId>hadoop-client-runtime</artifactId>
55+
<scope>provided</scope>
56+
</dependency>
57+
58+
<dependency>
59+
<groupId>org.apache.curator</groupId>
60+
<artifactId>curator-framework</artifactId>
61+
</dependency>
62+
63+
<dependency>
64+
<groupId>org.apache.curator</groupId>
65+
<artifactId>curator-recipes</artifactId>
66+
</dependency>
67+
68+
<dependency>
69+
<groupId>org.apache.zookeeper</groupId>
70+
<artifactId>zookeeper</artifactId>
71+
</dependency>
72+
73+
<!-- Begin: for EmbeddedZkServer -->
74+
<dependency>
75+
<groupId>org.apache.curator</groupId>
76+
<artifactId>curator-test</artifactId>
77+
<scope>provided</scope>
78+
</dependency>
79+
<!-- End: for EmbeddedZkServer -->
80+
81+
<dependency>
82+
<groupId>org.apache.kyuubi</groupId>
83+
<artifactId>kyuubi-common</artifactId>
84+
<version>${project.version}</version>
85+
<type>test-jar</type>
86+
<scope>test</scope>
87+
</dependency>
88+
89+
<dependency>
90+
<groupId>org.apache.hadoop</groupId>
91+
<artifactId>hadoop-minikdc</artifactId>
92+
<scope>test</scope>
93+
</dependency>
94+
95+
<dependency>
96+
<groupId>org.apache.directory.server</groupId>
97+
<artifactId>apacheds-service</artifactId>
98+
<scope>test</scope>
99+
</dependency>
100+
</dependencies>
101+
102+
<build>
103+
<outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
104+
<testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
105+
</build>
106+
</project>

‎kyuubi-ctl/src/main/java/org/apache/kyuubi/ctl/KyuubiCtlOptionParser.java‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.kyuubi.ctl;
19+
20+
import java.util.List;
21+
import java.util.regex.Matcher;
22+
import java.util.regex.Pattern;
23+
24+
abstract class KyuubiCtlOptionParser {
25+
protected final String CREATE = "create";
26+
protected final String GET = "get";
27+
protected final String DELETE = "delete";
28+
protected final String LIST = "list";
29+
protected final String SERVER = "server";
30+
protected final String ENGINE = "engine";
31+
32+
protected final String ZK_ADDRESS = "--zkAddress";
33+
protected final String NAMESPACE = "--namespace";
34+
protected final String USER = "--user";
35+
protected final String HOST = "--host";
36+
protected final String PORT = "--port";
37+
protected final String VERSION = "--version";
38+
39+
// Options that do not take arguments.
40+
protected final String HELP = "--help";
41+
protected final String VERBOSE = "--verbose";
42+
43+
final String[][] opts = {
44+
{ ZK_ADDRESS, "-zk" },
45+
{ NAMESPACE, "-ns" },
46+
{ USER, "-u" },
47+
{ HOST, "-h" },
48+
{ PORT, "-p" },
49+
{ VERSION, "-V" },
50+
};
51+
52+
/**
53+
* List of switches (command line options that do not take parameters) recognized by
54+
* kyuubi-ctl.
55+
*/
56+
final String[][] switches = {
57+
{ HELP, "-I" },
58+
{ VERBOSE, "-v" },
59+
};
60+
61+
/**
62+
* Parse action type and service type.
63+
*
64+
* @return offset of remaining arguments.
65+
*/
66+
protected abstract int parseActionAndService(List<String> args);
67+
68+
/**
69+
* Parse a list of kyuubi-ctl command line options.
70+
* <p>
71+
* See KyuubiCtlArguments.scala for a more formal description of available options.
72+
*
73+
* @throws IllegalArgumentException If an error is found during parsing.
74+
*/
75+
protected final void parse(List<String> args) {
76+
Pattern eqSeparatedOpt = Pattern.compile("(--[^=]+)=(.+)");
77+
78+
int idx = parseActionAndService(args);
79+
for (; idx < args.size(); idx++) {
80+
String arg = args.get(idx);
81+
String value = null;
82+
83+
Matcher m = eqSeparatedOpt.matcher(arg);
84+
if (m.matches()) {
85+
arg = m.group(1);
86+
value = m.group(2);
87+
}
88+
89+
// Look for options with a value.
90+
String name = findCliOption(arg, opts);
91+
if (name != null) {
92+
if (value == null) {
93+
if (idx == args.size() - 1) {
94+
throw new IllegalArgumentException(
95+
String.format("Missing argument for option '%s'.", arg));
96+
}
97+
idx++;
98+
value = args.get(idx);
99+
}
100+
if (!handle(name, value)) {
101+
break;
102+
}
103+
continue;
104+
}
105+
106+
// Look for a switch.
107+
name = findCliOption(arg, switches);
108+
if (name != null) {
109+
if (!handle(name, null)) {
110+
break;
111+
}
112+
continue;
113+
}
114+
115+
handleUnknown(arg);
116+
}
117+
}
118+
119+
/**
120+
* Callback for when an option with an argument is parsed.
121+
*
122+
* @param opt The long name of the cli option (might differ from actual command line).
123+
* @param value The value. This will be <i>null</i> if the option does not take a value.
124+
* @return Whether to continue parsing the argument list.
125+
*/
126+
protected abstract boolean handle(String opt, String value);
127+
128+
/**
129+
* Callback for when an unrecognized option is parsed.
130+
*
131+
* @param opt Unrecognized option from the command line.
132+
* @return Whether to continue parsing the argument list.
133+
*/
134+
protected abstract boolean handleUnknown(String opt);
135+
136+
private String findCliOption(String name, String[][] available) {
137+
for (String[] candidates : available) {
138+
for (String candidate : candidates) {
139+
if (candidate.equals(name)) {
140+
return candidates[0];
141+
}
142+
}
143+
}
144+
return null;
145+
}
146+
147+
protected String findSwitches(String name) {
148+
return findCliOption(name, switches);
149+
}
150+
}

‎kyuubi-ctl/src/main/scala/org/apache/kyuubi/ctl/KyuubiCtl.scala‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.kyuubi.ctl
19+
20+
private[ctl] object KyuubiCtlAction extends Enumeration {
21+
type KyuubiCtlAction = Value
22+
val CREATE, GET, DELETE, LIST, HELP = Value
23+
}
24+
25+
private[ctl] object KyuubiCtlActionService extends Enumeration {
26+
type KyuubiCtlActionService = Value
27+
val SERVER, ENGINE = Value
28+
}

0 commit comments

Comments
 (0)