Skip to content

Commit b87f07a

Browse files
kinghao007kinghao007
committed
merge 1.9.0
2 parents e3c84aa + 2a3d1b1 commit b87f07a

File tree

397 files changed

+23008
-2249
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

397 files changed

+23008
-2249
lines changed

‎.github/workflows/dlc.json‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
{
2-
"ignorePatterns": [
3-
{
4-
"pattern": "^http://localhost"
2+
"ignorePatterns": [
3+
{
4+
"pattern": "^http://localhost"
5+
},
6+
{
7+
"pattern": "^http://127.0.0.1"
58
},
69
{
7-
"pattern": "^http://127.0.0.1"
10+
"pattern": "^(https?://)?([a-zA-Z0-9-]+\\.)*bilibili\\.com"
811
}
912
],
1013
"timeout": "10s",

‎.github/workflows/integration-test.yml‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,8 @@ jobs:
152152
sleep 20
153153
done
154154
155+
bash ./linkis-dist/helm/scripts/prepare-for-spark.sh
156+
155157
#show linkis pod logs
156158
#POD_NAME=`kubectl get pods -n linkis -l app.kubernetes.io/instance=linkis-demo-cg-linkismanager -o jsonpath='{.items[0].metadata.name}'`
157159
#kubectl logs -n linkis ${POD_NAME} -f --tail=10000
@@ -170,16 +172,15 @@ jobs:
170172
171173
# Execute test by linkis-cli
172174
POD_NAME=`kubectl get pods -n linkis -l app.kubernetes.io/instance=linkis-demo-mg-gateway -o jsonpath='{.items[0].metadata.name}'`
173-
kubectl exec -it -n linkis ${POD_NAME} -- bash -c " \
175+
kubectl exec -n linkis ${POD_NAME} -- bash -c " \
174176
sh /opt/linkis/bin/linkis-cli -engineType shell-1 -codeType shell -code \"pwd\" ";
175177
176-
kubectl exec -it -n linkis ${POD_NAME} -- bash -c " \
178+
kubectl exec -n linkis ${POD_NAME} -- bash -c " \
177179
sh /opt/linkis/bin/linkis-cli -engineType python-python2 -codeType python -code 'print(\"hello\")' "
178180
179-
#todo
180-
#kubectl exec -it -n linkis ${POD_NAME} -- bash -c " \
181-
#sh /opt/linkis/bin/linkis-cli -engineType hive-3.1.3 -codeType hql -code 'show databases' "
181+
kubectl exec -n linkis ${POD_NAME} -- bash -c " \
182+
sh /opt/linkis/bin/linkis-cli -engineType hive-3.1.3 -codeType hql -code 'show databases' "
182183
183-
#kubectl exec -it -n linkis ${POD_NAME} -- bash -c " \
184-
#sh /opt/linkis/bin/linkis-cli -engineType spark-3.2.1 -codeType sql -code 'show databases' "
184+
kubectl exec -n linkis ${POD_NAME} -- bash -c " \
185+
sh /opt/linkis/bin/linkis-cli -engineType spark-3.2.1 -codeType sql -code 'show databases' "
185186
shell: bash

‎README.md‎

Lines changed: 54 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
</h2>
44

55
<p align="center">
6-
<strong> Linkis builds a computation middleware layer to facilitate connection,
6+
<strong> Linkis builds a computation middleware layer to facilitate connection,
77
governance and orchestration between the upper applications and the underlying data engines. </strong>
88
</p>
99
<p align="center">
@@ -14,41 +14,40 @@
1414
</p>
1515

1616
<p align="center">
17-
<a href="https://linkis.apache.org/docs/latest/introduction/" >
18-
<img src="https://img.shields.io/badge/document-English-blue.svg" alt="EN docs" />
19-
</a>
20-
<a href="https://linkis.apache.org/zh-CN/docs/latest/introduction/">
21-
<img src="https://img.shields.io/badge/文档-简体中文-blue.svg" alt="简体中文文档" />
22-
</a>
17+
<a href="https://linkis.apache.org/docs/latest/introduction/" ><!--
18+
--><img src="https://img.shields.io/badge/document-English-blue.svg" alt="EN docs" /><!--
19+
--></a>
20+
<a href="https://linkis.apache.org/zh-CN/docs/latest/introduction/"><!--
21+
--><img src="https://img.shields.io/badge/文档-简体中文-blue.svg" alt="简体中文文档" /><!--
22+
--></a>
2323
</p>
2424

2525
<p align="center">
26-
<a target="_blank" href="https://search.maven.org/search?q=g:org.apache.linkis%20AND%20a:linkis">
27-
<img src="https://img.shields.io/maven-central/v/org.apache.linkis/linkis.svg?label=maven%20central" />
28-
</a>
29-
<a target="_blank" href="https://github.com/apache/linkis/blob/master/LICENSE">
30-
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg?label=license" />
31-
</a>
32-
<a target="_blank" href="https://www.oracle.com/technetwork/java/javase/downloads/index.html">
33-
<img src="https://img.shields.io/badge/JDK-8-green.svg" />
34-
</a>
35-
<a target="_blank" href="https://github.com/apache/linkis/actions">
36-
<img src="https://github.com/apache/linkis/actions/workflows//build-backend.yml/badge.svg" />
37-
</a>
38-
39-
<a target="_blank" href='https://github.com/apache/linkis'>
40-
<img src="https://img.shields.io/github/forks/apache/linkis.svg" alt="github forks"/>
41-
</a>
42-
<a target="_blank" href='https://github.com/apache/linkis'>
43-
<img src="https://img.shields.io/github/stars/apache/linkis.svg" alt="github stars"/>
44-
</a>
45-
<a target="_blank" href='https://github.com/apache/linkis'>
46-
<img src="https://img.shields.io/github/contributors/apache/linkis.svg" alt="github contributors"/>
47-
</a>
48-
<a target="_blank" href="https://badges.toozhao.com/stats/01G7TRNN1PH9PMSCYWDF3EK4QT">
49-
<img src="https://badges.toozhao.com/badges/01G7TRNN1PH9PMSCYWDF3EK4QT/green.svg" />
50-
</a>
51-
26+
<a target="_blank" href="https://search.maven.org/search?q=g:org.apache.linkis%20AND%20a:linkis"><!--
27+
--><img src="https://img.shields.io/maven-central/v/org.apache.linkis/linkis.svg?label=maven%20central" /><!--
28+
--></a>
29+
<a target="_blank" href="https://github.com/apache/linkis/blob/master/LICENSE"><!--
30+
--><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg?label=license" /><!--
31+
--></a>
32+
<a target="_blank" href="https://www.oracle.com/technetwork/java/javase/downloads/index.html"><!--
33+
--><img src="https://img.shields.io/badge/JDK-8-green.svg" /><!--
34+
--></a>
35+
<a target="_blank" href="https://github.com/apache/linkis/actions"><!--
36+
--><img src="https://github.com/apache/linkis/actions/workflows//build-backend.yml/badge.svg" /><!--
37+
--></a>
38+
<a target="_blank" href='https://github.com/apache/linkis'><!--
39+
--><img src="https://img.shields.io/github/forks/apache/linkis.svg" alt="github forks"/><!--
40+
--></a>
41+
<a target="_blank" href='https://github.com/apache/linkis'><!--
42+
--><img src="https://img.shields.io/github/stars/apache/linkis.svg" alt="github stars"/><!--
43+
--></a>
44+
<a target="_blank" href='https://github.com/apache/linkis'><!--
45+
--><img src="https://img.shields.io/github/contributors/apache/linkis.svg" alt="github contributors"/><!--
46+
--></a>
47+
<a target="_blank" href="https://badges.toozhao.com/stats/01G7TRNN1PH9PMSCYWDF3EK4QT"><!--
48+
--><img src="https://badges.toozhao.com/badges/01G7TRNN1PH9PMSCYWDF3EK4QT/green.svg" /><!--
49+
--></a>
50+
5251
</p>
5352
<br/>
5453

@@ -89,21 +88,21 @@ Apache Linkis | DeepWiki : https://deepwiki.com/apache/linkis
8988

9089
# Engine Type
9190

92-
| **Engine name** | **Support underlying component version<br/>(default dependency version)** | **Linkis Version Requirements** | **Included in Release Package By Default** | **Description** |
93-
|:---- |:---- |:---- |:---- |:---- |
94-
|Spark|Apache >= 2.0.0, <br/>CDH >= 5.4.0, <br/>(default Apache Spark 3.2.1)|\>=1.0.3|Yes|Spark EngineConn, supports SQL , Scala, Pyspark and R code|
95-
|Hive|Apache >= 1.0.0, <br/>CDH >= 5.4.0, <br/>(default Apache Hive 3.1.3)|\>=1.0.3|Yes |Hive EngineConn, supports HiveQL code|
96-
|Python|Python >= 2.6, <br/>(default Python2*)|\>=1.0.3|Yes |Python EngineConn, supports python code|
97-
|Shell|Bash >= 2.0|\>=1.0.3|Yes|Shell EngineConn, supports Bash shell code|
98-
|JDBC|MySQL >= 5.0, Hive >=1.2.1, <br/>(default Hive-jdbc 2.3.4)|\>=1.0.3|No |JDBC EngineConn, already supports ClickHouse, DB2, DM, Greenplum, kingbase, MySQL, Oracle, PostgreSQL and SQLServer, can be extended quickly Support other DB, such as SQLite|
99-
|Flink |Flink >= 1.12.2, <br/>(default Apache Flink 1.12.2)|\>=1.0.2|No |Flink EngineConn, supports FlinkSQL code, also supports starting a new Yarn in the form of Flink Jar Application|
100-
|Pipeline|-|\>=1.0.2|No|Pipeline EngineConn, supports file import and export|
101-
|openLooKeng|openLooKeng >= 1.5.0, <br/>(default openLookEng 1.5.0)|\>=1.1.1|No|openLooKeng EngineConn, supports querying data virtualization engine with Sql openLooKeng|
102-
|Sqoop| Sqoop >= 1.4.6, <br/>(default Apache Sqoop 1.4.6)|\>=1.1.2|No|Sqoop EngineConn, support data migration tool Sqoop engine|
103-
|Presto|Presto >= 0.180|\>=1.2.0|No|Presto EngineConn, supports Presto SQL code|
104-
|ElasticSearch|ElasticSearch >=6.0|\>=1.2.0|No|ElasticSearch EngineConn, supports SQL and DSL code|
105-
|Trino | Trino >=371 | >=1.3.1 | No | Trino EngineConn， supports Trino SQL code |
106-
|Seatunnel | Seatunnel >=2.1.2 | >=1.3.1 | No | Seatunnel EngineConn， supportt Seatunnel SQL code |
91+
| **Engine name** | **Support underlying component version<br/>(default dependency version)** | **Linkis Version Requirements** | **Included in Release Package By Default** | **Description** |
92+
| :-------------- | :------------------------------------------------------------------------ | :------------------------------ | :----------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
93+
| Spark | Apache >= 2.0.0, <br/>CDH >= 5.4.0, <br/>(default Apache Spark 3.2.1) | \>=1.0.3 | Yes | Spark EngineConn, supports SQL , Scala, Pyspark and R code |
94+
| Hive | Apache >= 1.0.0, <br/>CDH >= 5.4.0, <br/>(default Apache Hive 3.1.3) | \>=1.0.3 | Yes | Hive EngineConn, supports HiveQL code |
95+
| Python | Python >= 2.6, <br/>(default Python2*) | \>=1.0.3 | Yes | Python EngineConn, supports python code |
96+
| Shell | Bash >= 2.0 | \>=1.0.3 | Yes | Shell EngineConn, supports Bash shell code |
97+
| JDBC | MySQL >= 5.0, Hive >=1.2.1, <br/>(default Hive-jdbc 2.3.4) | \>=1.0.3 | No | JDBC EngineConn, already supports ClickHouse, DB2, DM, Greenplum, kingbase, MySQL, Oracle, PostgreSQL and SQLServer, can be extended quickly Support other DB, such as SQLite |
98+
| Flink | Flink >= 1.12.2, <br/>(default Apache Flink 1.12.2) | \>=1.0.2 | No | Flink EngineConn, supports FlinkSQL code, also supports starting a new Yarn in the form of Flink Jar Application |
99+
| Pipeline | - | \>=1.0.2 | No | Pipeline EngineConn, supports file import and export |
100+
| openLooKeng | openLooKeng >= 1.5.0, <br/>(default openLookEng 1.5.0) | \>=1.1.1 | No | openLooKeng EngineConn, supports querying data virtualization engine with Sql openLooKeng |
101+
| Sqoop | Sqoop >= 1.4.6, <br/>(default Apache Sqoop 1.4.6) | \>=1.1.2 | No | Sqoop EngineConn, support data migration tool Sqoop engine |
102+
| Presto | Presto >= 0.180 | \>=1.2.0 | No | Presto EngineConn, supports Presto SQL code |
103+
| ElasticSearch | ElasticSearch >=6.0 | \>=1.2.0 | No | ElasticSearch EngineConn, supports SQL and DSL code |
104+
| Trino | Trino >=371 | >=1.3.1 | No | Trino EngineConn， supports Trino SQL code |
105+
| Seatunnel | Seatunnel >=2.1.2 | >=1.3.1 | No | Seatunnel EngineConn， supportt Seatunnel SQL code |
107106

108107
# Download
109108

@@ -152,13 +151,13 @@ npm run build
152151
```
153152

154153
### Bundled with MySQL JDBC Driver
155-
Due to the MySQL licensing restrictions, the MySQL Java Database Connectivity (JDBC) driver is not bundled with the
154+
Due to the MySQL licensing restrictions, the MySQL Java Database Connectivity (JDBC) driver is not bundled with the
156155
official released linkis image by default. However, at current stage, linkis still relies on this library to work properly.
157-
To solve this problem, we provide a script which can help to creating a custom image with mysql jdbc from the official
156+
To solve this problem, we provide a script which can help to creating a custom image with mysql jdbc from the official
158157
linkis image by yourself, the image created by this tool will be tagged as `linkis:with-jdbc` by default.
159158

160159
```shell
161-
$> LINKIS_IMAGE=linkis:1.3.1
160+
$> LINKIS_IMAGE=linkis:1.3.1
162161
$> ./linkis-dist/docker/scripts/make-linkis-image-with-mysql-jdbc.sh
163162
```
164163

@@ -167,7 +166,7 @@ Please refer to [Quick Deployment](https://linkis.apache.org/docs/latest/deploym
167166

168167
# Examples and Guidance
169168
- [User Manual](https://linkis.apache.org/docs/latest/user-guide/how-to-use)
170-
- [Engine Usage Documents](https://linkis.apache.org/docs/latest/engine-usage/overview)
169+
- [Engine Usage Documents](https://linkis.apache.org/docs/latest/engine-usage/overview)
171170
- [API Documents](https://linkis.apache.org/docs/latest/api/overview)
172171

173172
# Documentation & Vedio
@@ -186,19 +185,19 @@ Below is the Linkis architecture diagram. You can find more detailed architectur
186185

187186
# Contributing
188187

189-
Contributions are always welcomed, we need more contributors to build Linkis together. either code, or doc, or other supports that could help the community.
188+
Contributions are always welcomed, we need more contributors to build Linkis together. either code, or doc, or other supports that could help the community.
190189
For code and documentation contributions, please follow the [contribution guide](https://linkis.apache.org/community/how-to-contribute).
191190

192191
# Contact Us
193192

194193

195-
- Any questions or suggestions please kindly submit an [issue](https://github.com/apache/linkis/issues).
194+
- Any questions or suggestions please kindly submit an [issue](https://github.com/apache/linkis/issues).
196195
- By mail [[email protected]](mailto:[email protected])
197196
- You can scan the QR code below to join our WeChat group to get more immediate response
198197

199198
<img src="https://linkis.apache.org/Images/wedatasphere_contact_01.png" width="256"/>
200199

201200
# Who is Using Linkis
202201

203-
We opened an issue [[Who is Using Linkis]](https://github.com/apache/linkis/issues/23) for users to feedback and record who is using Linkis.
202+
We opened an issue [[Who is Using Linkis]](https://github.com/apache/linkis/issues/23) for users to feedback and record who is using Linkis.
204203
Since the first release of Linkis in 2019, it has accumulated more than **700** trial companies and **1000+** sandbox trial users, which involving diverse industries, from finance, banking, tele-communication, to manufactory, internet companies and so on.

0 commit comments

Comments
 (0)