Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
44e423b
on yarn job mode
Oct 21, 2018
bcdeaa2
flink1.6+perjob
Oct 29, 2018
7522d5b
flink1.6+perjob
Oct 29, 2018
8a4e043
Merge pull request #22 from DocLi/v1.6.0
yangsishu Oct 30, 2018
6e5aca3
Merge pull request #1 from DTStack/master
hongtao12310 Nov 8, 2018
83edbc2
SQL CEP 实现
Nov 12, 2018
396bac1
Merge branch 'v1.5.0' into v1.5.0
yangsishu Nov 13, 2018
3634f53
Merge pull request #41 from ambition119/v1.5.0
yangsishu Nov 13, 2018
379b9a5
extract outputformat
todd5167 Nov 30, 2018
7a498c9
add output package
todd5167 Dec 3, 2018
72df82e
case sensitive
Dec 3, 2018
a1660c0
oracle side and sink
todd5167 Dec 4, 2018
1e9b14d
fix abstract tableInfo not Serializable
zoudaokoulife Dec 4, 2018
508050c
modify reqRow to a interface
zoudaokoulife Dec 4, 2018
be12caa
the value is case sensitive
Dec 4, 2018
71a1524
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 4, 2018
d798386
oracle side and sink and modify jdbc getdate type
todd5167 Dec 4, 2018
930e919
add extendOutputFormat
todd5167 Dec 4, 2018
cd01ace
flink1.6.2
Dec 5, 2018
302670a
ignore case
Dec 5, 2018
06ddc4c
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 5, 2018
7c2e330
modify MathUtil date parse
todd5167 Dec 5, 2018
d604f91
Merge branch 'v1.5.0_dev' into v1.6.0_dev
Dec 5, 2018
a455b67
add cassandra side and sink
XuQianJin-Stars Dec 5, 2018
87c845f
Merge branch 'v1.5.0_dev' into v1.5.0_dev
yangsishu Dec 5, 2018
0d4b0fd
Merge pull request #55 from XuQianJin-Stars/v1.5.0_dev
yangsishu Dec 5, 2018
1b3bd26
Merge branch 'v1.5.0_dev' into v1.6.0_dev
Dec 5, 2018
3226f6f
bugfix
Dec 5, 2018
d91101c
bugfix
Dec 5, 2018
b382bbe
add cassandra md
XuQianJin-Stars Dec 6, 2018
9189a73
Merge branch 'v1.5.0_dev' of https://github.com/XuQianJin-Stars/flink…
XuQianJin-Stars Dec 6, 2018
a363415
Merge pull request #56 from XuQianJin-Stars/v1.5.0_dev
yangsishu Dec 6, 2018
cd8ff38
Merge branch 'v1.5.0_dev' into v1.6.0_dev
Dec 6, 2018
765bff6
Update README.md
yangsishu Dec 6, 2018
88c701a
fix flink-1.6 local-mode run bug
zoudaokoulife Dec 6, 2018
b41cc0e
password is null
Dec 10, 2018
0b601d7
Merge pull request #2 from DTStack/master
hongtao12310 Dec 10, 2018
735f956
Merge pull request #3 from DTStack/v1.6.0_dev
hongtao12310 Dec 10, 2018
43df5ed
fix kafka offset reset option
hongtao12310 Dec 10, 2018
a75a0b7
fix bug for kafka offset
todd5167 Dec 10, 2018
9c817ec
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 10, 2018
fea6ff5
Merge pull request #60 from infinivision/v1.6.0_dev
zoudaokoulife Dec 10, 2018
41e268c
fix up 用户标识符SqlParse强制为大写,应该保持用户输入不变
Dec 10, 2018
f8d6923
add default database
Dec 10, 2018
8119acd
Merge pull request #61 from ambition119/v1.6.0_dev
yangsishu Dec 10, 2018
f223993
Update README.md
yangsishu Dec 11, 2018
a635352
add pattern match for kafka topic and fix groupId invalid, check met…
todd5167 Dec 11, 2018
88b7da2
add password
Dec 11, 2018
7981f63
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 11, 2018
72a90a9
remove groupId check
todd5167 Dec 11, 2018
d783df6
groupid check null
todd5167 Dec 11, 2018
c9ecc83
reset ExtendOutputFormat
todd5167 Dec 11, 2018
339ea02
remove toUppercase
todd5167 Dec 11, 2018
ead22fa
modify compare fun for repalce sql
todd5167 Dec 11, 2018
b5c994a
Merge branch 'v1.5.0_release'
Dec 11, 2018
21f3f42
merge v1.5.0
Dec 11, 2018
ab515e7
add tmp table to cache
Dec 12, 2018
21f3020
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 12, 2018
7b91b15
Merge branch 'master' of github.com:DTStack/flinkStreamSQL into v1.5.…
Dec 12, 2018
53e9dad
Merge branch 'v1.5.0_dev' into v1.6.0_dev
Dec 12, 2018
3ae08ec
case upper and lower
todd5167 Dec 12, 2018
fcd56da
Merge pull request #5 from DTStack/v1.6.0_dev
hongtao12310 Dec 12, 2018
7406d68
case upper and lower 2
todd5167 Dec 12, 2018
126ae98
Merge branch 'v1.5.0_dev' into v1.5.0
todd5167 Dec 12, 2018
9a83ecc
fix conflict
todd5167 Dec 12, 2018
3bf0023
Merge branch 'v1.5.0' into v1.6.0_dev
todd5167 Dec 12, 2018
931f779
Merge pull request #6 from DTStack/v1.6.0_dev
hongtao12310 Dec 13, 2018
df96ed2
fix kafka offset reset check
hongtao12310 Dec 13, 2018
dc25cf5
fix kafka offset check
hongtao12310 Dec 13, 2018
110f352
fix kafka offset check
hongtao12310 Dec 13, 2018
a4b18db
support int
Dec 13, 2018
26bbe35
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 13, 2018
0016a51
Update README.md
yangsishu Dec 14, 2018
a9d4663
Merge branch 'master' of github.com:DTStack/flinkStreamSQL into v1.5.…
Dec 14, 2018
387cd28
youhua
Dec 17, 2018
91d0890
add kafka sink and read csv,txt format data
todd5167 Dec 18, 2018
f925d41
add class description
todd5167 Dec 18, 2018
774eb84
add defaultParallelism to options
todd5167 Dec 18, 2018
c56c0ad
add serversocket source
todd5167 Dec 18, 2018
94bc99f
code optimize
Dec 19, 2018
b6d2473
async cache get data order or unorder
todd5167 Dec 19, 2018
10a3ebb
modify default cachemode order
todd5167 Dec 19, 2018
e9cc74a
add console sink
XuQianJin-Stars Dec 20, 2018
8837213
Merge pull request #73 from XuQianJin-Stars/v1.6.0_dev
yangsishu Dec 20, 2018
fad6c3a
code
Dec 20, 2018
044b942
Merge branch 'v1.6.0_dev' into v1.6.0_dev
yangsishu Dec 20, 2018
3e17ce0
Merge pull request #68 from infinivision/v1.6.0_dev
yangsishu Dec 20, 2018
83e7ddc
fix kafka offset bug
zoudaokoulife Dec 20, 2018
14f1b08
修改注释
zoudaokoulife Dec 20, 2018
ba29e96
change the doc
XuQianJin-Stars Dec 20, 2018
551222a
Merge pull request #74 from XuQianJin-Stars/v1.6.0_dev
yangsishu Dec 21, 2018
d69121e
code optimese
Dec 21, 2018
d17dcf7
Merge branch 'v1.6.0_dev' of https://github.com/DTStack/flinkStreamSQ…
Dec 21, 2018
009c971
add timezone
todd5167 Dec 22, 2018
987eea8
fix watermark lasttime
todd5167 Dec 22, 2018
a7000cd
side async param
todd5167 Dec 22, 2018
05f10bc
fix asyn rdb write only one record
todd5167 Dec 24, 2018
740f0b5
fix parse bug
todd5167 Dec 25, 2018
8c75469
console sink add log and serversocketSource modify exception scope
todd5167 Dec 25, 2018
916da8b
fix kafka cvs pattern error and remove kafka08 monitor
todd5167 Dec 26, 2018
4540349
more kafka bugfix
Dec 26, 2018
bf0a388
yarnper submit bugfix
Dec 27, 2018
5926e47
Merge branch 'v1.5.0_dev' of github.com:DTStack/flinkStreamSQL into v…
Dec 28, 2018
61d81e8
remove primary key field space
todd5167 Jan 8, 2019
85c2737
hbase remove field to upper
todd5167 Jan 9, 2019
b82edb9
fix mongo read asyncside info bug
todd5167 Jan 10, 2019
12bd803
fix redis async read info bug
todd5167 Jan 10, 2019
84a44b2
mongo rdb syncside bug fix
todd5167 Jan 10, 2019
70b1c77
将所有变量添加到env上
zoudaokoulife Jan 11, 2019
ea8cee4
Merge remote-tracking branch 'origin/v1.5.0_dev' into v1.5.0_dev
zoudaokoulife Jan 11, 2019
075064a
fix rdb asynside bug
todd5167 Jan 11, 2019
e3c19ad
Merge branch 'v1.5.0_dev' into v1.5.0_release
todd5167 Jan 11, 2019
7061719
Merge branch 'v1.5.0_release' into v1.6.0_devfix
todd5167 Jan 11, 2019
6ea1740
fix main args parse
todd5167 Jan 21, 2019
7fb8317
Update kafkaSource.md
yangsishu Feb 27, 2019
6952f53
Update kafkaSource.md
yangsishu Feb 27, 2019
2c8409c
Lex.MYSQL
Feb 27, 2019
1b2512e
null
Feb 28, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,17 @@

# 已支持
* 源表:kafka 0.9,1.x版本
* 维表:mysql,hbase,mongo,redis
* 结果表:mysql,hbase,elasticsearch5.x,mongo,redis
* 维表:mysql,SQlServer,oracle,hbase,mongo,redis,cassandra
* 结果表:mysql,SQlServer,oracle,hbase,elasticsearch5.x,mongo,redis,cassandra,console

# 后续开发计划
* 增加oracle维表,结果表功能
* 增加SQlServer维表,结果表功能
* 增加kafka结果表功能
* 增加SQL支持CEP
* 维表快照
* sql优化(谓词下移等)
* serverSocket 源表
* console 结果表
* kafka avro格式
* topN

## 1 快速起步
### 1.1 运行模式
Expand Down Expand Up @@ -145,19 +146,23 @@ sh submit.sh -sql D:\sideSql.txt -name xctest -remoteSqlPluginPath /opt/dtstack
## 2 结构
### 2.1 源表插件
* [kafka 源表插件](docs/kafkaSource.md)
* [serverSocket 源表插件](docs/serverSocketSource.md)

### 2.2 结果表插件
* [elasticsearch 结果表插件](docs/elasticsearchSink.md)
* [hbase 结果表插件](docs/hbaseSink.md)
* [mysql 结果表插件](docs/mysqlSink.md)
* [mongo 结果表插件](docs/mongoSink.md)
* [redis 结果表插件](docs/redisSink.md)
* [cassandra 结果表插件](docs/cassandraSink.md)
* [console 结果表插件](docs/consoleSink.md)

### 2.3 维表插件
* [hbase 维表插件](docs/hbaseSide.md)
* [mysql 维表插件](docs/mysqlSide.md)
* [mongo 维表插件](docs/mongoSide.md)
* [redis 维表插件](docs/redisSide.md)
* [cassandra 维表插件](docs/cassandraSide.md)

## 3 性能指标(新增)

Expand Down Expand Up @@ -188,7 +193,7 @@ sh submit.sh -sql D:\sideSql.txt -name xctest -remoteSqlPluginPath /opt/dtstack

```

CREATE (scala|table) FUNCTION CHARACTER_LENGTH WITH com.dtstack.Kun
CREATE (scala|table) FUNCTION CHARACTER_LENGTH WITH com.dtstack.Kun;


CREATE TABLE MyTable(
Expand Down
88 changes: 88 additions & 0 deletions cassandra/cassandra-side/cassandra-all-side/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>sql.side.cassandra</artifactId>
<groupId>com.dtstack.flink</groupId>
<version>1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

<artifactId>sql.side.all.cassandra</artifactId>
<name>cassandra-all-side</name>

<packaging>jar</packaging>

<dependencies>
<dependency>
<groupId>com.dtstack.flink</groupId>
<artifactId>sql.side.cassandra.core</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>

</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>

<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.2</version>
<executions>
<execution>
<id>copy-resources</id>
<!-- here the phase you need -->
<phase>package</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<copy todir="${basedir}/../../../plugins/cassandraallside">
<fileset dir="target/">
<include name="${project.artifactId}-${project.version}.jar" />
</fileset>
</copy>

<move file="${basedir}/../../../plugins/cassandraallside/${project.artifactId}-${project.version}.jar"
tofile="${basedir}/../../../plugins/cassandraallside/${project.name}.jar" />
</tasks>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Loading