我正在使用windows docker desktop 10上的clickhouse docker图像:
https://hub.docker.com/r/yandex/clickhouse-server/
我已经得到了容器和运行,并在数据加载。我遇到了这样一个问题,ch抱怨在xyz行之前需要一个逗号,但在notepad++中打开文件后,我知道实际上应该有一个逗号:
Code: 27. DB::Exception: Cannot parse input: expected , before . . .
否则会有一个关于线尾的问题:
Code: 117. DB::Exception: Expected end of line: (at row 127249)
它还抱怨:
Could not print diagnostic info because two last rows aren't in buffer (rare case)
我注意到,对于相对较小的文件,我没有遇到任何问题(少于3万行)。但更大的文件是个问题。我以前测试过这些文件,所以我知道它们是好的和可加载的。这似乎是图像中clickhouse的一个问题,因为它甚至不能打印出诊断。有什么问题吗?
编辑:示例
下面用这些数据我得到了上面提到的一个错误。我使用r编写要加载的1000000行文件的脚本:
# generate my data-----------------------------------------------------------
library(data.table)
set.seed(22)
u = runif(1000000, 0, 60) # "noise" to add or subtract from some timepoint
x = runif(1000000, 0, 1)
my_table =
data.table(
pudt=as.POSIXct(u, origin = "2017-02-03 08:00:00"),
count = round(x,2)
)
my_table[
,pudt:=as.character(pudt)]
# write out--------------------------------------
fwrite(my_table, "my_data.csv", row.names = F, col.names = F)
# create my table in clickhouse client
CREATE TABLE test(
pudt DateTime,
count Float32
)engine = Log;
# load the data in powershell-----
$files = Get-ChildItem "where my files are . . . "
foreach ($f in $files){
$outfile = $f.FullName | Write-Host
Get-Date | Write-Host
"Start loading" + $f.FullName | Write-Host
`cat $f.FullName | docker run -i --rm --link some-clickhouse-server:clickhouse-client yandex/clickhouse-client -m --host some-clickhouse-server --query="INSERT INTO test FORMAT CSV"`
"End loading" + $f.FullName | Write-Host
[GC]::Collect()
}
我在这里得到的错误是:
Code: 117. DB::Exception: Expected end of line: (at row 144020)
Could not print diagnostic info because two last rows aren't in buffer (rare case)
我查看了文件,但没有发现我意识到的问题:
1条答案
按热度按时间voj3qocg1#
似乎这是ch的官方错误,我将测试并查看:
https://groups.google.com/forum/#!主题/clickhouse/Obtz5B7\ fw
更新:
通过在13.9上构建自定义clickhouse客户端映像,解决了这个问题。现在很好用。