Repository navigation
[client-v2] JSON Support #1833
Description
Activity
@chernser any idea when 0.7.2 Java client will be released?
Hi @dzmitry-biatenia-smop!
We plan to release 0.7.2 by end of November. However this functionality is not in the 0.7.2: it is a complex and experimental feature - so it will take time to add it to the client.
However 0.7.2 might have a minimal support when JSON columns can be read/written as a string.How would you use JSON? Do you have a DTO object or do you work directly with some generic classes generated by GSON or Jackson ?
Thanks!
@chernser the basic case - I was able to create a column with JSON type in Clickhouse table with flag allow_experimental_json_type enabled, and even was able to insert some JSON data there with com.clickhouse.client.api.Client#execute(), but when can running any com.clickhouse.client.api.Client.query("select * from table_with_json_column") I'm getting Unsupported format: "JSON" coming from here because response has JSON type here.
I tried to set output_format_binary_write_json_as_string , but seems it's not supported by my version of server 24.8.4@dzmitry-biatenia-smop
I will fix this issue for sure.@chernser are you going to release that fix separately? any ETA for that fix?
@dzmitry-biatenia-smop
patch should be release by end of this week.
release is by end of the month
another option is to take nightly build.@chernser can you please notify me when that patch is ready or point me to the correct nightly build?
@dzmitry-biatenia-smop
yes, sure!Reacted by Dima BiateniaGood day, @dzmitry-biatenia-smop ! I've just merged the JSON support (see example).
Nightly build should be ready in a several minutes. Here is how to use it https://gist.github.com/chernser/b4eacfde70093847449e3aef8e51ae8e@chernser I tried nightly build as per example you've provided with dependency on new nightly build,
<repository> <id>ossrh</id> <name>Sonatype OSSRH</name> <url>https://s01.oss.sonatype.org/content/repositories/snapshots/</url> </repository>with version
<dependency> <groupId>com.clickhouse</groupId> <artifactId>client-v2</artifactId> <version>0.7.1-SNAPSHOT</version> </dependency>and event could not build a client with those options:
.serverSetting(ClientSettings.INPUT_FORMAT_BINARY_READ_JSON_AS_STRING, "1")
.serverSetting(ClientSettings.OUTPUT_FORMAT_BINARY_WRITE_JSON_AS_STRING, "1")
getting
Setting input_format_binary_read_json_as_string is neither a builtin setting nor started with the prefix 'SQL_' registered for user-defined settings: Maybe you meant ['input_format_json_read_bools_as_strings','input_format_json_read_objects_as_strings']. (UNKNOWN_SETTING) (version 24.8.4.13 (official build))
or
Setting output_format_binary_write_json_as_string is neither a builtin setting nor started with the prefix 'SQL_' registered for user-defined settings: Maybe you meant ['output_format_arrow_string_as_string','output_format_parquet_string_as_string']. (UNKNOWN_SETTING) (version 24.8.4.13 (official build))
errors.
When I'm trying to read JSON in java client without those options, I'm getting another runtime error:
Reading JSON from binary is not implemented yetPlease, advise
Reacted by JiutwoHi @chernser!
Are there any updates or plans to resolve this issue? We’re very interested in leveraging the potential of JSON in ClickHouse for working with data in Metabase, but as far as I understand, this is currently a blocker.
Is there any way we could help accelerate resolving this, if so, feel free to reach out.21 remaining items
Hi @chernser! When reading binary JSON,
getString()can return Java Map-style text instead of valid JSON. I’d like to address this in the driver, without requiringoutput_format_binary_write_json_as_string=1.Is this already being worked on? If so, is there a branch or PR I could contribute to?
If not, would you support adding this, and what existing code would you recommend reusing?
Or should serialization to valid JSON be handled on the client side, using a library like Gson on the returned Map?Good day, @bakhtiiartashbolotov !
You can contribute anyway: code, spec, usecase.
First we need to have approved specification for the feature.
For this feature we need separate reader that can convert toGSONorJacksonobjects. But it would be nice to have some use-cases.
We have something similar forJSONEachRowHi @chernser! I added two small demos using the same binary fixtures:
- Map demo: existing reader output serialized with Gson.
- Reader demo: existing decoder with captured Tuple names and JSON path grouping.
For a JSON path typed as
Tuple(x Int32, y String):Map → Gson: {"a":[1,"z"]} Reader demo: {"a":{"x":1,"y":"z"}}Both branches contain only tests and documentation, with no production/JDBC changes or JSON-as-string server setting. The reader demo is a test-only prototype, not a complete separate reader.
Memory trade-offs are documented qualitatively; there are no numerical memory measurements or speed comparisons.
Are these use cases a useful basis for the specification? What return type and integration point would you prefer for the separate reader?
Description
Client should support JSON Data type (https://clickhouse.com/docs/en/integrations/data-formats/json/overview).
Considerations:
See also: