Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue found on page 'Loading JSON' #3977

Open
maxmoehl opened this issue Oct 28, 2024 · 1 comment
Open

Issue found on page 'Loading JSON' #3977

maxmoehl opened this issue Oct 28, 2024 · 1 comment
Assignees
Labels

Comments

@maxmoehl
Copy link

I was importing large JSON objects with many fields and duckdb refused to expand each field into its own column and instead give me a single STRUCT... or MAP(VARCHAR, VARCHAR) column. When messing around with the parameters I stumbled across this list of available arguments:

D SELECT * FROM read_json('./test-flat.json', format='array', auto_detectt=true) LIMIT 1;
Binder Error: Invalid named parameter "auto_detectt" for function read_json
Candidates:
    map_inference_threshold BIGINT
    records VARCHAR
    timestampformat VARCHAR
    field_appearance_threshold DOUBLE
    date_format VARCHAR
    dateformat VARCHAR
    sample_size BIGINT
    columns ANY
    convert_strings_to_integers BOOLEAN
    format VARCHAR
    ignore_errors BOOLEAN
    maximum_object_size UINTEGER
    maximum_depth BIGINT
    auto_detect BOOLEAN
    union_by_name BOOLEAN
    maximum_sample_files BIGINT
    compression VARCHAR
    timestamp_format VARCHAR
    hive_types ANY
    hive_partitioning BOOLEAN
    hive_types_autocast BOOLEAN
    filename ANY

It seems like not all of them are documented, the one I needed in my case was map_inference_threshold which apparently has a default which is too low for my objects, after setting it to -1 the columns got generated. Searching via GitHub also yields no results.

Page URL: https://duckdb.org/docs/data/json/loading_json.html

@szarnyasg
Copy link
Collaborator

Hi @maxmoehl, thanks for reporting this. Indeed there are missing flags – I'm working on documenting them in #3922.

@szarnyasg szarnyasg self-assigned this Oct 30, 2024
@szarnyasg szarnyasg added the json label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants