Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading TXT file as CSV with ST_Read() #329

Open
SrNetoChan opened this issue Jun 5, 2024 · 6 comments
Open

Reading TXT file as CSV with ST_Read() #329

SrNetoChan opened this issue Jun 5, 2024 · 6 comments

Comments

@SrNetoChan
Copy link

I am trying to use st_read to read a *.txt as a CSV without needing to rename the file.

This works fine:

SELECT * FROM st_read('CarrisMetropolitana/stops.csv', allowed_drivers=['CSV']);

But the original file as a txt extension, so this fails

select * from st_read('CarrisMetropolitana/stops.txt',allowed_drivers=['CSV']);

IO Error: GDAL Error (4): `CarrisMetropolitana/stops.txt' not recognized as a supported file format.

Of course I can just rename all files, but what I really wanted to do is read it directly from a compressed GTFS zip file, which fails:

SELECT * FROM st_read('/vsizip/CarrisMetropolitana.zip/stops.txt', allowed_drivers=['CSV']);

But if inside the ziz there's a the same file as with csv extentionit works pretty well.

SELECT * FROM st_read('/vsizip/CarrisMetropolitana.zip/stops.csv', allowed_drivers=['CSV']);
@Mause
Copy link
Member

Mause commented Jun 5, 2024

Is there a reason you aren't using the in-built csv support in duckdb, rather than the support in the spatial extension?

@SrNetoChan
Copy link
Author

@Mause, thanks for your interest. The reason for using st_read is to be able to read files from inside a zip file. Something that is supported by GDAL Virtual File Systems. It also allow to read tat directly from a https hosted file. Which was my final goal. I wasn't able to find a way to open files from a zip file using the in-built csv support. Am I missing something?

@SrNetoChan
Copy link
Author

As far as I was able to find only gzipped files would work. I had no success with normal ZIP files.

@Maxxen
Copy link
Member

Maxxen commented Jun 5, 2024

I could have sworn we ship GDAL's GTFS driver, but apparently we have it disabled. Ill push a fix in my next PR that enables it.

@rouault
Copy link

rouault commented Sep 7, 2024

I am trying to use st_read to read a *.txt as a CSV

This will be fixed with GDAL 3.10 thanks to OSGeo/gdal@06e3fb6

@SrNetoChan
Copy link
Author

Thanks @rouault !! That will be very handy!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants