fix: error on CREATE EXTERNAL TABLE with no files and no explicit schema

[adriangb]

Which issue does this PR close?

• Closes #.

Rationale for this change

When you point CREATE EXTERNAL TABLE at an empty directory (or one that does not exist yet) without specifying an explicit column list, DataFusion silently creates a table with 0 columns. Any query against that table then fails with a confusing "column not found" / "no such column" error that gives no hint that the underlying issue is actually that schema inference had nothing to look at.

This is the same root cause as the discussion on #21806 (comment) — that thread covered it from the angle of benchmark runners hitting it, but the confusion is not specific to benchmarks. Failing at CREATE EXTERNAL TABLE time with a clear, actionable message seemed like the right fix overall.

What changes are included in this PR?

ListingOptions::infer_schema now returns a Plan error when the location yields no files (after the existing 0-byte filter), telling the user to either add data files or declare an explicit schema:

Error during planning: No files found at file:///tmp/empty_dir/. Cannot infer schema from an empty location; either add data files or declare an explicit schema for the table.

Pre-declaring an empty table with an explicit schema (e.g. CREATE EXTERNAL TABLE t(x int) STORED AS PARQUET LOCATION '...' for later INSERT) still works — the inference path is only triggered when no schema is provided.

Are these changes tested?

Yes. New cases in datafusion/sqllogictest/test_files/ddl.slt cover:

• Parquet, CSV, and JSON over an empty location without an explicit schema → all return the new Plan error.
• An empty location with an explicit schema → still works and queries cleanly.
• Schema inference still succeeds once files exist at the location, so the new check does not regress the happy path.

Are there any user-facing changes?

Yes — CREATE EXTERNAL TABLE ... LOCATION '<empty-dir>' without an explicit schema now errors at planning time instead of creating a 0-column table. Anyone relying on the previous behavior must add an explicit schema declaration. The error message tells them how.

Use of AI

This code was written fully by AI. @adriangb gave it a detailed plan and reviewed the code by hand once this PR was opened and CI was green.

[adriangb]

Related context: #21806 (comment) — that thread surfaced the same root cause from the angle of benchmark runners hitting it. The reasoning for fixing it here at the planning layer rather than in the runner is that the confusion isn't specific to benchmarks, so erroring on CREATE EXTERNAL TABLE over an empty location seemed better as a general fix.

[adriangb]

This just means we carry around memory for the ObjectMeta of zero sized files until a couple lines later. I think this is not a big problem.

The alternative is that we error even where there are 0 byte files present. I think that's an interesting discussion: e.g. a completely empty data.csv. Or hive partitioned directories with no data. I think all of these should still require an explicit schema or error, but there are tests that check the opposite behavior:

• test_csv_empty_file — registers tests/data/empty_0_byte.csv (0 bytes, no header, no data) and runs SELECT * FROM empty.
• test_csv_multiple_empty_files — folder of 0-byte CSVs. Same situation.
• it_can_read_empty_ndjson — 0-byte JSON file. Same.
• test_read_empty_parquet — 0-byte parquet file. Same.
• test_read_partitioned_empty_parquet — partition dir with a 0-byte parquet.