[BUG] QueryShardException[failed to create query: field expansion for [*] matches too many fields, limit: 1024, got: 1572]

[Hailong-am]

What is the bug?
When i accidentally run an incorrect PPL command, it doesn't report PPL syntax error instead of it reporting query exception

POST _plugins/_ppl/
{
  "query": "source source=train-ticket-metrics-81 | head 10"
}

"{\n \"error\": {\n \"reason\": \"There was internal problem at backend\",\n \"details\": \"java.sql.SQLException: exception while executing query: Failed to fetch data from the index: the background task failed or interrupted.\\n Inner error: Failed to execute phase [query], all shards failed; shardFailures {[R3azv5VmSbWa-Pduq7IN8A][train-ticket-metrics-81][0]: RemoteTransportException[[bcd07463160c][10.118.165.75:9310][indices:data/read/search[phase/query]]]; nested: QueryShardException[failed to create query: field expansion for [*] matches too many fields, limit: 1024, got: 1572]; nested: IllegalArgumentException[field expansion for [*] matches too many fields, limit: 1024, got: 1572]; }\",\n \"type\": \"RuntimeException\"\n },\n \"status\": 500\n}"

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

What is the expected behavior?

syntax error reported

What is your host/environment?

• OS: [e.g. iOS]
• Version [e.g. 22]
• Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?

The index contains 1572 fileds

[penghuo]

Your query source source=train-ticket-metrics-81 is syntactically valid PPL—the first source parses as a search term, generating SELECT * FROM train-ticket-metrics-81 WHERE source=<value>. This triggers field expansion on your 1572-field index, hitting OpenSearch's 1024-field limit (indices.query.bool.max_clause_count).

Why confusing: You get an OpenSearch infrastructure error (QueryShardException: field expansion...) instead of a clear PPL error. The error doesn't explain the duplicate source keyword issue.

Expected: A clear message like: "Query failed: Index has 1572 fields, exceeds limit of 1024. Use explicit field selection with '| fields field1, field2' or contact admin to increase cluster limit."

Related Issues

• [RFC] Error Report-building Mechanism for PPL Errors #4919 - RFC for better PPL error handling
• [BUG] GROUP BY Missing field returns error #743 - Similar confusing "internal problem at backend" errors

[Hailong-am]

Your query source source=train-ticket-metrics-81 is syntactically valid PPL—the first source parses as a search term, generating SELECT * FROM train-ticket-metrics-81 WHERE source=<value>. This triggers field expansion on your 1572-field index, hitting OpenSearch's 1024-field limit (indices.query.bool.max_clause_count).

Why confusing: You get an OpenSearch infrastructure error (QueryShardException: field expansion...) instead of a clear PPL error. The error doesn't explain the duplicate source keyword issue.

Expected: A clear message like: "Query failed: Index has 1572 fields, exceeds limit of 1024. Use explicit field selection with '| fields field1, field2' or contact admin to increase cluster limit."

Related Issues

* [[RFC] Error Report-building Mechanism for PPL Errors #4919](https://github.com/opensearch-project/sql/issues/4919) - RFC for better PPL error handling

* [[BUG] GROUP BY Missing field returns error #743](https://github.com/opensearch-project/sql/issues/743) - Similar confusing "internal problem at backend" errors

Thanks @penghuo, another confusing things is that source=testindex where a_field = 'xxxx', the pipe | is missing in the PPL and the PPL didn't reports error and just return 0 rows. I would expect there have an error saying that | is missing.

[penghuo]

Thanks @penghuo, another confusing things is that source=testindex where a_field = 'xxxx', the pipe | is missing in the PPL and the PPL didn't reports error and just return 0 rows. I would expect there have an error saying that | is missing.

Thanks @Hailong-am for reporting this confusing behavior.

Analysis

The query source=testindex where a_field = 'xxxx' is actually syntactically valid PPL, which is why no error is reported. However, it doesn't do what you might expect.

What's happening

When the pipe | is missing, where is parsed as a free-text search term (not as a command). The query becomes:

  - `source=testindex` → from clause
  - `where` → search for "where" in **any field**
  - `a_field = 'xxxx'` → field comparison

How to debug with Explain API

You can use the PPL explain API to see the actual DSL query being generated:

Incorrect query (missing pipe):

  POST /_plugins/_ppl/_explain
  {"query": "source=testindex where a_field = "xxxx""}

Response shows:

"query_string": {"query": "(where) AND (a_field:xxxx)"}

This searches for documents where any field contains "where" AND a_field = 'xxxx' → Returns 0 rows (unless a document actually contains the word "where" in some field)