Add table operations test (seaweedfs#8241)

* Add Trino blog operations test

* Update test/s3tables/catalog_trino/trino_blog_operations_test.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* feat: add table bucket path helpers and filer operations

- Add table object root and table location mapping directories
- Implement ensureDirectory, upsertFile, deleteEntryIfExists helpers
- Support table location bucket mapping for S3 access

* feat: manage table bucket object roots on creation/deletion

- Create .objects directory for table buckets on creation
- Clean up table object bucket paths on deletion
- Enable S3 operations on table bucket object roots

* feat: add table location mapping for Iceberg REST

- Track table location bucket mappings when tables are created/updated/deleted
- Enable location-based routing for S3 operations on table data

* feat: route S3 operations to table bucket object roots

- Route table-s3 bucket names to mapped table paths
- Route table buckets to object root directories
- Support table location bucket mapping lookup

* feat: emit table-s3 locations from Iceberg REST

- Generate unique table-s3 bucket names with UUID suffix
- Store table metadata under table bucket paths
- Return table-s3 locations for Trino compatibility

* fix: handle missing directories in S3 list operations

- Propagate ErrNotFound from ListEntries for non-existent directories
- Treat missing directories as empty results for list operations
- Fixes Trino non-empty location checks on table creation

* test: improve Trino CSV parsing for single-value results

- Sanitize Trino output to skip jline warnings
- Handle single-value CSV results without header rows
- Strip quotes from numeric values in tests

* refactor: use bucket path helpers throughout S3 API

- Replace direct bucket path operations with helper functions
- Leverage centralized table bucket routing logic
- Improve maintainability with consistent path resolution

* fix: add table bucket cache and improve filer error handling

- Cache table bucket lookups to reduce filer overhead on repeated checks
- Use filer_pb.CreateEntry and filer_pb.UpdateEntry helpers to check resp.Error
- Fix delete order in handler_bucket_get_list_delete: delete table object before directory
- Make location mapping errors best-effort: log and continue, don't fail API
- Update table location mappings to delete stale prior bucket mappings on update
- Add 1-second sleep before timestamp time travel query to ensure timestamps are in past
- Fix CSV parsing: examine all lines, not skip first; handle single-value rows

* fix: properly handle stale metadata location mapping cleanup

- Capture oldMetadataLocation before mutation in handleUpdateTable
- Update updateTableLocationMapping to accept both old and new locations
- Use passed-in oldMetadataLocation to detect location changes
- Delete stale mapping only when location actually changes
- Pass empty string for oldLocation in handleCreateTable (new tables have no prior mapping)
- Improve logging to show old -> new location transitions

* refactor: cleanup imports and cache design

- Remove unused 'sync' import from bucket_paths.go
- Use filer_pb.UpdateEntry helper in setExtendedAttribute and deleteExtendedAttribute for consistent error handling
- Add dedicated tableBucketCache map[string]bool to BucketRegistry instead of mixing concerns with metadataCache
- Improve cache separation: table buckets cache is now separate from bucket metadata cache

* fix: improve cache invalidation and add transient error handling

Cache invalidation (critical fix):
- Add tableLocationCache to BucketRegistry for location mapping lookups
- Clear tableBucketCache and tableLocationCache in RemoveBucketMetadata
- Prevents stale cache entries when buckets are deleted/recreated

Transient error handling:
- Only cache table bucket lookups when conclusive (found or ErrNotFound)
- Skip caching on transient errors (network, permission, etc)
- Prevents marking real table buckets as non-table due to transient failures

Performance optimization:
- Cache tableLocationDir results to avoid repeated filer RPCs on hot paths
- tableLocationDir now checks cache before making expensive filer lookups
- Cache stores empty string for 'not found' to avoid redundant lookups

Code clarity:
- Add comment to deleteDirectory explaining DeleteEntry response lacks Error field

* go fmt

* fix: mirror transient error handling in tableLocationDir and optimize bucketDir

Transient error handling:
- tableLocationDir now only caches definitive results
- Mirrors isTableBucket behavior to prevent treating transient errors as permanent misses
- Improves reliability on flaky systems or during recovery

Performance optimization:
- bucketDir avoids redundant isTableBucket call via bucketRoot
- Directly use s3a.option.BucketsPath for regular buckets
- Saves one cache lookup for every non-table bucket operation

* fix: revert bucketDir optimization to preserve bucketRoot logic

The optimization to directly use BucketsPath bypassed bucketRoot's logic
and caused issues with S3 list operations on delimiter+prefix cases.

Revert to using path.Join(s3a.bucketRoot(bucket), bucket) which properly
handles all bucket types and ensures consistent path resolution across
the codebase.

The slight performance cost of an extra cache lookup is worth the correctness
and consistency benefits.

* feat: move table buckets under /buckets

Add a table-bucket marker attribute, reuse bucket metadata cache for table bucket detection, and update list/validation/UI/test paths to treat table buckets as /buckets entries.

* Fix S3 Tables code review issues

- handler_bucket_create.go: Fix bucket existence check to properly validate
  entryResp.Entry before setting s3BucketExists flag (nil Entry should not
  indicate existing bucket)
- bucket_paths.go: Add clarifying comment to bucketRoot() explaining unified
  buckets root path for all bucket types
- file_browser_data.go: Optimize by extracting table bucket check early to
  avoid redundant WithFilerClient call

* Fix list prefix delimiter handling

* Handle list errors conservatively

* Fix Trino FOR TIMESTAMP query - use past timestamp

Iceberg requires the timestamp to be strictly in the past.
Use current_timestamp - interval '1' second instead of current_timestamp.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Author: chrislusf

test/s3tables/catalog_trino/trino_blog_operations_test.go

@@ -0,0 +1,172 @@
+package catalog_trino
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestTrinoBlogOperations(t *testing.T) {
+	env := setupTrinoTest(t)
+	defer env.Cleanup(t)
+
+	schemaName := "blog_ns_" + randomString(6)
+	customersTable := "customers_" + randomString(6)
+	trinoCustomersTable := "trino_customers_" + randomString(6)
+
+	runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("CREATE SCHEMA IF NOT EXISTS iceberg.%s", schemaName))
+	defer runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("DROP SCHEMA IF EXISTS iceberg.%s", schemaName))
+	defer runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("DROP TABLE IF EXISTS iceberg.%s.%s", schemaName, trinoCustomersTable))
+	defer runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("DROP TABLE IF EXISTS iceberg.%s.%s", schemaName, customersTable))
+
+	createCustomersSQL := fmt.Sprintf(`CREATE TABLE IF NOT EXISTS iceberg.%s.%s (
+    customer_sk INT,
+    customer_id VARCHAR,
+    salutation VARCHAR,
+    first_name VARCHAR,
+    last_name VARCHAR,
+    preferred_cust_flag VARCHAR,
+    birth_day INT,
+    birth_month INT,
+    birth_year INT,
+    birth_country VARCHAR,
+    login VARCHAR
+) WITH (
+    format = 'PARQUET',
+    sorted_by = ARRAY['customer_id']
+)`, schemaName, customersTable)
+	runTrinoSQL(t, env.trinoContainer, createCustomersSQL)
+
+	insertCustomersSQL := fmt.Sprintf(`INSERT INTO iceberg.%s.%s VALUES
+    (1, 'AAAAA', 'Mrs', 'Amanda', 'Olson', 'Y', 8, 4, 1984, 'US', 'aolson'),
+    (2, 'AAAAB', 'Mr',  'Leonard', 'Eads',  'N', 22, 6, 2001, 'US', 'leads'),
+    (3, 'BAAAA', 'Mr',  'David',   'White', 'Y', 16, 2, 1999, 'US', 'dwhite'),
+    (4, 'BBAAA', 'Mr',  'Melvin',  'Lee',   'N', 30, 3, 1973, 'US', 'mlee'),
+    (5, 'AACAA', 'Mr',  'Donald',  'Holt',  'N', 2,  6, 1982, 'CA', 'dholt'),
+    (6, 'ABAAA', 'Mrs', 'Jacqueline', 'Harvey', 'N', 5, 12, 1988, 'US', 'jharvey'),
+    (7, 'BBAAA', 'Ms',  'Debbie',  'Ward',  'N', 6,  1, 2006, 'MX', 'dward'),
+    (8, 'ACAAA', 'Mr',  'Tim',     'Strong', 'N', 15, 7, 1976, 'US', 'tstrong')
+`, schemaName, customersTable)
+	runTrinoSQL(t, env.trinoContainer, insertCustomersSQL)
+
+	countOutput := runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT count(*) FROM iceberg.%s.%s", schemaName, customersTable))
+	rowCount := mustParseCSVInt64(t, countOutput)
+	if rowCount != 8 {
+		t.Fatalf("expected 8 rows in customers table, got %d", rowCount)
+	}
+
+	output := runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT first_name FROM iceberg.%s.%s WHERE customer_sk = 1", schemaName, customersTable))
+	if !strings.Contains(output, "Amanda") {
+		t.Fatalf("expected sample query to include Amanda, got: %s", output)
+	}
+
+	ctasSQL := fmt.Sprintf(`CREATE TABLE iceberg.%s.%s
+WITH (
+    format = 'PARQUET'
+)
+AS SELECT * FROM iceberg.%s.%s`, schemaName, trinoCustomersTable, schemaName, customersTable)
+	runTrinoSQL(t, env.trinoContainer, ctasSQL)
+
+	countOutput = runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT count(*) FROM iceberg.%s.%s", schemaName, trinoCustomersTable))
+	rowCount = mustParseCSVInt64(t, countOutput)
+	if rowCount != 8 {
+		t.Fatalf("expected 8 rows in CTAS table, got %d", rowCount)
+	}
+
+	runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("ALTER TABLE iceberg.%s.%s ADD COLUMN updated_at TIMESTAMP", schemaName, trinoCustomersTable))
+	output = runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("DESCRIBE iceberg.%s.%s", schemaName, trinoCustomersTable))
+	if !strings.Contains(output, "updated_at") {
+		t.Fatalf("expected updated_at column in describe output, got: %s", output)
+	}
+
+	runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("UPDATE iceberg.%s.%s SET updated_at = current_timestamp", schemaName, trinoCustomersTable))
+
+	// Sleep to ensure timestamps are in the past for time travel queries
+	time.Sleep(1 * time.Second)
+
+	snapshotOutput := runTrinoSQL(t, env.trinoContainer, fmt.Sprintf(`SELECT snapshot_id FROM iceberg.%s."%s$snapshots" ORDER BY committed_at DESC LIMIT 1`, schemaName, trinoCustomersTable))
+	snapshotID := mustParseCSVInt64(t, snapshotOutput)
+	if snapshotID == 0 {
+		t.Fatalf("expected snapshot ID from snapshots table, got 0")
+	}
+
+	filesOutput := runTrinoSQL(t, env.trinoContainer, fmt.Sprintf(`SELECT file_path FROM iceberg.%s."%s$files" LIMIT 1`, schemaName, trinoCustomersTable))
+	if !hasCSVDataRow(filesOutput) {
+		t.Fatalf("expected files metadata rows, got: %s", filesOutput)
+	}
+
+	historyOutput := runTrinoSQL(t, env.trinoContainer, fmt.Sprintf(`SELECT made_current_at FROM iceberg.%s."%s$history" LIMIT 1`, schemaName, trinoCustomersTable))
+	if !hasCSVDataRow(historyOutput) {
+		t.Fatalf("expected history metadata rows, got: %s", historyOutput)
+	}
+
+	countOutput = runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT count(*) FROM iceberg.%s.%s FOR VERSION AS OF %d", schemaName, trinoCustomersTable, snapshotID))
+	versionCount := mustParseCSVInt64(t, countOutput)
+	if versionCount != 8 {
+		t.Fatalf("expected 8 rows for version time travel, got %d", versionCount)
+	}
+
+	// Use current_timestamp - interval '1 second' to ensure it's in the past (Iceberg requirement)
+	countOutput = runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT count(*) FROM iceberg.%s.%s FOR TIMESTAMP AS OF (current_timestamp - interval '1' second)", schemaName, trinoCustomersTable))
+	timestampCount := mustParseCSVInt64(t, countOutput)
+	if timestampCount != 8 {
+		t.Fatalf("expected 8 rows for timestamp time travel, got %d", timestampCount)
+	}
+
+	runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("DELETE FROM iceberg.%s.%s WHERE customer_sk = 8", schemaName, trinoCustomersTable))
+	countOutput = runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT count(*) FROM iceberg.%s.%s", schemaName, trinoCustomersTable))
+	rowCount = mustParseCSVInt64(t, countOutput)
+	if rowCount != 7 {
+		t.Fatalf("expected 7 rows after delete, got %d", rowCount)
+	}
+
+	runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("ALTER TABLE iceberg.%s.%s EXECUTE rollback_to_snapshot(%d)", schemaName, trinoCustomersTable, snapshotID))
+	countOutput = runTrinoSQL(t, env.trinoContainer, fmt.Sprintf("SELECT count(*) FROM iceberg.%s.%s", schemaName, trinoCustomersTable))
+	rowCount = mustParseCSVInt64(t, countOutput)
+	if rowCount != 8 {
+		t.Fatalf("expected 8 rows after rollback, got %d", rowCount)
+	}
+}
+
+func hasCSVDataRow(output string) bool {
+	lines := strings.Split(strings.TrimSpace(output), "\n")
+	if len(lines) == 0 {
+		return false
+	}
+	for _, line := range lines {
+		if strings.TrimSpace(line) != "" {
+			return true
+		}
+	}
+	return false
+}
+
+func mustParseCSVInt64(t *testing.T, output string) int64 {
+	t.Helper()
+	value := mustFirstCSVValue(t, output)
+	parsed, err := strconv.ParseInt(value, 10, 64)
+	if err != nil {
+		t.Fatalf("failed to parse int from output %q: %v", output, err)
+	}
+	return parsed
+}
+
+func mustFirstCSVValue(t *testing.T, output string) string {
+	t.Helper()
+	lines := strings.Split(strings.TrimSpace(output), "\n")
+	if len(lines) == 0 {
+		t.Fatalf("expected CSV output with data row, got: %q", output)
+	}
+	for _, line := range lines {
+		line = strings.TrimSpace(line)
+		if line == "" {
+			continue
+		}
+		parts := strings.Split(line, ",")
+		return strings.Trim(parts[0], "\"")
+	}
+	t.Fatalf("no CSV data rows found in output: %q", output)
+	return ""
+}

test/s3tables/catalog_trino/trino_catalog_test.go

@@ -319,8 +319,9 @@ func (env *TestEnvironment) writeTrinoConfig(t *testing.T, warehouseBucket strin
 	config := fmt.Sprintf(`connector.name=iceberg
 iceberg.catalog.type=rest
 iceberg.rest-catalog.uri=http://host.docker.internal:%d
-iceberg.rest-catalog.warehouse=s3://%s/
+iceberg.rest-catalog.warehouse=s3tablescatalog/%s
 iceberg.file-format=PARQUET
+iceberg.unique-table-location=true
 
 # S3 storage config
 fs.native-s3.enabled=true
@@ -415,7 +416,7 @@ func runTrinoSQL(t *testing.T, containerName, sql string) string {
 		logs, _ := exec.Command("docker", "logs", containerName).CombinedOutput()
 		t.Fatalf("Trino command failed: %v\nSQL: %s\nOutput:\n%s\nTrino logs:\n%s", err, sql, string(output), string(logs))
 	}
-	return string(output)
+	return sanitizeTrinoOutput(string(output))
 }
 
 func createTableBucket(t *testing.T, env *TestEnvironment, bucketName string) {
@@ -439,6 +440,30 @@ func createTableBucket(t *testing.T, env *TestEnvironment, bucketName string) {
 	t.Logf("Created table bucket: %s", bucketName)
 }
 
+func sanitizeTrinoOutput(output string) string {
+	lines := strings.Split(strings.TrimSpace(output), "\n")
+	filtered := make([]string, 0, len(lines))
+	for _, line := range lines {
+		if strings.Contains(line, "org.jline.utils.Log") {
+			continue
+		}
+		if strings.Contains(line, "Unable to create a system terminal") {
+			continue
+		}
+		if strings.HasPrefix(line, "WARNING:") {
+			continue
+		}
+		if strings.TrimSpace(line) == "" {
+			continue
+		}
+		filtered = append(filtered, line)
+	}
+	if len(filtered) == 0 {
+		return ""
+	}
+	return strings.Join(filtered, "\n") + "\n"
+}
+
 func createObjectBucket(t *testing.T, env *TestEnvironment, bucketName string) {
 	t.Helper()
 

weed/admin/dash/file_browser_data.go

@@ -6,7 +6,9 @@ import (
 	"strings"
 	"time"
 
+	"github.com/seaweedfs/seaweedfs/weed/glog"
 	"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
+	"github.com/seaweedfs/seaweedfs/weed/s3api/s3tables"
 )
 
 // FileEntry represents a file or directory entry in the file browser
@@ -218,25 +220,33 @@ func (s *AdminServer) GetFileBrowser(dir string, lastFileName string, pageSize i
 		}
 	}
 
-	// Check if this is a bucket path
+	// Check if this is a bucket path and determine if it's a table bucket
 	isBucketPath := false
 	bucketName := ""
+	isTableBucketPath := false
+	tableBucketName := ""
 	if strings.HasPrefix(dir, "/buckets/") {
 		isBucketPath = true
 		pathParts := strings.Split(strings.Trim(dir, "/"), "/")
 		if len(pathParts) >= 2 {
 			bucketName = pathParts[1]
-		}
-	}
-
-	// Check if this is a table bucket path
-	isTableBucketPath := false
-	tableBucketName := ""
-	if strings.HasPrefix(dir, "/table-buckets/") {
-		isTableBucketPath = true
-		pathParts := strings.Split(strings.Trim(dir, "/"), "/")
-		if len(pathParts) >= 2 {
-			tableBucketName = pathParts[1]
+			// Check table bucket status early to avoid second WithFilerClient call
+			if err := s.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
+				resp, err := filer_pb.LookupEntry(context.Background(), client, &filer_pb.LookupDirectoryEntryRequest{
+					Directory: "/buckets",
+					Name:      bucketName,
+				})
+				if err != nil {
+					return err
+				}
+				if s3tables.IsTableBucketEntry(resp.Entry) {
+					isTableBucketPath = true
+					tableBucketName = bucketName
+				}
+				return nil
+			}); err != nil {
+				glog.V(1).Infof("file browser table bucket lookup failed for %s: %v", bucketName, err)
+			}
 		}
 	}
 
@@ -287,12 +297,8 @@ func (s *AdminServer) generateBreadcrumbs(dir string) []BreadcrumbItem {
 		displayName := part
 		if len(breadcrumbs) == 1 && part == "buckets" {
 			displayName = "Object Store Buckets"
-		} else if len(breadcrumbs) == 1 && part == "table-buckets" {
-			displayName = "Table Buckets"
 		} else if len(breadcrumbs) == 2 && strings.HasPrefix(dir, "/buckets/") {
 			displayName = "📦 " + part // Add bucket icon to bucket name
-		} else if len(breadcrumbs) == 2 && strings.HasPrefix(dir, "/table-buckets/") {
-			displayName = "🧊 " + part
 		}
 
 		breadcrumbs = append(breadcrumbs, BreadcrumbItem{

weed/admin/dash/file_browser_data_test.go

@@ -51,15 +51,6 @@ func TestGenerateBreadcrumbs(t *testing.T) {
 				{Name: "📦 mybucket", Path: "/buckets/mybucket"},
 			},
 		},
-		{
-			name: "table bucket path",
-			path: "/table-buckets/mytablebucket",
-			expected: []BreadcrumbItem{
-				{Name: "Root", Path: "/"},
-				{Name: "Table Buckets", Path: "/table-buckets"},
-				{Name: "🧊 mytablebucket", Path: "/table-buckets/mytablebucket"},
-			},
-		},
 		{
 			name: "bucket nested path",
 			path: "/buckets/mybucket/folder",
@@ -70,16 +61,6 @@ func TestGenerateBreadcrumbs(t *testing.T) {
 				{Name: "folder", Path: "/buckets/mybucket/folder"},
 			},
 		},
-		{
-			name: "table bucket nested path",
-			path: "/table-buckets/mytablebucket/folder",
-			expected: []BreadcrumbItem{
-				{Name: "Root", Path: "/"},
-				{Name: "Table Buckets", Path: "/table-buckets"},
-				{Name: "🧊 mytablebucket", Path: "/table-buckets/mytablebucket"},
-				{Name: "folder", Path: "/table-buckets/mytablebucket/folder"},
-			},
-		},
 		{
 			name: "path with trailing slash",
 			path: "/folder/",
@@ -195,11 +176,6 @@ func TestParentPathCalculationLogic(t *testing.T) {
 			currentDir: "/buckets/mybucket",
 			expected:   "/buckets",
 		},
-		{
-			name:       "table bucket directory",
-			currentDir: "/table-buckets/mytablebucket",
-			expected:   "/table-buckets",
-		},
 	}
 
 	for _, tt := range tests {

weed/admin/dash/s3tables_management.go

@@ -100,6 +100,9 @@ func (s *AdminServer) GetS3TablesBucketsData(ctx context.Context) (S3TablesBucke
 			if strings.HasPrefix(entry.Entry.Name, ".") {
 				continue
 			}
+			if !s3tables.IsTableBucketEntry(entry.Entry) {
+				continue
+			}
 			metaBytes, ok := entry.Entry.Extended[s3tables.ExtendedKeyMetadata]
 			if !ok {
 				continue