Skip to content

Commit a60e466

Browse files
authored
Fix index and slice expressions for query (#405)
* Fix index and slice expressions for query Support negative step for slice expressions
1 parent 44aed55 commit a60e466

File tree

8 files changed

+600
-181
lines changed

8 files changed

+600
-181
lines changed

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ Or use a query:
7474
q, _ := query.Compile("$..[user,password]")
7575
results := q.Execute(config)
7676
for ii, item := range results.Values() {
77-
fmt.Println("Query result %d: %v", ii, item)
77+
fmt.Printf("Query result %d: %v\n", ii, item)
7878
}
7979
```
8080

@@ -99,9 +99,9 @@ Go-toml provides two handy command line tools:
9999
go install github.com/pelletier/go-toml/cmd/tomljson
100100
tomljson --help
101101
```
102-
102+
103103
* `jsontoml`: Reads a JSON file and outputs a TOML representation.
104-
104+
105105
```
106106
go install github.com/pelletier/go-toml/cmd/jsontoml
107107
jsontoml --help

query/README.md

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
# Query package
2+
3+
## Overview
4+
5+
Package query performs JSONPath-like queries on a TOML document.
6+
7+
The query path implementation is based loosely on the JSONPath specification:
8+
http://goessner.net/articles/JsonPath/.
9+
10+
The idea behind a query path is to allow quick access to any element, or set
11+
of elements within TOML document, with a single expression.
12+
13+
```go
14+
result, err := query.CompileAndExecute("$.foo.bar.baz", tree)
15+
```
16+
17+
This is roughly equivalent to:
18+
19+
```go
20+
next := tree.Get("foo")
21+
if next != nil {
22+
next = next.Get("bar")
23+
if next != nil {
24+
next = next.Get("baz")
25+
}
26+
}
27+
result := next
28+
```
29+
30+
err is nil if any parsing exception occurs.
31+
32+
If no node in the tree matches the query, result will simply contain an empty list of
33+
items.
34+
35+
As illustrated above, the query path is much more efficient, especially since
36+
the structure of the TOML file can vary. Rather than making assumptions about
37+
a document's structure, a query allows the programmer to make structured
38+
requests into the document, and get zero or more values as a result.
39+
40+
## Query syntax
41+
42+
The syntax of a query begins with a root token, followed by any number
43+
sub-expressions:
44+
45+
```
46+
$
47+
Root of the TOML tree. This must always come first.
48+
.name
49+
Selects child of this node, where 'name' is a TOML key
50+
name.
51+
['name']
52+
Selects child of this node, where 'name' is a string
53+
containing a TOML key name.
54+
[index]
55+
Selcts child array element at 'index'.
56+
..expr
57+
Recursively selects all children, filtered by an a union,
58+
index, or slice expression.
59+
..*
60+
Recursive selection of all nodes at this point in the
61+
tree.
62+
.*
63+
Selects all children of the current node.
64+
[expr,expr]
65+
Union operator - a logical 'or' grouping of two or more
66+
sub-expressions: index, key name, or filter.
67+
[start:end:step]
68+
Slice operator - selects array elements from start to
69+
end-1, at the given step. All three arguments are
70+
optional.
71+
[?(filter)]
72+
Named filter expression - the function 'filter' is
73+
used to filter children at this node.
74+
```
75+
76+
## Query Indexes And Slices
77+
78+
Index expressions perform no bounds checking, and will contribute no
79+
values to the result set if the provided index or index range is invalid.
80+
Negative indexes represent values from the end of the array, counting backwards.
81+
82+
```go
83+
// select the last index of the array named 'foo'
84+
query.CompileAndExecute("$.foo[-1]", tree)
85+
```
86+
87+
Slice expressions are supported, by using ':' to separate a start/end index pair.
88+
89+
```go
90+
// select up to the first five elements in the array
91+
query.CompileAndExecute("$.foo[0:5]", tree)
92+
```
93+
94+
Slice expressions also allow negative indexes for the start and stop
95+
arguments.
96+
97+
```go
98+
// select all array elements except the last one.
99+
query.CompileAndExecute("$.foo[0:-1]", tree)
100+
```
101+
102+
Slice expressions may have an optional stride/step parameter:
103+
104+
```go
105+
// select every other element
106+
query.CompileAndExecute("$.foo[0::2]", tree)
107+
```
108+
109+
Slice start and end parameters are also optional:
110+
111+
```go
112+
// these are all equivalent and select all the values in the array
113+
query.CompileAndExecute("$.foo[:]", tree)
114+
query.CompileAndExecute("$.foo[::]", tree)
115+
query.CompileAndExecute("$.foo[::1]", tree)
116+
query.CompileAndExecute("$.foo[0:]", tree)
117+
query.CompileAndExecute("$.foo[0::]", tree)
118+
query.CompileAndExecute("$.foo[0::1]", tree)
119+
```
120+
121+
## Query Filters
122+
123+
Query filters are used within a Union [,] or single Filter [] expression.
124+
A filter only allows nodes that qualify through to the next expression,
125+
and/or into the result set.
126+
127+
```go
128+
// returns children of foo that are permitted by the 'bar' filter.
129+
query.CompileAndExecute("$.foo[?(bar)]", tree)
130+
```
131+
132+
There are several filters provided with the library:
133+
134+
```
135+
tree
136+
Allows nodes of type Tree.
137+
int
138+
Allows nodes of type int64.
139+
float
140+
Allows nodes of type float64.
141+
string
142+
Allows nodes of type string.
143+
time
144+
Allows nodes of type time.Time.
145+
bool
146+
Allows nodes of type bool.
147+
```
148+
149+
## Query Results
150+
151+
An executed query returns a Result object. This contains the nodes
152+
in the TOML tree that qualify the query expression. Position information
153+
is also available for each value in the set.
154+
155+
```go
156+
// display the results of a query
157+
results := query.CompileAndExecute("$.foo.bar.baz", tree)
158+
for idx, value := results.Values() {
159+
fmt.Println("%v: %v", results.Positions()[idx], value)
160+
}
161+
```
162+
163+
## Compiled Queries
164+
165+
Queries may be executed directly on a Tree object, or compiled ahead
166+
of time and executed discretely. The former is more convenient, but has the
167+
penalty of having to recompile the query expression each time.
168+
169+
```go
170+
// basic query
171+
results := query.CompileAndExecute("$.foo.bar.baz", tree)
172+
173+
// compiled query
174+
query, err := toml.Compile("$.foo.bar.baz")
175+
results := query.Execute(tree)
176+
177+
// run the compiled query again on a different tree
178+
moreResults := query.Execute(anotherTree)
179+
```
180+
181+
## User Defined Query Filters
182+
183+
Filter expressions may also be user defined by using the SetFilter()
184+
function on the Query object. The function must return true/false, which
185+
signifies if the passed node is kept or discarded, respectively.
186+
187+
```go
188+
// create a query that references a user-defined filter
189+
query, _ := query.Compile("$[?(bazOnly)]")
190+
191+
// define the filter, and assign it to the query
192+
query.SetFilter("bazOnly", func(node interface{}) bool{
193+
if tree, ok := node.(*Tree); ok {
194+
return tree.Has("baz")
195+
}
196+
return false // reject all other node types
197+
})
198+
199+
// run the query
200+
query.Execute(tree)
201+
```

query/doc.go

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
// items.
2626
//
2727
// As illustrated above, the query path is much more efficient, especially since
28-
// the structure of the TOML file can vary. Rather than making assumptions about
28+
// the structure of the TOML file can vary. Rather than making assumptions about
2929
// a document's structure, a query allows the programmer to make structured
3030
// requests into the document, and get zero or more values as a result.
3131
//
@@ -35,7 +35,7 @@
3535
// sub-expressions:
3636
//
3737
// $
38-
// Root of the TOML tree. This must always come first.
38+
// Root of the TOML tree. This must always come first.
3939
// .name
4040
// Selects child of this node, where 'name' is a TOML key
4141
// name.
@@ -57,7 +57,7 @@
5757
// sub-expressions: index, key name, or filter.
5858
// [start:end:step]
5959
// Slice operator - selects array elements from start to
60-
// end-1, at the given step. All three arguments are
60+
// end-1, at the given step. All three arguments are
6161
// optional.
6262
// [?(filter)]
6363
// Named filter expression - the function 'filter' is
@@ -80,25 +80,23 @@
8080
// Slice expressions also allow negative indexes for the start and stop
8181
// arguments.
8282
//
83-
// // select all array elements.
83+
// // select all array elements except the last one.
8484
// query.CompileAndExecute("$.foo[0:-1]", tree)
8585
//
8686
// Slice expressions may have an optional stride/step parameter:
8787
//
8888
// // select every other element
89-
// query.CompileAndExecute("$.foo[0:-1:2]", tree)
89+
// query.CompileAndExecute("$.foo[0::2]", tree)
9090
//
9191
// Slice start and end parameters are also optional:
9292
//
9393
// // these are all equivalent and select all the values in the array
9494
// query.CompileAndExecute("$.foo[:]", tree)
95-
// query.CompileAndExecute("$.foo[0:]", tree)
96-
// query.CompileAndExecute("$.foo[:-1]", tree)
97-
// query.CompileAndExecute("$.foo[0:-1:]", tree)
95+
// query.CompileAndExecute("$.foo[::]", tree)
9896
// query.CompileAndExecute("$.foo[::1]", tree)
97+
// query.CompileAndExecute("$.foo[0:]", tree)
98+
// query.CompileAndExecute("$.foo[0::]", tree)
9999
// query.CompileAndExecute("$.foo[0::1]", tree)
100-
// query.CompileAndExecute("$.foo[:-1:1]", tree)
101-
// query.CompileAndExecute("$.foo[0:-1:1]", tree)
102100
//
103101
// Query Filters
104102
//
@@ -126,8 +124,8 @@
126124
//
127125
// Query Results
128126
//
129-
// An executed query returns a Result object. This contains the nodes
130-
// in the TOML tree that qualify the query expression. Position information
127+
// An executed query returns a Result object. This contains the nodes
128+
// in the TOML tree that qualify the query expression. Position information
131129
// is also available for each value in the set.
132130
//
133131
// // display the results of a query
@@ -139,7 +137,7 @@
139137
// Compiled Queries
140138
//
141139
// Queries may be executed directly on a Tree object, or compiled ahead
142-
// of time and executed discretely. The former is more convenient, but has the
140+
// of time and executed discretely. The former is more convenient, but has the
143141
// penalty of having to recompile the query expression each time.
144142
//
145143
// // basic query
@@ -155,7 +153,7 @@
155153
// User Defined Query Filters
156154
//
157155
// Filter expressions may also be user defined by using the SetFilter()
158-
// function on the Query object. The function must return true/false, which
156+
// function on the Query object. The function must return true/false, which
159157
// signifies if the passed node is kept or discarded, respectively.
160158
//
161159
// // create a query that references a user-defined filter
@@ -166,7 +164,7 @@
166164
// if tree, ok := node.(*Tree); ok {
167165
// return tree.Has("baz")
168166
// }
169-
// return false // reject all other node types
167+
// return false // reject all other node types
170168
// })
171169
//
172170
// // run the query

0 commit comments

Comments
 (0)