Skip to content

Commit 158e531

Browse files
fxamackerfxamacker
authored
Add v2.0 API, faster CBOR, fewer allocs
Implement v2.0 API for easier CBOR encoding/decoding. See issue #117 for summary. Replaced EncOptions.TimeRFC3339 bool with TimeMode values: * TimeUnix // secs, converted to smallest CBOR integer that fits * TimeUnixMicro // μs, CBOR float with subsecs in fractional part * TimeUnixDynamic // secs or μs, either int or float depending on empty subsecs * TimeRFC3339 // secs, string * TimeRFC3339Nano // ns, string with trailing zeros removed Decode CBOR time NaN and Infinity values to Go's "zero time" value. This is how CBOR Null values are decoded to time.Time. Removed deprecated options (Canonical & CTAP2Canoical) in EncOptions. Removed Valid() function. Improved performance and memory alloc by * using pointer receivers for types implementing sort.Interface and others. * storing pointer in sync.Pool. * storing pointer as map value in sync.Map. Used roughly 20 linters and resolved many reported issues. Switched to GitHub Actions for CI to lint, test, and check code coverage. Extensive improvements to README.md. Thanks @x448 for API design ideas, travis & github ci configs, filing issues, and improving README.md. Closes: #107 Closes: #119 Closes: #120 Closes: #123 Closes: #124 Closes: #127
1 parent 7489a7c commit 158e531

18 files changed

+1641
-1360
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,10 @@
1-
# GitHub Actions - CI for Go to build & test. See ci-go-cover.yml for code coverage.
1+
# GitHub Actions - CI for Go to build & test. See ci-go-cover.yml and linters.yml for code coverage and linters.
22
# https://github.com/fxamacker/cbor/workflows/ci.yml
33
name: ci
44
on: [push]
55
jobs:
66

7-
# Check linters on latest-ubuntu with default version of Go.
8-
lint:
9-
name: Lint
10-
runs-on: ubuntu-latest
11-
steps:
12-
- name: Checkout code
13-
uses: actions/checkout@v1
14-
with:
15-
fetch-depth: 2
16-
- name: Install golangci-lint
17-
run: |
18-
go version
19-
curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.23.1
20-
- name: Run required linters in .golangci.yml plus hard-coded ones here
21-
# TODO: remove --issues-exit-code=0 after deciding which linters are required vs optional.
22-
run: $(go env GOPATH)/bin/golangci-lint run --timeout=5m --issues-exit-code=0 -E deadcode -E errcheck -E gofmt -E golint -E gosec -E govet -E ineffassign -E maligned -E staticcheck -E structcheck -E unconvert -E varcheck
23-
- name: Run optional linters (not required to pass)
24-
run: $(go env GOPATH)/bin/golangci-lint run --timeout=5m --issues-exit-code=0 -E dupl -E gocritic -E gosimple -E lll -E prealloc -E deadcode -E errcheck -E gofmt -E golint -E gosec -E govet -E ineffassign -E maligned -E staticcheck -E structcheck -E unconvert -E varcheck
25-
267
# Test on various OS with default Go version.
27-
# As of January 26, 2020:
28-
# 1. Specifying Go version adds 10s because it gets installed. Someday, maybe latest 2-3 versions will be preinstalled + symlink used.
29-
# 2. It takes windows-latest around 47s while others take 6s-16s. Not sure if Nano is used or a more full-featured (slower) Windows.
308
tests:
319
name: Test on ${{matrix.os}}
3210
runs-on: ${{ matrix.os }}

‎.github/workflows/linters.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Go Linters - GitHub Actions
2+
name: linters
3+
on: [push]
4+
jobs:
5+
6+
# Check linters on latest-ubuntu with default version of Go.
7+
lint:
8+
name: Lint
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout code
12+
uses: actions/checkout@v2
13+
- name: Install golangci-lint
14+
run: |
15+
go version
16+
curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.23.1
17+
- name: Run required linters in .golangci.yml plus hard-coded ones here
18+
run: $(go env GOPATH)/bin/golangci-lint run --timeout=5m -E deadcode -E errcheck -E gofmt -E golint -E gosec -E govet -E ineffassign -E maligned -E staticcheck -E structcheck -E unconvert -E varcheck
19+
- name: Run optional linters (not required to pass)
20+
run: $(go env GOPATH)/bin/golangci-lint run --timeout=5m --issues-exit-code=0 -E dupl -E gocritic -E gosimple -E lll -E prealloc -E deadcode -E errcheck -E gofmt -E golint -E gosec -E govet -E ineffassign -E maligned -E staticcheck -E structcheck -E unconvert -E varcheck
21+

‎.golangci.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ linters-settings:
88
statements: 50
99
goconst:
1010
min-len: 2
11-
min-occurrences: 2
11+
min-occurrences: 3
1212
gocritic:
1313
enabled-tags:
1414
- diagnostic
@@ -62,6 +62,10 @@ linters:
6262

6363

6464
issues:
65+
# max-issues-per-linter default is 50. Set to 0 to disable limit.
66+
max-issues-per-linter: 0
67+
# max-same-issues default is 3. Set to 0 to disable limit.
68+
max-same-issues: 0
6569
# Excluding configuration per-path, per-linter, per-text and per-source
6670
exclude-rules:
6771
- path: _test\.go

‎CBOR_BENCHMARKS.md‎

Lines changed: 46 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -26,34 +26,34 @@ Benchmarks use data representing the following values:
2626

2727
Decoding Benchmark | Time | Memory | Allocs
2828
--- | ---: | ---: | ---:
29-
BenchmarkUnmarshal/CBOR_bool_to_Go_interface_{}-2 | 117 ns/op | 16 B/op | 1 allocs/op
30-
BenchmarkUnmarshal/CBOR_bool_to_Go_bool-2 | 91.4 ns/op | 1 B/op | 1 allocs/op
31-
BenchmarkUnmarshal/CBOR_positive_int_to_Go_interface_{}-2 | 140 ns/op | 24 B/op | 2 allocs/op
32-
BenchmarkUnmarshal/CBOR_positive_int_to_Go_uint64-2 | 103 ns/op | 8 B/op | 1 allocs/op
33-
BenchmarkUnmarshal/CBOR_negative_int_to_Go_interface_{}-2 | 139 ns/op | 24 B/op | 2 allocs/op
34-
BenchmarkUnmarshal/CBOR_negative_int_to_Go_int64-2 | 102 ns/op | 8 B/op | 1 allocs/op
35-
BenchmarkUnmarshal/CBOR_float_to_Go_interface_{}-2 | 141 ns/op | 24 B/op | 2 allocs/op
29+
BenchmarkUnmarshal/CBOR_bool_to_Go_interface_{}-2 | 115 ns/op | 16 B/op | 1 allocs/op
30+
BenchmarkUnmarshal/CBOR_bool_to_Go_bool-2 | 91.1 ns/op | 1 B/op | 1 allocs/op
31+
BenchmarkUnmarshal/CBOR_positive_int_to_Go_interface_{}-2 | 138 ns/op | 24 B/op | 2 allocs/op
32+
BenchmarkUnmarshal/CBOR_positive_int_to_Go_uint64-2 | 104 ns/op | 8 B/op | 1 allocs/op
33+
BenchmarkUnmarshal/CBOR_negative_int_to_Go_interface_{}-2 | 138 ns/op | 24 B/op | 2 allocs/op
34+
BenchmarkUnmarshal/CBOR_negative_int_to_Go_int64-2 | 104 ns/op | 8 B/op | 1 allocs/op
35+
BenchmarkUnmarshal/CBOR_float_to_Go_interface_{}-2 | 139 ns/op | 24 B/op | 2 allocs/op
3636
BenchmarkUnmarshal/CBOR_float_to_Go_float64-2 | 102 ns/op | 8 B/op | 1 allocs/op
37-
BenchmarkUnmarshal/CBOR_bytes_to_Go_interface_{}-2 | 186 ns/op | 80 B/op | 3 allocs/op
38-
BenchmarkUnmarshal/CBOR_bytes_to_Go_[]uint8-2 | 189 ns/op | 64 B/op | 2 allocs/op
39-
BenchmarkUnmarshal/CBOR_text_to_Go_interface_{}-2 | 220 ns/op | 80 B/op | 3 allocs/op
40-
BenchmarkUnmarshal/CBOR_text_to_Go_string-2 | 182 ns/op | 64 B/op | 2 allocs/op
41-
BenchmarkUnmarshal/CBOR_array_to_Go_interface_{}-2 |1190 ns/op | 672 B/op | 29 allocs/op
42-
BenchmarkUnmarshal/CBOR_array_to_Go_[]int-2 | 1177 ns/op | 272 B/op | 3 allocs/op
43-
BenchmarkUnmarshal/CBOR_map_to_Go_interface_{}-2 | 3021 ns/op | 1421 B/op | 30 allocs/op
44-
BenchmarkUnmarshal/CBOR_map_to_Go_map[string]interface_{}-2 | 3961 ns/op | 964 B/op | 19 allocs/op
45-
BenchmarkUnmarshal/CBOR_map_to_Go_map[string]string-2 | 2732 ns/op | 741 B/op | 5 allocs/op
37+
BenchmarkUnmarshal/CBOR_bytes_to_Go_interface_{}-2 | 181 ns/op | 80 B/op | 3 allocs/op
38+
BenchmarkUnmarshal/CBOR_bytes_to_Go_[]uint8-2 | 190 ns/op | 64 B/op | 2 allocs/op
39+
BenchmarkUnmarshal/CBOR_text_to_Go_interface_{}-2 | 211 ns/op | 80 B/op | 3 allocs/op
40+
BenchmarkUnmarshal/CBOR_text_to_Go_string-2 | 181 ns/op | 64 B/op | 2 allocs/op
41+
BenchmarkUnmarshal/CBOR_array_to_Go_interface_{}-2 |1147 ns/op | 672 B/op | 29 allocs/op
42+
BenchmarkUnmarshal/CBOR_array_to_Go_[]int-2 | 1138 ns/op | 272 B/op | 3 allocs/op
43+
BenchmarkUnmarshal/CBOR_map_to_Go_interface_{}-2 | 2962 ns/op | 1420 B/op | 30 allocs/op
44+
BenchmarkUnmarshal/CBOR_map_to_Go_map[string]interface_{}-2 | 3849 ns/op | 964 B/op | 19 allocs/op
45+
BenchmarkUnmarshal/CBOR_map_to_Go_map[string]string-2 | 2654 ns/op | 740 B/op | 5 allocs/op
4646

4747
Encoding Benchmark | Time | Memory | Allocs
4848
--- | ---: | ---: | ---:
49-
BenchmarkMarshal/Go_bool_to_CBOR_bool-2 | 89.2 ns/op | 1 B/op | 1 allocs/op
50-
BenchmarkMarshal/Go_uint64_to_CBOR_positive_int-2 | 99.2 ns/op | 16 B/op | 1 allocs/op
51-
BenchmarkMarshal/Go_int64_to_CBOR_negative_int-2 | 92.6 ns/op | 3 B/op | 1 allocs/op
52-
BenchmarkMarshal/Go_float64_to_CBOR_float-2 | 99.5 ns/op | 16 B/op | 1 allocs/op
53-
BenchmarkMarshal/Go_[]uint8_to_CBOR_bytes-2 | 124 ns/op | 32 B/op | 1 allocs/op
54-
BenchmarkMarshal/Go_string_to_CBOR_text-2 | 121 ns/op | 48 B/op | 1 allocs/op
55-
BenchmarkMarshal/Go_[]int_to_CBOR_array-2 | 490 ns/op | 32 B/op | 1 allocs/op
56-
BenchmarkMarshal/Go_map[string]string_to_CBOR_map-2 | 2189 ns/op | 576 B/op | 28 allocs/op
49+
BenchmarkMarshal/Go_bool_to_CBOR_bool-2 | 83.3 ns/op | 1 B/op | 1 allocs/op
50+
BenchmarkMarshal/Go_uint64_to_CBOR_positive_int-2 | 93.4 ns/op | 16 B/op | 1 allocs/op
51+
BenchmarkMarshal/Go_int64_to_CBOR_negative_int-2 | 88.3 ns/op | 3 B/op | 1 allocs/op
52+
BenchmarkMarshal/Go_float64_to_CBOR_float-2 | 95.4 ns/op | 16 B/op | 1 allocs/op
53+
BenchmarkMarshal/Go_[]uint8_to_CBOR_bytes-2 | 119 ns/op | 32 B/op | 1 allocs/op
54+
BenchmarkMarshal/Go_string_to_CBOR_text-2 | 114 ns/op | 48 B/op | 1 allocs/op
55+
BenchmarkMarshal/Go_[]int_to_CBOR_array-2 | 470 ns/op | 32 B/op | 1 allocs/op
56+
BenchmarkMarshal/Go_map[string]string_to_CBOR_map-2 | 2064 ns/op | 576 B/op | 28 allocs/op
5757

5858
## Go structs
5959

@@ -74,13 +74,13 @@ Benchmarks use struct and map[string]interface{} representing the following valu
7474

7575
Decoding Benchmark | Time | Memory | Allocs
7676
--- | ---: | ---: | ---:
77-
BenchmarkUnmarshal/CBOR_map_to_Go_map[string]interface{}-2 | 6693 ns/op | 2621 B/op | 73 allocs/op
78-
BenchmarkUnmarshal/CBOR_map_to_Go_struct-2 | 4694 ns/op | 1172 B/op | 10 allocs/op
77+
BenchmarkUnmarshal/CBOR_map_to_Go_map[string]interface{}-2 | 6380 ns/op | 2620 B/op | 73 allocs/op
78+
BenchmarkUnmarshal/CBOR_map_to_Go_struct-2 | 4603 ns/op | 1173 B/op | 10 allocs/op
7979

8080
Encoding Benchmark | Time | Memory | Allocs
8181
--- | ---: | ---: | ---:
82-
BenchmarkMarshal/Go_map[string]interface{}_to_CBOR_map-2 | 4638 ns/op | 1072 B/op | 45 allocs/op
83-
BenchmarkMarshal/Go_struct_to_CBOR_map-2 | 2994 ns/op | 720 B/op | 28 allocs/op
82+
BenchmarkMarshal/Go_map[string]interface{}_to_CBOR_map-2 | 4394 ns/op | 1072 B/op | 45 allocs/op
83+
BenchmarkMarshal/Go_struct_to_CBOR_map-2 | 2770 ns/op | 720 B/op | 28 allocs/op
8484

8585
## Go structs with "keyasint" struct tag
8686

@@ -116,13 +116,13 @@ type T struct {
116116

117117
Decoding Benchmark | Time | Memory | Allocs
118118
--- | ---: | ---: | ---:
119-
BenchmarkUnmarshal/CBOR_map_to_Go_map[int]interface{}-2| 6525 ns/op | 2516 B/op | 70 allocs/op
120-
BenchmarkUnmarshal/CBOR_map_to_Go_struct_keyasint-2 | 4881 ns/op | 1236 B/op | 18 allocs/op
119+
BenchmarkUnmarshal/CBOR_map_to_Go_map[int]interface{}-2| 6222 ns/op | 2516 B/op | 70 allocs/op
120+
BenchmarkUnmarshal/CBOR_map_to_Go_struct_keyasint-2 | 4782 ns/op | 1237 B/op | 18 allocs/op
121121

122122
Encoding Benchmark | Time | Memory | Allocs
123123
--- | ---: | ---: | ---:
124-
BenchmarkMarshal/Go_map[int]interface{}_to_CBOR_map-2 | 4469 ns/op | 992 B/op | 45 allocs/op
125-
BenchmarkMarshal/Go_struct_keyasint_to_CBOR_map-2 | 3011 ns/op | 704 B/op | 28 allocs/op
124+
BenchmarkMarshal/Go_map[int]interface{}_to_CBOR_map-2 | 4307 ns/op | 992 B/op | 45 allocs/op
125+
BenchmarkMarshal/Go_struct_keyasint_to_CBOR_map-2 | 2777 ns/op | 704 B/op | 28 allocs/op
126126

127127
## Go structs with "toarray" struct tag
128128

@@ -159,13 +159,13 @@ type T struct {
159159

160160
Decoding Benchmark | Time | Memory | Allocs
161161
--- | ---: | ---: | ---:
162-
BenchmarkUnmarshal/CBOR_array_to_Go_[]interface{}-2 | 5186 ns/op | 2404 B/op | 67 allocs/op
163-
BenchmarkUnmarshal/CBOR_array_to_Go_struct_toarray-2 | 4443 ns/op | 1165 B/op | 9 allocs/op
162+
BenchmarkUnmarshal/CBOR_array_to_Go_[]interface{}-2 | 5010 ns/op | 2404 B/op | 67 allocs/op
163+
BenchmarkUnmarshal/CBOR_array_to_Go_struct_toarray-2 | 4301 ns/op | 1164 B/op | 9 allocs/op
164164

165165
Encoding Benchmark | Time | Memory | Allocs
166166
--- | ---: | ---: | ---:
167-
BenchmarkMarshal/Go_[]interface{}_to_CBOR_map-2 | 3352 ns/op | 704 B/op | 28 allocs/op
168-
BenchmarkMarshal/Go_struct_toarray_to_CBOR_array-2 | 3024 ns/op | 704 B/op | 28 allocs/op
167+
BenchmarkMarshal/Go_[]interface{}_to_CBOR_map-2 | 3128 ns/op | 704 B/op | 28 allocs/op
168+
BenchmarkMarshal/Go_struct_toarray_to_CBOR_array-2 | 2741 ns/op | 704 B/op | 28 allocs/op
169169

170170
## COSE data
171171

@@ -205,15 +205,15 @@ Benchmarks use COSE data from https://tools.ietf.org/html/rfc8392#appendix-A sec
205205

206206
Decoding Benchmark | Time | Memory | Allocs
207207
--- | ---: | ---: | ---:
208-
BenchmarkUnmarshalCOSE/128-Bit_Symmetric_Key-2 | 763 ns/op | 280 B/op | 9 allocs/op
209-
BenchmarkUnmarshalCOSE/256-Bit_Symmetric_Key-2 | 765 ns/op | 296 B/op | 9 allocs/op
210-
BenchmarkUnmarshalCOSE/ECDSA_P256_256-Bit_Key-2 | 1388 ns/op | 448 B/op | 18 allocs/op
208+
BenchmarkUnmarshalCOSE/128-Bit_Symmetric_Key-2 | 761 ns/op | 280 B/op | 9 allocs/op
209+
BenchmarkUnmarshalCOSE/256-Bit_Symmetric_Key-2 | 768 ns/op | 296 B/op | 9 allocs/op
210+
BenchmarkUnmarshalCOSE/ECDSA_P256_256-Bit_Key-2 | 1385 ns/op | 448 B/op | 18 allocs/op
211211

212212
Encoding Benchmark | Time | Memory | Allocs
213213
--- | ---: | ---: | ---:
214-
BenchmarkMarshalCOSE/128-Bit_Symmetric_Key-2 | 537 ns/op | 224 B/op | 2 allocs/op
215-
BenchmarkMarshalCOSE/256-Bit_Symmetric_Key-2 | 538 ns/op | 240 B/op | 2 allocs/op
216-
BenchmarkMarshalCOSE/ECDSA_P256_256-Bit_Key-2 | 683 ns/op | 320 B/op | 2 allocs/op
214+
BenchmarkMarshalCOSE/128-Bit_Symmetric_Key-2 | 510 ns/op | 224 B/op | 2 allocs/op
215+
BenchmarkMarshalCOSE/256-Bit_Symmetric_Key-2 | 512 ns/op | 240 B/op | 2 allocs/op
216+
BenchmarkMarshalCOSE/ECDSA_P256_256-Bit_Key-2 | 674 ns/op | 320 B/op | 2 allocs/op
217217

218218
## CWT claims data
219219

@@ -233,11 +233,11 @@ Benchmarks use CTW claims data from https://tools.ietf.org/html/rfc8392#appendix
233233

234234
Decoding Benchmark | Time | Memory | Allocs
235235
--- | ---: | ---: | ---:
236-
BenchmarkUnmarshalCWTClaims-2 | 1027 ns/op | 240 B/op | 13 allocs/op
236+
BenchmarkUnmarshalCWTClaims-2 | 1029 ns/op | 240 B/op | 13 allocs/op
237237

238238
Encoding Benchmark | Time | Memory | Allocs
239239
--- | ---: | ---: | ---:
240-
BenchmarkMarshalCWTClaims-2 | 474 ns/op | 176 B/op | 2 allocs/op
240+
BenchmarkMarshalCWTClaims-2 | 445 ns/op | 176 B/op | 2 allocs/op
241241

242242
## SenML data
243243

@@ -257,8 +257,8 @@ Benchmarks use SenML data from https://tools.ietf.org/html/rfc8428#section-6
257257

258258
Decoding Benchmark | Time | Memory | Allocs
259259
--- | ---: | ---: | ---:
260-
BenchmarkUnmarshalSenML-2 | 4361 ns/op | 1720 B/op | 40 allocs/op
260+
BenchmarkUnmarshalSenML-2 | 4279 ns/op | 1720 B/op | 40 allocs/op
261261

262262
Encoding Benchmark | Time | Memory | Allocs
263263
--- | ---: | ---: | ---:
264-
BenchmarkMarshalSenML-2 | 2975 ns/op | 272 B/op | 2 allocs/op
264+
BenchmarkMarshalSenML-2 | 2906 ns/op | 272 B/op | 2 allocs/op

0 commit comments

Comments
 (0)