Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
7fbdedd
OSRB docs review, first pass
progala Jul 21, 2022
78273c9
Merge branch 'lvrfrc87' into docs-review
lvrfrc87 Jul 26, 2022
a3a355d
Merge branch 'develop' into docs-review
lvrfrc87 Jul 26, 2022
95f50b6
Merge branch 'main' into docs-review
lvrfrc87 Jul 29, 2022
ee9abee
readme updates
scetron Jul 29, 2022
f9a6c6e
migrate get_value method into function and rename
lvrfrc87 Jul 29, 2022
2e2363f
Merge branch 'docs-review' of github.com:networktocode-llc/netcompare…
lvrfrc87 Jul 29, 2022
c9600d9
updates
scetron Jul 29, 2022
4966f04
Merge branches 'docs-review' and 'docs-review' of https://github.com/…
scetron Jul 29, 2022
2d67a40
updates
scetron Jul 29, 2022
163424f
edits
scetron Jul 29, 2022
25383ef
update evaluate signature for check method
lvrfrc87 Aug 1, 2022
12c2b9c
Merge branch 'docs-review' of github.com:networktocode-llc/netcompare…
lvrfrc87 Aug 1, 2022
42c349a
Update readme
scetron Aug 9, 2022
d6302d4
Update README.md
scetron Aug 9, 2022
8b7f78c
Update README.md
scetron Aug 9, 2022
83fb2cb
Update README.md
scetron Aug 9, 2022
f7921c5
Update README.md
scetron Aug 9, 2022
f88d414
Update README.md
lvrfrc87 Aug 12, 2022
b17daa8
Update README.md
lvrfrc87 Aug 12, 2022
c78af66
Update README.md
lvrfrc87 Aug 12, 2022
aa96d5e
Update README.md
lvrfrc87 Aug 12, 2022
2a98b99
Update README.md
lvrfrc87 Aug 12, 2022
557b7d9
Update README.md
lvrfrc87 Aug 12, 2022
27023d0
Update README.md
lvrfrc87 Aug 12, 2022
f58f020
Update README.md
lvrfrc87 Aug 12, 2022
1cbfe60
Update README.md
lvrfrc87 Aug 12, 2022
4a91e4b
Update README.md
lvrfrc87 Aug 12, 2022
b065dff
Update README.md
lvrfrc87 Aug 12, 2022
7267441
Update README.md
lvrfrc87 Aug 12, 2022
07d30eb
Update README.md
lvrfrc87 Aug 12, 2022
3f2da46
Update README.md
lvrfrc87 Aug 12, 2022
1b106e7
Update README.md
lvrfrc87 Aug 12, 2022
076aa90
initial add mkdocs
scetron Aug 13, 2022
e0e7be0
updates
scetron Aug 13, 2022
c878465
Update README.md
dwight525 Aug 16, 2022
26fcaa4
Update usage.md
dwight525 Aug 17, 2022
a37e384
Update architecture.md
dwight525 Aug 17, 2022
1e0f844
Update index.md
dwight525 Aug 17, 2022
4289f41
Merge pull request #77 from dwight525/patch-4. Index.md doc edits.
scetron Aug 18, 2022
bc05a15
Merge pull request #76 from dwight525/patch-3
scetron Aug 18, 2022
e47a5df
Merge pull request #75 from dwight525/patch-2
scetron Aug 18, 2022
930f2b6
Merge pull request #74 from dwight525/patch-1
scetron Aug 18, 2022
dc8b0c5
updates
scetron Aug 23, 2022
a0969ab
Update extract_data_from_json
scetron Aug 23, 2022
52a784b
update readme suggestions
scetron Aug 23, 2022
e83833a
update mkdocs
scetron Aug 23, 2022
7e6ea96
remove comment
scetron Aug 23, 2022
8748393
few more edits
scetron Aug 23, 2022
b3a31db
replace output arg with data
lvrfrc87 Aug 24, 2022
6cfb0a4
fix tests
lvrfrc87 Aug 24, 2022
e1e72d9
Update README.md
progala Aug 24, 2022
989945e
doc updates from review
scetron Aug 24, 2022
da13fc6
Merge branch 'docs-review' of https://github.com/networktocode-llc/ne…
scetron Aug 24, 2022
c623920
not-range to not-in-range
scetron Aug 24, 2022
bf546d7
update/check usage examples
scetron Aug 25, 2022
451cb18
add workflow summary to usage
scetron Aug 25, 2022
eb8eaf7
update arch examples
scetron Aug 25, 2022
a341561
Update docs/usage.md
progala Aug 26, 2022
55672ed
Update docs/usage.md
progala Aug 26, 2022
a6bcff3
Update docs/usage.md
progala Aug 26, 2022
01a17e0
Update docs/usage.md
progala Aug 26, 2022
3dc349e
Update docs/usage.md
progala Aug 26, 2022
278bb38
update architecture image
scetron Aug 26, 2022
2dff682
fix links
scetron Aug 26, 2022
d3a569a
Update docs/index.md
progala Aug 26, 2022
a4e9eb3
edits
scetron Aug 26, 2022
97817f3
update netcompare -> jdiff
scetron Aug 30, 2022
f70f683
Merge pull request #68 from networktocode-llc/docs-review
scetron Aug 31, 2022
87d260d
Update the GHA workflow (#78)
ubajze Sep 1, 2022
701228c
Create requirements.txt
ubajze Sep 1, 2022
40fb64b
Release 0.0.1
Sep 1, 2022
f45f899
Merge pull request #80 from networktocode/release-v0.0.1
Sep 1, 2022
916129c
Update mkdocs settings (#81)
ubajze Sep 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
doc updates from review
  • Loading branch information
scetron committed Aug 24, 2022
commit 989945e5ccfab229b9c4ecdcb3f6cc0822bc985d
8 changes: 2 additions & 6 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The library heavily relies on [JMESPath](https://jmespath.org/) for traversing t
## Getting Started


First, you import the CheckType class.
First, you import the `CheckType` class.

```python
from jdiff import CheckType
Expand Down Expand Up @@ -39,18 +39,14 @@ This results in a tuple:
This diff can also show whether any keys were added or deleted.
The second value returned will be the boolean result of the check. In this case, the two data structures were not an exact match.

| Stephen - we may want to remove these next two paragraphs
For instance, the reference state can be collected from the network directly using any method that returns structured data: Ansible, NAPALM, Nornir to name a few. You could also choose to generate the reference state from an SoT, such as [Nautobot](https://github.com/nautobot/nautobot/), and have a true intended state.

`jdiff` is perfectly suited to work with data gathered from network devices via show commands, Ansible playbooks, or in applications such as [Nautobot](https://github.com/nautobot/nautobot/) or [Netbox](https://github.com/netbox-community/netbox). `jdiff` is focused on being the 'plumbing' behind a full network automation validation solution.

## Checking Data Structures

As shown in the example, the check evaluation both performs a diff and tests the objects. All of the concrete `CheckTypes` both perform the diff and their specified check.

More on the **check** part: The check provides a way to test some keys or values in our collected data. The check portion is focused on providing a boolean result of the test. There are a few different ways to check our data.

These are the different checks that can be performed on the data. These both describe the type of check and are also used as the argument to instantiate that type of check with the create method: `CheckType.create("check_type")`.
Below are the names of checks provided by the library. These both describe the type of check performed against the data and are used as an argument to instantiate that type of check with the `create` method, e.g. `CheckType.create("check_type")`.

- `exact_match`: the keys and values must match exactly between the two objects
- `tolerance`: the keys must match and the values can differ according to the 'tolerance' value provided
Expand Down
39 changes: 20 additions & 19 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Comparison and testing of the data structures in 'jdiff' is performed through on

A `jdiff` `CheckType` accepts two Python dictionaries as input: the reference object and the comparison object. The reference object is used as the intended or accepted state and its keys and values are compared against the key-value pairs in the comparison object.

It's worth pointing out that `jdiff` is focused on the comparison of the two objects and the testing of the values, not retrieving the data.
It's worth pointing out that `jdiff` is focused on the comparison of the two objects and the testing of the values, not retrieving the data from the external system.

## Extracting Data For Comparison

Expand Down Expand Up @@ -121,15 +121,15 @@ Examples:
}

```
Create an instance of CheckType object with 'exact_match' as check-type argument.
Create an instance of `CheckType` object with 'exact_match' as check-type argument.

```python
>>> my_check = CheckType.create(check_type="exact_match")
>>> my_check
>>> <jdiff.check_types.ExactMatchType object at 0x10ac00f10>
```

Use the evaluate method to return the result.
Use the `evaluate` method to return the result.

```python
>>> result = my_check.evaluate(reference_data, comparison_data)
Expand Down Expand Up @@ -167,7 +167,7 @@ Let's see a better way to run `exact_match` for this specific case. Because ther
False)
```

In this case, we only want to compare the value of a single key, the `interfaceStatus` key. So we define the JMESPath logic to take the name and the interfaceStatus values from all the interface objects in the data object.
In this case, we only want to compare the value of a single key, the `interfaceStatus` key. So we define the JMESPath expression to take the name and the interfaceStatus values from all the interface objects in the data object.


### Tolerance
Expand All @@ -176,6 +176,8 @@ The `tolerance` test checks for the deviation between the value or count of the

The `tolerance` argument must be a `float > 0`. The calculation is percentage based, and the test of the values may be +/- the `tolerance` percentage.

This check can test whether the difference between two values is within a specified tolerance percentage. It could be useful in cases where values like route metrics or optical power levels fluctuate by a small amount. It might be desirable to treat these values as equal if the deviation is within a given range. You can pass in the result of `len()` to count the number of objects returned within your data.

Let's have a look at a couple of examples:

```python
Expand Down Expand Up @@ -235,29 +237,30 @@ Let's have a look at a couple of examples:
... }
>>> my_check = CheckType.create("tolerance")
```
We will define a JMESPath search for the values we want to test and extract from the reference and comparison objects.
We will define a JMESPath expression for the values we want to test and extract from the reference and comparison objects.
```python
>>> my_jmspath = "global.$peers$.*.*.ipv4.[accepted_prefixes,received_prefixes,sent_prefixes]"
>>> reference_value = extract_data_from_json(reference_data, my_jmspath)
>>> comparison_value = extract_data_from_json(comparison_data, my_jmspath)
```
Define a tolerance to pass into the test.

Define a tolerance percentage.
```python
my_tolerance=10
```
Pass the extracted values and tolerance into the test and `evaluate`.
Pass the extracted values and tolerance as arguments to the `evaluate` method.
```python
>>> actual_results = my_check.evaluate(reference_value, comparison_value, tolerance=my_tolerance)
```
the `tolerance` check returns the values that are not within 10%
The `tolerance` check returns the values that differ by more than 10%.
```python
>>> actual_results
({'10.1.0.0': {'accepted_prefixes': {'new_value': 500, 'old_value': 900},
'received_prefixes': {'new_value': 599, 'old_value': 999},
'sent_prefixes': {'new_value': 511, 'old_value': 1011}}},
False)
```
The last example fails, because none of the values are within 10% of the old_value.
The last example fails, because none of the values are within 10% of the `old_value`.

When we switch one of the values (`received_prefixes`) to a value within 10%, that value will not be shown, but the test fails because the others are still out of tolerance:

Expand All @@ -276,8 +279,6 @@ Let's increase the tolerance:
({}, True)
```

This check can test whether the difference between two values is within a specified tolerance percentage. It could be useful in cases where values like route metrics or optical power levels fluctuate by a small amount. It might be desirable to treat these values as equal if the deviation is within a given range. You can pass in the result of len() to count the number of objects returned within your data.

### Parameter Match

The `parameter_match` check provides a way to test key-value pairs against baseline values.
Expand Down Expand Up @@ -327,7 +328,7 @@ For example, in network data:
>>> comparison_value = extract_data_from_json(comparison_data, my_jmspath)
```

This test requires a mode argument; match in this case matches the keys and values in the "params" to the keys and values in the data.
This test requires a `mode` argument; `match` in this case matches the keys and values in the "params" to the keys and values in the data.
```python
>>> actual_results = my_check.evaluate(
post_value,
Expand All @@ -341,7 +342,7 @@ This test requires a mode argument; match in this case matches the keys and valu
({'Management1': {'interfaceStatus': 'down'}}, False)
```

mode: no-match - return the keys and values from the test object that do not match the keys and values provided in "params"
mode: `no-match` - return the keys and values from the test object that do not match the keys and values provided in "params"
```python
>>> my_parameter_match =
>>> actual_results = my_check.evaluate(
Expand Down Expand Up @@ -390,19 +391,19 @@ Let's run an example where we want to check the `burnedInAddress` key has a stri
... ]
... }
>>> # Python regex for matching MAC address string
>>> regex_args = {"regex": "(?:[0-9a-fA-F]:?){12}", "mode": "match"}
>>> mac_regex = "(?:[0-9a-fA-F]:?){12}"
>>> path = "result[*].interfaces.*.[$name$,burnedInAddress]"
>>> check = CheckType.create(check_type="regex")
>>> value = check.extract_data_from_json(data, path)
>>> value
[{'Management1': {'burnedInAddress': '08:00:27:e6:b2:f8'}}]
>>> result = check.evaluate(value, **regex_args)
>>> result = check.evaluate(value, regex=regex, mode="match")
>>> # The test is passed as the burnedInAddress value matches our regex
>>> result
({}, True)
>>> # What if we want "no-match"?
>>> regex_args = {"regex": "(?:[0-9a-fA-F]:?){12}", "mode": "no-match"}
>>> result = check.evaluate(value, **regex_args)
>>> mac_regex = "(?:[0-9a-fA-F]:?){12}"
>>> result = check.evaluate(value, regex=mac_regex, mode="no-match")
>>> # jdiff returns the failing data, as the regex matches the value
>>> result
({'Management1': {'burnedInAddress': '08:00:27:e6:b2:f8'}}, False)
Expand All @@ -429,8 +430,8 @@ The `operator` check is a collection of more specific checks divided into catego
- `in-range: [20, 70]`: check if value is in range between 20 and 70


4. `not-range`: Check if the value of a specified element is outside of a given numeric range.
- `not-range: [5, 40]`: checks if value is not in range between 5 and 40
4. `not-in-range`: Check if the value of a specified element is outside of a given numeric range.
- `not-in-range: [5, 40]`: checks if value is not in range between 5 and 40

#### `bool` Operators

Expand Down