Merge in peer collection fix from 1.4.0.7 (#1148)

* Removed unused external lib folder

* revert back external lib

* upgrade to .NET 6

* Fixed Issue of POA IntegrationTests and Patricia Test

* Shorten long test name

* Migrate changes from PR #957

* Set official release version

* upgrade to .net 6

* Fix uni tests

* Remove specific runtime version

* Bump version to 1.3.3.0

* Bump all versions

* Bump collateral and federated peg versions

* Bump NBitcoin version

* Fix duplicate publish file output issue

* Fix duplicate publish file output issue (#1027)

* GetEntireState and potential block skipping fix (#1037)

* Bump all versions to 1.3.3.0

* Create initial classes

* Add SignalR event

* WIP

* Done

* Payload Fixes

* Update MultiSigStateMonitorBehavior.cs

* Update MultiSigStateMonitorBehavior.cs

* Merge pull request #1040 from zeptin/transactionsigners-20220815

[API] Add gettransactionsigners endpoint

* Update FederationGatewayController.cs

* Add switch disable monitoring

* Update MultiSigStateMonitorBehavior.cs

* Add net6 version suffix

* Remove duplicate package reference

* Merge pull request #1045 from quantumagi/fixhistrecvquery

Fix history receive query

* Bump all versions to 1.3.3.0

* Fix decompiler Address references

* Temporary pre-merge

* Merge pull request #1048 from quantumagi/singlenodestartdate

Delegate node StartTime to NodeStats only

* Create WatchedSrc721Contracts setting instance

* Add already include check.

* Add index to filter

* Update launch script to enable watcherc721 contract addreses

* Add logging and timeout

* Bump build version

* Update Mint Log for NFTs

* Fix console NFT log

* Small console log change

* Extend receipt searcher to support multiple contracts

* Update NFT indexer to batch receipt searches

* Use extension method for bloom filter testing

* Allow wildcard contract address receipt searches

* Check contract type via local call

* Track last updated block centrally

* Changes per review

* Refactor to use async loop

* Fix cherry picking

* Merge pull request #1055 from zeptin/receiptsearch-20220910

NFT indexing contract address fix

* Update NFTIndexer logs

* Update NFTTransferIndexer.cs

* Remove test line

* Expedited updates from 1.4 (#1058)

* Bump build revision

* Remove UnityApi from StraxD

* Bump all versions to 1.4.0.0

* Update Dockerfiles for .NET 6

* Fix Dockerfile

* Add "getverboseaddressesbalances2" with POST API (#1061) (#1063)

* Add getverboseaddressesbalances2 with POST API

* Remove arguments

* Update tests

* Renaming based on feedback

* Fix comment

Co-authored-by: quantumagi <someguy.fromafrica@gmail.com>

* Use dummy address for local calls by default (#1067)

* Bump build version for testing

* Update Token Contract addresses for ETH Sepolia

* Bump build version for testing

* Fix TSZ1 token contract address

* Merge pull request #973 from SatyaKarki/release/1.4.0.0

Add TransactionAddedToMemoryPool SignalR Event

* Merge pull request #977 from SatyaKarki/release/1.4.0.0

add memorySizePool property for SignalR event

* Merge pull request #989 from SatyaKarki/release/1.4.0.0

Create and Publish MiningStatistics SignalR event

* Merge pull request #1032 from SatyaKarki/release/1.4.0.0

Add Consensus manager SignalR event

* Merge pull request #1038 from SatyaKarki/release/1.4.0.0

Add header height in consensus manager SignalR event

* Merge pull request #1048 from quantumagi/singlenodestartdate

Delegate node StartTime to NodeStats only

* Merge pull request #1047 from SatyaKarki/release/1.4.0.0

Add node start time

* Merge pull request #1060 from SatyaKarki/release/1.4.0.0

Add a new peer connection signalr event and send list of connections in signalr message

* Fix build post merge of Satya's SignalR changes

* Small bit of cleanup post merge

* Bump build version to 1.4.0.4 for testing

* Merge pull request #942 from SatyaKarki/release/1.4.0.0

enable SignalR in CirrusMinerD

* Publish federation wallet changes via signalr

* Bump build version to 1.4.0.5

* Merge pull request #1084 from zeptin/dumpprivkey-20221031

[RPC] Add dumpprivkey RPC command

* Publish Address Indexer Tip

* Bump build version

* WIP

* Fix ref

* Fix status_information_is_returned test

* Bump build version

* Merge legacy PRs into release 1.4.0.7 (#1112)

* Bump all versions to 1.3.4.0

* Bump all versions to 1.4.0.0

* Set version to RC

* Always publish mining statistics

* Add activation height for multisig fee sorting

* Fix GateWayPairStarts pair

* Update release 1.4 activation height

* Bump build version to 1.4.0.1

* Fix multisig fee sort

* Bump build revision to 1.4.0.2

* Set release 1.4 activation height

* Set official release version

* Merge legacy PRs into release 1.4.0.0 (#1111)

* Add dumpprivkey RPC command

* Revert superfluous change

* Add additional sanity test

* Fix test

* [RPC] Add createrawtransaction RPC method (#1100)

* Add createrawtransaction RPC method

* Better handling of Sequence field, and safer defaults

* Fix case where fundrawtransaction is called for the watch-only account

* [RPC] Fix createrawtransaction zero-input handling (#1102)

* Add createrawtransaction RPC method

* Better handling of Sequence field, and safer defaults

* Fix case where fundrawtransaction is called for the watch-only account

* Add protocol version overloads for CreateTransaction

* Check that createrawtransaction RPC client works for SFN/bitcoind

* Modify createrawtransaction RPC to return string response

* Fix RPC middleware to correctly handle non-JSON RPC responses

* Remove accidental duplication

* Fix test

* Revert middleware change and add additional test

Co-authored-by: Kevin Loubser <zeptin@gmail.com>
Co-authored-by: Francois de la Rouviere <fassadlr@gmail.com>

Co-authored-by: Francois de la Rouviere <fassadlr@gmail.com>
Co-authored-by: Kevin Loubser <zeptin@gmail.com>

* Merge legacy PRs into release 1.4.0.7 - part 2 of 2 (#1113)

* Merge legacy PRs into release 1.4.0.7 - part 2 of 2

* Override base CreateTransaction

* Fix VerboseAddressBalancesData (#1115)

* Bump versions to 1.4.0.7 (#1114)

* Bump versions to 1.4.0.7

* Revert unintended change

* Fix CI issues (#1120)

* Improve missing input logging with debug log (#1121)

* Fix CI errors on 1.4.0.7 (#1122)

* Fix FailsFV testcase

* Don't dispose already disposed runners

* Improve error logging in FullNodeFeatureExecutor

* Simplify AsyncLock

* Fix

* Increase timeout

* Disable failing Ubuntu and OSX test case temporarily

* Stabilize test

* Improve test-case

* Implement bounded LRU map for peer known inv hashes

* Remove spurious 'using'

* Make WalletSyncManager more robust

* Add fix from upstream NBitcoin

* Add missing payload attribute

* Don't process deposits prior to already processed deposits (#1124)

* Don't process deposits prior to already processed deposits

* Fix MaturedBlocksSyncManagerTests

* Update src/Stratis.Features.FederatedPeg/TargetChain/MaturedBlocksSyncManager.cs

Co-authored-by: zeptin <zeptin@gmail.com>

* Fix BlocksAreRequestedIfThereIsSomethingToRequestAsync

---------

Co-authored-by: zeptin <zeptin@gmail.com>

* Limit how often the wallet is saved (#1125)

* Limit how often the wallet is saved

* Tweak

* Force wallet save in stop

* Throw error on duplicate withdrawal detected in federation wallet (#1126)

* Limit how often the wallet is saved

* Tweak

* Force wallet save in stop

* Throw error on duplicate withdrawal detected in federation wallet

* Stop application

* Fix

* Just display an error for now

* Can only explicitly remove transient transactions

* Fix conversion in benchmarks

* Add list shuffle helper method

* Add P2WSH handling to wallet database

* Fix signalr config

* Add check for empty list

* Extend the PR 624 changes to reward distribution (#1128)

* Add maturity height to logging (#1127)

* Add maturity height to logging

* Restore removed code section

* Send potentially suspended tx's to CCTS

* Fix

* Logging and code improvements

* Fix null and enforce sync

* Resolve some error scenarios

* Cleanup

* Remove redundant code (#1129)

* Prevent non-ascii characters being omitted from console (#1130)

* Use deposit blocktime for conversions (#1142)

* Use deposit blocktime for conversions

* Fix build errors in tests

* Update Launch Scripts

* Additional endpoint functionality (#1146)

* Check if peer collection modified while trimming (#1143)

---------

Co-authored-by: Satya Karki <spacegrlsatya@gmail.com>
Co-authored-by: Francois de la Rouviere <fassadlr@gmail.com>
Co-authored-by: Nikita Koifman <nik7966@icloud.com>
Co-authored-by: noescape0 <NoEscape0@ya.ru>
Co-authored-by: quantumagi <someguy.fromafrica@gmail.com>
Co-authored-by: Iain McCain <iain.mccain@stratisplatform.com>
Co-authored-by: Nikita Koifman <nikita.koifman@stratisplatform.com>

Author: 7 people

src/Stratis.Bitcoin/P2P/PeerAddressManager.cs

@@ -150,25 +150,25 @@ private PeerAddress AddPeerWithoutCleanup(IPEndPoint endPoint, IPAddress source)
         public void AddPeers(IEnumerable<IPEndPoint> endPoints, IPAddress source)
         {
             // Pre-filter for peers that exist already.
-            List<IPEndPoint> cleanedList = endPoints.Where(proposed => !this.peerInfoByPeerAddress.ContainsKey(proposed)).ToList();
+            IEnumerable<IPEndPoint> cleanedList = endPoints.Where(proposed => !this.peerInfoByPeerAddress.ContainsKey(proposed));
 
-            if (cleanedList.Count == 0)
+            if (cleanedList.IsEmpty())
                 return;
 
-            this.EnsureMaxItems(cleanedList);
+            cleanedList = this.EnsureMaxItems(cleanedList);
 
             foreach (IPEndPoint endPoint in cleanedList)
                 this.AddPeerWithoutCleanup(endPoint, source);
 
             this.EnsureMaxItemsPerSource(source);
         }
 
-        private void EnsureMaxItems(IEnumerable<IPEndPoint> endPoints)
+        private IEnumerable<IPEndPoint> EnsureMaxItems(IEnumerable<IPEndPoint> endPoints)
         {
             int numberToEvict = (this.peerInfoByPeerAddress.Count + endPoints.Count()) - MaxAddressesToStore;
 
             if (numberToEvict <= 0)
-                return;
+                return endPoints;
 
             // Otherwise, we need to figure out whether to evict already-stored addresses or just trim the incoming list to fit.
             // If we never evict already-stored addresses there is a potential risk that we only store dud addresses forever and land up with no valid peers.
@@ -177,7 +177,8 @@ private void EnsureMaxItems(IEnumerable<IPEndPoint> endPoints)
 
             foreach (IPEndPoint endPoint in this.peerInfoByPeerAddress.Keys)
             {
-                PeerAddress address = this.peerInfoByPeerAddress[endPoint];
+                if (!this.peerInfoByPeerAddress.TryGetValue(endPoint, out PeerAddress address))
+                    continue;
 
                 // If the peer is not 'good', or we have never connected to them, then they are an eviction candidate.
                 if (address.Attempted ||
@@ -196,16 +197,22 @@ private void EnsureMaxItems(IEnumerable<IPEndPoint> endPoints)
                 if (numberToEvict == 0)
                     break;
 
-                this.peerInfoByPeerAddress.Remove(endPointToEvict, out _);
+                this.peerInfoByPeerAddress.TryRemove(endPointToEvict, out _);
 
                 numberToEvict--;
             }
 
-            // Note: numberToEvict can never be longer than the endPoints.
+            if (numberToEvict >= endPoints.Count())
+            {
+                return new List<IPEndPoint>();
+            }
+
             if (numberToEvict > 0)
             {
-                endPoints = endPoints.Skip(numberToEvict);
+                return endPoints.Skip(numberToEvict);
             }
+
+            return endPoints;
         }
 
         private void EnsureMaxItemsPerSource(IPAddress source)