Duplicate tproxy MG test in integration tests #1262
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1262 +/- ##
=======================================
Coverage 19.30% 19.30%
=======================================
Files 164 164
Lines 10849 10849
=======================================
Hits 2094 2094
Misses 8755 8755
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Bencher Report
|
| Branch | 2024-11-14-rewrite-tproxy-test |
| Testbed | sv2 |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|
| client_sv2_handle_message_common | 📈 view plot 🚷 view threshold | 44.46 (-0.24%) | 46.74 (95.12%) |
| client_sv2_handle_message_mining | 📈 view plot 🚷 view threshold | 95.77 (+25.55%) | 106.45 (89.96%) |
| client_sv2_mining_message_submit_standard | 📈 view plot 🚷 view threshold | 14.70 (+0.18%) | 14.82 (99.18%) |
| client_sv2_mining_message_submit_standard_serialize | 📈 view plot 🚷 view threshold | 264.39 (-0.47%) | 316.25 (83.60%) |
| client_sv2_mining_message_submit_standard_serialize_deserialize | 📈 view plot 🚷 view threshold | 613.36 (+0.36%) | 715.73 (85.70%) |
| client_sv2_open_channel | 📈 view plot 🚷 view threshold | 166.10 (+8.75%) | 193.23 (85.96%) |
| client_sv2_open_channel_serialize | 📈 view plot 🚷 view threshold | 287.13 (+0.66%) | 314.13 (91.40%) |
| client_sv2_open_channel_serialize_deserialize | 📈 view plot 🚷 view threshold | 388.74 (+2.43%) | 410.11 (94.79%) |
| client_sv2_setup_connection | 📈 view plot 🚷 view threshold | 161.64 (+0.62%) | 171.52 (94.24%) |
| client_sv2_setup_connection_serialize | 📈 view plot 🚷 view threshold | 458.83 (-1.26%) | 540.95 (84.82%) |
| client_sv2_setup_connection_serialize_deserialize | 📈 view plot 🚷 view threshold | 1,003.50 (+2.67%) | 1,060.03 (94.67%) |
|
| Branch | 2024-11-14-rewrite-tproxy-test |
| Testbed | sv1 |
🚨 1 Alert
| Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|
| serialize_deserialize_subscribe | L2 Accesses accesses | 📈 plot 🚨 alert (🔔) 🚷 threshold | 45.00 (+20.65%) | 44.77 (100.50%) |
Click to view all benchmark results
| Benchmark | Estimated Cycles | Benchmark Result estimated cycles (Result Δ%) | Upper Boundary estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Δ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| get_authorize | 📈 view plot 🚷 view threshold | 8,391.00 (-0.88%) | 8,731.43 (96.10%) | 📈 view plot 🚷 view threshold | 3,689.00 (-1.61%) | 3,856.17 (95.66%) | 📈 view plot 🚷 view threshold | 5,156.00 (-1.92%) | 5,424.48 (95.05%) | 📈 view plot 🚷 view threshold | 10.00 (+37.93%) | 13.26 (75.43%) | 📈 view plot 🚷 view threshold | 91.00 (+0.39%) | 95.14 (95.65%) |
| get_submit | 📈 view plot 🚷 view threshold | 95,185.00 (-0.26%) | 95,895.88 (99.26%) | 📈 view plot 🚷 view threshold | 59,346.00 (-0.19%) | 59,681.23 (99.44%) | 📈 view plot 🚷 view threshold | 85,225.00 (-0.21%) | 85,755.59 (99.38%) | 📈 view plot 🚷 view threshold | 46.00 (+2.81%) | 63.30 (72.68%) | 📈 view plot 🚷 view threshold | 278.00 (-0.76%) | 289.21 (96.13%) |
| get_subscribe | 📈 view plot 🚷 view threshold | 7,872.00 (-1.87%) | 8,277.48 (95.10%) | 📈 view plot 🚷 view threshold | 2,765.00 (-2.47%) | 2,937.44 (94.13%) | 📈 view plot 🚷 view threshold | 3,847.00 (-2.89%) | 4,119.74 (93.38%) | 📈 view plot 🚷 view threshold | 14.00 (+17.05%) | 20.62 (67.91%) | 📈 view plot 🚷 view threshold | 113.00 (-1.15%) | 118.50 (95.36%) |
| serialize_authorize | 📈 view plot 🚷 view threshold | 12,112.00 (-1.39%) | 12,567.90 (96.37%) | 📈 view plot 🚷 view threshold | 5,266.00 (-1.03%) | 5,423.66 (97.09%) | 📈 view plot 🚷 view threshold | 7,327.00 (-1.25%) | 7,582.62 (96.63%) | 📈 view plot 🚷 view threshold | 12.00 (+31.06%) | 16.34 (73.45%) | 📈 view plot 🚷 view threshold | 135.00 (-1.91%) | 142.60 (94.67%) |
| serialize_deserialize_authorize | 📈 view plot 🚷 view threshold | 24,636.00 (-0.23%) | 25,225.28 (97.66%) | 📈 view plot 🚷 view threshold | 9,838.00 (-0.54%) | 10,031.81 (98.07%) | 📈 view plot 🚷 view threshold | 13,876.00 (-0.63%) | 14,182.69 (97.84%) | 📈 view plot 🚷 view threshold | 38.00 (+9.62%) | 43.64 (87.07%) | 📈 view plot 🚷 view threshold | 302.00 (+0.15%) | 315.04 (95.86%) |
| serialize_deserialize_handle_authorize | 📈 view plot 🚷 view threshold | 30,156.00 (-0.56%) | 30,750.55 (98.07%) | 📈 view plot 🚷 view threshold | 12,015.00 (-0.55%) | 12,201.76 (98.47%) | 📈 view plot 🚷 view threshold | 16,996.00 (-0.66%) | 17,290.22 (98.30%) | 📈 view plot 🚷 view threshold | 63.00 (+13.34%) | 67.56 (93.25%) | 📈 view plot 🚷 view threshold | 367.00 (-0.72%) | 381.53 (96.19%) |
| serialize_deserialize_handle_submit | 📈 view plot 🚷 view threshold | 126,291.00 (-0.14%) | 126,897.10 (99.52%) | 📈 view plot 🚷 view threshold | 73,200.00 (-0.12%) | 73,507.79 (99.58%) | 📈 view plot 🚷 view threshold | 104,911.00 (-0.15%) | 105,431.79 (99.51%) | 📈 view plot 🚷 view threshold | 111.00 (+3.81%) | 132.56 (83.74%) | 📈 view plot 🚷 view threshold | 595.00 (-0.23%) | 605.13 (98.33%) |
| serialize_deserialize_handle_subscribe | 📈 view plot 🚷 view threshold | 27,748.00 (-0.34%) | 28,598.31 (97.03%) | 📈 view plot 🚷 view threshold | 9,584.00 (-0.68%) | 9,751.31 (98.28%) | 📈 view plot 🚷 view threshold | 13,533.00 (-0.84%) | 13,805.60 (98.03%) | 📈 view plot 🚷 view threshold | 71.00 (+11.78%) | 76.16 (93.23%) | 📈 view plot 🚷 view threshold | 396.00 (-0.12%) | 416.65 (95.04%) |
| serialize_deserialize_submit | 📈 view plot 🚷 view threshold | 115,179.00 (-0.06%) | 115,799.30 (99.46%) | 📈 view plot 🚷 view threshold | 68,060.00 (-0.07%) | 68,391.94 (99.51%) | 📈 view plot 🚷 view threshold | 97,649.00 (-0.09%) | 98,216.95 (99.42%) | 📈 view plot 🚷 view threshold | 69.00 (+10.03%) | 78.22 (88.21%) | 📈 view plot 🚷 view threshold | 491.00 (-0.06%) | 498.37 (98.52%) |
| serialize_deserialize_subscribe | 📈 view plot 🚷 view threshold | 23,188.00 (-0.32%) | 23,985.79 (96.67%) | 📈 view plot 🚷 view threshold | 8,143.00 (-0.75%) | 8,306.70 (98.03%) | 📈 view plot 🚷 view threshold | 11,448.00 (-0.94%) | 11,711.40 (97.75%) | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 45.00 (+20.65%) | 44.77 (100.50%) | 📈 view plot 🚷 view threshold | 329.00 (-0.04%) | 347.65 (94.64%) |
| serialize_submit | 📈 view plot 🚷 view threshold | 99,570.00 (-0.28%) | 100,305.37 (99.27%) | 📈 view plot 🚷 view threshold | 61,408.00 (-0.16%) | 61,714.56 (99.50%) | 📈 view plot 🚷 view threshold | 88,085.00 (-0.18%) | 88,589.46 (99.43%) | 📈 view plot 🚷 view threshold | 50.00 (+7.08%) | 64.23 (77.84%) | 📈 view plot 🚷 view threshold | 321.00 (-1.14%) | 333.74 (96.18%) |
| serialize_subscribe | 📈 view plot 🚷 view threshold | 11,317.00 (-0.77%) | 11,677.15 (96.92%) | 📈 view plot 🚷 view threshold | 4,118.00 (-1.54%) | 4,280.64 (96.20%) | 📈 view plot 🚷 view threshold | 5,707.00 (-1.92%) | 5,972.62 (95.55%) | 📈 view plot 🚷 view threshold | 16.00 (+24.27%) | 20.79 (76.95%) | 📈 view plot 🚷 view threshold | 158.00 (+0.16%) | 163.13 (96.86%) |
|
| Branch | 2024-11-14-rewrite-tproxy-test |
| Testbed | sv2 |
🚨 5 Alerts
| Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|
| client_sv2_handle_message_mining | RAM Accesses accesses | 📈 plot 🚨 alert (🔔) 🚷 threshold | 141.00 (+1.44%) | 140.89 (100.08%) |
| client_sv2_mining_message_submit_standard_serialize | L2 Accesses accesses | 📈 plot 🚨 alert (🔔) 🚷 threshold | 55.00 (+26.69%) | 53.21 (103.37%) |
| client_sv2_mining_message_submit_standard_serialize_deserialize | L2 Accesses accesses | 📈 plot 🚨 alert (🔔) 🚷 threshold | 94.00 (+15.55%) | 90.08 (104.35%) |
| client_sv2_open_channel_serialize | L2 Accesses accesses | 📈 plot 🚨 alert (🔔) 🚷 threshold | 44.00 (+25.52%) | 43.35 (101.50%) |
| client_sv2_open_channel_serialize_deserialize | L2 Accesses accesses | 📈 plot 🚨 alert (🔔) 🚷 threshold | 86.00 (+15.59%) | 85.49 (100.60%) |
Click to view all benchmark results
| Benchmark | Estimated Cycles | Benchmark Result estimated cycles (Result Δ%) | Upper Boundary estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Δ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| client_sv2_handle_message_common | 📈 view plot 🚷 view threshold | 2,084.00 (-0.52%) | 2,208.22 (94.37%) | 📈 view plot 🚷 view threshold | 470.00 (-0.70%) | 488.52 (96.21%) | 📈 view plot 🚷 view threshold | 734.00 (-0.27%) | 757.25 (96.93%) | 📈 view plot 🚷 view threshold | 4.00 (-24.48%) | 13.61 (29.39%) | 📈 view plot 🚷 view threshold | 38.00 (-0.18%) | 41.21 (92.22%) |
| client_sv2_handle_message_mining | 📈 view plot 🚷 view threshold | 8,272.00 (+0.84%) | 8,283.31 (99.86%) | 📈 view plot 🚷 view threshold | 2,137.00 (-0.03%) | 2,143.08 (99.72%) | 📈 view plot 🚷 view threshold | 3,157.00 (-0.12%) | 3,170.57 (99.57%) | 📈 view plot 🚷 view threshold | 36.00 (+1.52%) | 41.78 (86.16%) | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 141.00 (+1.44%) | 140.89 (100.08%) |
| client_sv2_mining_message_submit_standard | 📈 view plot 🚷 view threshold | 6,270.00 (-0.16%) | 6,395.41 (98.04%) | 📈 view plot 🚷 view threshold | 1,747.00 (-0.22%) | 1,766.88 (98.87%) | 📈 view plot 🚷 view threshold | 2,550.00 (-0.15%) | 2,575.60 (99.01%) | 📈 view plot 🚷 view threshold | 16.00 (-4.97%) | 24.17 (66.20%) | 📈 view plot 🚷 view threshold | 104.00 (-0.06%) | 107.26 (96.96%) |
| client_sv2_mining_message_submit_standard_serialize | 📈 view plot 🚷 view threshold | 14,753.00 (+0.18%) | 14,948.39 (98.69%) | 📈 view plot 🚷 view threshold | 4,691.00 (-0.08%) | 4,710.88 (99.58%) | 📈 view plot 🚷 view threshold | 6,743.00 (-0.23%) | 6,782.02 (99.42%) | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 55.00 (+26.69%) | 53.21 (103.37%) | 📈 view plot 🚷 view threshold | 221.00 (-0.21%) | 227.32 (97.22%) |
| client_sv2_mining_message_submit_standard_serialize_deserialize | 📈 view plot 🚷 view threshold | 27,729.00 (+0.70%) | 27,823.02 (99.66%) | 📈 view plot 🚷 view threshold | 10,642.00 (+0.55%) | 10,656.16 (99.87%) | 📈 view plot 🚷 view threshold | 15,499.00 (+0.63%) | 15,518.78 (99.87%) | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 94.00 (+15.55%) | 90.08 (104.35%) | 📈 view plot 🚷 view threshold | 336.00 (+0.28%) | 341.69 (98.34%) |
| client_sv2_open_channel | 📈 view plot 🚷 view threshold | 4,444.00 (+1.15%) | 4,591.60 (96.79%) | 📈 view plot 🚷 view threshold | 1,458.00 (-0.23%) | 1,476.52 (98.75%) | 📈 view plot 🚷 view threshold | 2,154.00 (-0.28%) | 2,184.13 (98.62%) | 📈 view plot 🚷 view threshold | 10.00 (+13.88%) | 16.90 (59.16%) | 📈 view plot 🚷 view threshold | 64.00 (+2.31%) | 67.89 (94.27%) |
| client_sv2_open_channel_serialize | 📈 view plot 🚷 view threshold | 14,045.00 (-0.03%) | 14,389.80 (97.60%) | 📈 view plot 🚷 view threshold | 5,061.00 (-0.07%) | 5,079.52 (99.64%) | 📈 view plot 🚷 view threshold | 7,315.00 (-0.15%) | 7,351.56 (99.50%) | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 44.00 (+25.52%) | 43.35 (101.50%) | 📈 view plot 🚷 view threshold | 186.00 (-0.57%) | 196.92 (94.45%) |
| client_sv2_open_channel_serialize_deserialize | 📈 view plot 🚷 view threshold | 22,825.00 (+0.71%) | 22,912.12 (99.62%) | 📈 view plot 🚷 view threshold | 8,037.00 (+0.18%) | 8,072.87 (99.56%) | 📈 view plot 🚷 view threshold | 11,685.00 (+0.12%) | 11,749.78 (99.45%) | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 86.00 (+15.59%) | 85.49 (100.60%) | 📈 view plot 🚷 view threshold | 306.00 (+0.84%) | 311.08 (98.37%) |
| client_sv2_setup_connection | 📈 view plot 🚷 view threshold | 4,666.00 (-0.40%) | 4,774.79 (97.72%) | 📈 view plot 🚷 view threshold | 1,499.00 (-0.22%) | 1,517.52 (98.78%) | 📈 view plot 🚷 view threshold | 2,276.00 (-0.11%) | 2,299.21 (98.99%) | 📈 view plot 🚷 view threshold | 9.00 (-5.65%) | 16.59 (54.24%) | 📈 view plot 🚷 view threshold | 67.00 (-0.57%) | 69.79 (96.01%) |
| client_sv2_setup_connection_serialize | 📈 view plot 🚷 view threshold | 16,181.00 (+0.09%) | 16,437.42 (98.44%) | 📈 view plot 🚷 view threshold | 5,960.00 (-0.06%) | 5,978.52 (99.69%) | 📈 view plot 🚷 view threshold | 8,651.00 (-0.15%) | 8,691.38 (99.54%) | 📈 view plot 🚷 view threshold | 50.00 (+25.66%) | 52.90 (94.52%) | 📈 view plot 🚷 view threshold | 208.00 (-0.32%) | 215.76 (96.40%) |
| client_sv2_setup_connection_serialize_deserialize | 📈 view plot 🚷 view threshold | 35,703.00 (+0.42%) | 35,768.05 (99.82%) | 📈 view plot 🚷 view threshold | 14,885.00 (+0.23%) | 14,910.00 (99.83%) | 📈 view plot 🚷 view threshold | 21,868.00 (+0.23%) | 21,923.04 (99.75%) | 📈 view plot 🚷 view threshold | 107.00 (+16.09%) | 111.32 (96.12%) | 📈 view plot 🚷 view threshold | 380.00 (+0.19%) | 384.68 (98.78%) |
GitGab19
reviewed
Nov 25, 2024
There was a problem hiding this comment.
I tested it many times, and some of them I got this error, I don't know if you already noticed it:
running 2 tests
thread 'translation_proxy' panicked at tests-integration/tests/common/mod.rs:162:14:
called `Result::unwrap()` on an `Err` value: JsonRpc(Transport(SocketError(Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" })))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test translation_proxy ... FAILED
thread 'success_pool_template_provider_connection' panicked at tests-integration/tests/common/mod.rs:162:14:
called `Result::unwrap()` on an `Err` value: JsonRpc(Transport(SocketError(Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" })))
test success_pool_template_provider_connection ... FAILED
failures:
failures:
success_pool_template_provider_connection
translation_proxy
test result: FAILED. 0 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 37.93s
error: test failed, to rerun pass `-p integration-test --test pool_integration`
Comment on lines
+60
to
+100
| // covers | ||
| // https://github.com/stratum-mining/stratum/blob/main/test/message-generator/test/translation-proxy/translation-proxy.json |
There was a problem hiding this comment.
Can you add the explanation here like you did for the other test?
| @@ -396,9 +396,9 @@ pub async fn start_sv2_translator(upstream: SocketAddr) -> SocketAddr { | |||
| .expect("failed"); | |||
| let listening_address = get_available_address(); | |||
| let listening_port = listening_address.port(); | |||
| let hashrate = measure_hashrate(3) as f32 / 100.0; | |||
| let hashrate = measure_hashrate(3) as f32 / 20.0; | |||
There was a problem hiding this comment.
I did test many times, and some of them it was failing. So I tried to change this in:
Suggested change
| let hashrate = measure_hashrate(3) as f32 / 20.0; | |
| let hashrate = measure_hashrate(1) as f32 / 20.0; |
With that, it never failed again. Don't know if it's random or not btw :)
There was a problem hiding this comment.
Joking, I just got it again:
test translation_proxy ... FAILED
failures:
failures:
translation_proxy
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.83s
There was a problem hiding this comment.
tbh I still don't understand why we are dividing the measured hashrate
that will make tProxy set a very low difficulty and make CPU miner submit way more shares than necessary
jbesraa
force-pushed
the
2024-11-14-rewrite-tproxy-test
branch
from
7ee9303
to
8389498
Compare
plebhash
reviewed
Nov 27, 2024
| // covers | ||
| // https://github.com/stratum-mining/stratum/blob/main/test/message-generator/test/translation-proxy/translation-proxy.json | ||
| #[tokio::test] | ||
| async fn translation_proxy() { |
There was a problem hiding this comment.
I don't think it makes sense to add this to a file called pool_integration.rs
There was a problem hiding this comment.
most importantly, as we already discussed in the past with @Shourya742 , by having multiple tests in the same file, we end up executing all of them in parallel
this can cause problems with CPU miners
plebhash
reviewed
Nov 27, 2024
| ) | ||
| .await; | ||
| let _tp = common::start_template_provider(tp_addr.port()).await; | ||
| let _pool_1 = common::start_pool(Some(pool_addr), Some(tp_addr)).await; |
There was a problem hiding this comment.
why is this called _pool_1?
plebhash
reviewed
Nov 27, 2024
Comment on lines
+117
to
+118
| let jds_addr = common::start_jds(tp_addr).await; | ||
| let jdc_addr = common::start_jdc(pool_jdc_sniffer_addr, tp_addr, jds_addr).await; |
There was a problem hiding this comment.
why do we have JDC + JDS on this test?
this should be JD-less, tProxy should simply connect to Pool
plebhash
reviewed
Nov 27, 2024
Comment on lines
+123
to
+154
| assert_common_message!( | ||
| &pool_jdc_sniffer.next_message_from_downstream(), | ||
| SetupConnection | ||
| ); | ||
| assert_common_message!( | ||
| &pool_jdc_sniffer.next_message_from_upstream(), | ||
| SetupConnectionSuccess | ||
| ); | ||
| assert_mining_message!( | ||
| &pool_jdc_sniffer.next_message_from_downstream(), | ||
| OpenExtendedMiningChannel | ||
| ); | ||
| assert_mining_message!( | ||
| &pool_jdc_sniffer.next_message_from_upstream(), | ||
| OpenExtendedMiningChannelSuccess | ||
| ); | ||
| assert_mining_message!( | ||
| &pool_jdc_sniffer.next_message_from_upstream(), | ||
| NewExtendedMiningJob | ||
| ); | ||
| assert_mining_message!( | ||
| &pool_jdc_sniffer.next_message_from_downstream(), | ||
| SetCustomMiningJob | ||
| ); | ||
| assert_mining_message!( | ||
| &pool_jdc_sniffer.next_message_from_upstream(), | ||
| SetNewPrevHash | ||
| ); | ||
| assert_mining_message!( | ||
| &pool_jdc_sniffer.next_message_from_downstream(), | ||
| SubmitSharesExtended | ||
| ); |
There was a problem hiding this comment.
I don't understand these assertions
what's the point of asserting messages between JDC and Pool?
the original MG test doesn't even have JDC + JDS
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #1208