Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rest server #839

Closed
wants to merge 111 commits into from
Closed

Conversation

cschuchardt88
Copy link
Member

@cschuchardt88 cschuchardt88 commented Nov 8, 2023

RestServer

In this section you will learn about RestServer plugin and how it works.

Dependencies

  • Microsoft.AspNetCore.JsonPatch.dll Required
  • Microsoft.AspNetCore.Mvc.NewtonsoftJson.dll Required
  • Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer.dll Required
  • Microsoft.AspNetCore.Mvc.Versioning.dll Required
  • Microsoft.OpenApi.dll Required
  • Newtonsoft.Json.Bson.dll Required
  • Newtonsoft.Json.dll Required
  • System.ServiceProcess.ServiceController.dll linux maybe
  • Microsoft.AspNetCore.Mvc.Versioning.dll Required
  • Microsoft.AspNetCore.Mvc.Versioning.dll Required
  • Microsoft.AspNetCore.Mvc.Versioning.dll Required
  • Microsoft.OpenApi.dll Swagger (optional)
  • Swashbuckle.AspNetCore.Swagger.dll Swagger (optional)
  • Swashbuckle.AspNetCore.SwaggerGen.dll Swagger (optional)
  • Swashbuckle.AspNetCore.SwaggerUI.dll Swagger (optional)
  • Swashbuckle.AspNetCore.Newtonsoft.dll Swagger (optional)
  • RestServer.xml Swagger UI (optional)

In Docker

RestServer
|   |   |-- Microsoft.AspNetCore.JsonPatch.dll
|   |   |-- Microsoft.AspNetCore.Mvc.NewtonsoftJson.dll
|   |   |-- Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer.dll
|   |   |-- Microsoft.AspNetCore.Mvc.Versioning.dll
|   |   |-- Microsoft.OpenApi.dll
|   |   |-- Neo.ConsoleService.dll
|   |   |-- Newtonsoft.Json.Bson.dll
|   |   |-- Newtonsoft.Json.dll
|   |   |-- RestServer.deps.json
|   |   |-- RestServer.dll
|   |   |-- RestServer.pdb
|   |   |-- RestServer.xml
|   |   |-- Swashbuckle.AspNetCore.Newtonsoft.dll
|   |   |-- Swashbuckle.AspNetCore.Swagger.dll
|   |   |-- Swashbuckle.AspNetCore.SwaggerGen.dll
|   |   |-- Swashbuckle.AspNetCore.SwaggerUI.dll
|   |   |-- System.ServiceProcess.ServiceController.dll
|   |   |-- config.json
|   |   `-- runtimes
|   |       `-- win
|   |           `-- lib
|   |               `-- net6.0
|   |                   `-- System.ServiceProcess.ServiceController.dll

These files go in the same directory as the RestServer.dll. In neo-cli
plugins/RestServer/ folder.

Response Headers

Name Value(s) Description
server neo-cli/3.6.0 RestServer/3.6.0 neo-cli and RestServer version.

JSON Serializer

RestServer uses custom Newtonsoft Json Converters to serialize controller action
responses and route parameters.

One Way Binding - Write only.

  • Neo.SmartContract.ContractState
  • Neo.SmartContract.NefFile
  • Neo.SmartContract.MethodToken
  • Neo.SmartContract.Native.TrimmedBlock
  • Neo.SmartContract.Manifest.ContractAbi
  • Neo.SmartContract.Manifest.ContractGroup
  • Neo.SmartContract.Manifest.ContractManifest
  • Neo.SmartContract.Manifest.ContractPermission
  • Neo.SmartContract.Manifest.ContractPermissionDescriptor
  • Neo.Network.P2P.Payloads.Block
  • Neo.Network.P2P.Payloads.Header
  • Neo.Network.P2P.Payloads.Signer
  • Neo.Network.P2P.Payloads.TransactionAttribute
  • Neo.Network.P2P.Payloads.Transaction
  • Neo.Network.P2P.Payloads.Witness

Two Way Binding - Read & Write

  • System.Guid
  • System.ReadOnlyMemory<T>
  • Neo.BigDecimal
  • Neo.UInt160
  • Neo.UInt256
  • Neo.Cryptography.ECC.ECPoint
  • Neo.VM.Types.Array
  • Neo.VM.Types.Boolean
  • Neo.VM.Types.Buffer
  • Neo.VM.Types.ByteString
  • Neo.VM.Types.Integer
  • Neo.VM.Types.InteropInterface
  • Neo.VM.Types.Null
  • Neo.VM.Types.Map
  • Neo.VM.Types.Pointer
  • Neo.VM.Types.StackItem
  • Neo.VM.Types.Struct

Remote Endpoints

Parametes {hash} can be any Neo N3 address or scripthash; {address} can be any Neo N3 address only; {number} and {index} can be any uint32.

Parameter Examples

  • {hash} - 0xef4073a0f2b305a38ec4050e4d3d28bc40ea63f5 or NiHURyS83nX2mpxtA7xq84cGxVbHojj5Wc
  • {address} - NiHURyS83nX2mpxtA7xq84cGxVbHojj5Wc
  • {number} - 1
  • {index} - 2500000

Paths

  • Utils
    • [GET] /api/v1/utils/{hash}/address
    • [GET] /api/v1/utils/{address}/scripthash
    • [GET] /api/v1/utils/{hash}/{address}/validate
  • Node
    • [GET] /api/v1/node
    • [GET] /api/v1/node/peers
    • [GET] /api/v1/node/plugins
    • [GET] /api/v1/node/settings
  • Ledger
    • [GET] /api/v1/ledger/neo/accounts
    • [GET] /api/v1/ledger/gas/accounts
    • [GET] /api/v1/ledger/blocks?page={number}&size={number}
    • [GET] /api/v1/ledger/blocks/height
    • [GET] /api/v1/ledger/blocks/{index}
    • [GET] /api/v1/ledger/blocks/{index}/header
    • [GET] /api/v1/ledger/blocks/{index}/witness
    • [GET] /api/v1/ledger/blocks/{index}/transactions?page={number}&size={number}
    • [GET] /api/v1/ledger/transactions/{hash}
    • [GET] /api/v1/ledger/transactions/{hash}/witnesses
    • [GET] /api/v1/ledger/transactions/{hash}/signers
    • [GET] /api/v1/ledger/transactions/{hash}/attributes
    • [GET] /api/v1/ledger/memorypool?page={number}&size={number}
    • [GET] /api/v1/ledger/memorypool/verified?page={number}&size={number}
    • [GET] /api/v1/ledger/memorypool/unverified?page={number}&size={number}
    • [GET] /api/v1/ledger/memorypool/count
  • Tokens
    • [GET] /api/v1/tokens/balanceof/{address}
    • NFTs
      • [GET] /api/v1/tokens/nep-11?page={number}&size={number}
      • [GET] /api/v1/tokens/nep-11/count
      • [GET] /api/v1/tokens/nep-11/{hash}/balanceof/{address}
    • NEP-17
      • [GET] /api/v1/tokens/nep-17?page={number}&size={number}
      • [GET] /api/v1/tokens/nep-17/count
      • [GET] /api/v1/tokens/nep-17/{hash}/balanceof/{address}
  • Contracts
    • [GET] /api/v1/contracts?page={number}&size={number}
    • [GET] /api/v1/contracts/count
    • [GET] /api/v1/contracts/{hash}
    • [GET] /api/v1/contracts/{hash}/abi
    • [GET] /api/v1/contracts/{hash}/manifest
    • [GET] /api/v1/contracts/{hash}/nef
    • [GET] /api/v1/contracts/{hash}/storage
  • Wallet
    • [POST] /api/v1/wallet/open
    • [POST] /api/v1/wallet/create
    • [POST] /api/v1/wallet/{session}/address/create
    • [GET] /api/v1/wallet/{session}/address/list
    • [GET] /api/v1/wallet/{session}/asset/list
    • [GET] /api/v1/wallet/{session}/balance/list
    • [POST] /api/v1/wallet/{session}/changepassword
    • [GET] /api/v1/wallet/{session}/close
    • [GET] /api/v1/wallet/{session}/delete/{address}
    • [GET] /api/v1/wallet/{session}/export/{address}
    • [GET] /api/v1/wallet/{session}/export
    • [GET] /api/v1/wallet/{session}/gas/unclaimed
    • [GET] /api/v1/wallet/{session}/key/list
    • [POST] /api/v1/wallet/{session}/import
    • [POST] /api/v1/wallet/{session}/import/multisigaddress
    • [POST] /api/v1/wallet/{session}/transfer

Christopher R. Schuchardt and others added 30 commits August 16, 2023 17:55
Added HTTP Basic Auth.
Changed most classes to internal that didnt need to be public.
Added Rest Server Middleware.
Added the ability to block urls paths (for module rest server plugins).
Added the ability to enable CORS and whitelist of urls.
Added HTTP user and pass configurable.
Added hot load of Controllers from plugin assemblies
Changed Controllers to be "ApiController" class
Added MaxTransactionFee to config
Added Anything for Kestrel to be configurable
Added Nep11 token controller.
Changed namespaces and file names around.
Added peers route for node controller
Changed RemoteNodeModel and TokenBalanceModel namespaces
Added Neo/Gas controller
Added send raw transactions relay
Fixed formating
Added blacklist/disable controllers
Removed DisableRoute list from config (wasn't being used)
… format

Added InvokeScript to helper class
Added ConvertToScriptHash for address to scripthash
Changed all controller parameters for json converters
Added custom error handling
Fixed wallet sessions
Added wallet session manager
removed development mode (not needed)
Added session timeout to config file
Added max allow sessions to config file
Added swagger configurable in config.json
Changed controllers to reflect swagger
Changed wallet controller to reset expiring for wallets per request
Added new json converter for block, transaction and contract
removed the model associated with blocks, transactions, and contracts
changed json settings for new json converters that were added
Added Console Commands for wallet sessions
Fixed UInt160, UInt256 json converters value not being set right in certain situations.
Fixed wallet transfers for null signers
Fixed Wallet transfers with invalid data or output messages
Changed wallet route paths around to be suit the means
Fixed ECPoint json converter for reading
Fixed UInt160 json converter
Changed error out for invalid serialization of json on route parameters
Fixed with dotnet format
Changed restricted wallet path to Environment.CurrentDirectory
Changed NEP-11 Models to IReadOnlyDictionary and IEnumerable.
Fixed Guid json converter with values.
shargon
shargon previously approved these changes Jan 9, 2024
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(AccountDetails[]))]
public IActionResult ShowGasAccounts()
{
var accounts = NativeContract.GAS.ListAccounts(_neosystem.StoreView, _neosystem.Settings);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider mainnet and some public server providing this API. Won't it be easy to DOS it by doing these accounts requests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be something to test. I am using the same logic as core; that uses the storage API. But i do see your point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't have these methods unrestricted.

/// <returns>An array of the Transaction object.</returns>
/// <response code="200">Successful</response>
/// <response code="400">An error occurred. See Response for details.</response>
[HttpGet("memorypool", Name = "GetMemoryPoolTransactions")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why memorypool is a part of ledger? I'd say it's a part of node, there is nothing here that is a part of the ledger yet.

Also, ledger can be associated with the respective native contract.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Good point. I was thinking more that, node would be settings, peers, connections. However for ledger that would be, more like transactions and blocks; the actual data related the ledger as in; the stuff that keeps track of the accounts and data.

{
if (skip < 0 || take < 0 || take > RestServerSettings.Current.MaxPageSize)
throw new InvalidParameterRangeException();
return Ok(_neosystem.MemPool.Skip((skip - 1) * take).Take(take));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory pool can change from request to request, I'm not sure paging really fits here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes i thought about that too. But if we were as busy as etherium and blocks can have 10,000 transactions in it. I figure paging would be good.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vote up for paging removal, no practical usage for this option.

{
internal static class RestErrorCodes
{
//=========================Rest Codes=========================
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API is different, but neo-project/proposals#156 is likely still applicable, just two codes are not really usable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thx. I was wondering about the codes. Is there a list somewhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right in the proposal. We should have some unification between REST and RPC codes.

@cschuchardt88
Copy link
Member Author

@shargon @Jim8y
This already been tested and reviewed by @superboyiii. However @vncoelho caused a seen, that ended up being false.

@shargon
Copy link
Member

shargon commented Feb 13, 2024

I will review it likely the next week (working of bug fixes now).

using Newtonsoft.Json.Linq;
using System.Numerics;

namespace Neo.Plugins.RestServer.Newtonsoft.Json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cal we move all of these converters to Neo.Json project?

  • More organized.
  • Reusable.
  • Two PR, easy to review.

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am using NewtonSoft.Json library. However would be nice to have in a library.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orderd a new machine, will arrive two days later. My current machine is too weak.

BTW, how hard is it for you to build a Restful Mock testing system?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well you have to mock the asp.net core to access controllers. Which can be a pain. After that it's pretty easy.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move it to Neo.Json, easier to approve by chunks

Copy link
Member Author

@cschuchardt88 cschuchardt88 Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shargon

Let's move it to Neo.Json, easier to approve by chunks

Won't be able to do because of circular dependencies.

@Jim8y Jim8y added Need Update This pr needs to be updated. and removed waiting for review labels Feb 15, 2024
@cschuchardt88
Copy link
Member Author

Ready for merge #839 (comment)

@@ -0,0 +1,108 @@
## RestServer Plugin
In this section of you will learn how to make a `neo-cli` plugin that integrates with `RestServer`
plugin. Lets take a look at [Example Plugin](/examples/RestServerPlugin).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still have an example? Because currently an example plugin link navigates to nowhere.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a start https://github.com/cschuchardt88/neo-modules/blob/RestServer/docs/RestServer/Addons.md however looks like some stuff is missing.

- **Type** - _Must have a base class of [error](#error-class)._

## Error Class
Needs to be the same as `RestServer` of else there will be some inconsistencies
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/of else/otherwise

Comment on lines +41 to +50
Properties `Code`, `Name` and `Message` values can be whatever you desire.

**Model**
```csharp
public class ErrorModel
{
public int Code { get; set; };
public string Name { get; set; };
public string Message { get; set; };
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The case is they can't. We have neo-project/proposals#156 for RPC errors, and if we're merging this PR, then I think we need some unified set of error codes for REST as far.

```
Notice that the _above_ example also returns with HTTP status code of `204 No Content`.
This action `route` also extends the `contracts` API. Adding method `sayHello`. Routes
can be what you like as well. But if you want to extend on any existing controller you
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/extend on/extend


namespace Neo.Plugins.RestServer.Binder
{
internal class UInt160Binder : IModelBinder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about the same provider for UInt256?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no alternative formatting for UInt256. Unlike UInt160 can be scripthash or 160HASH format.

{
if (skip < 0 || take < 0 || take > RestServerSettings.Current.MaxPageSize)
throw new InvalidParameterRangeException();
return Ok(_neosystem.MemPool.Skip((skip - 1) * take).Take(take));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vote up for paging removal, no practical usage for this option.

Comment on lines +81 to +85
[HttpGet("settings", Name = "GetNodeProtocolSettings")]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(ProtocolSettingsModel))]
public IActionResult GetSettings() =>
Ok(_neosystem.Settings.ToModel());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be extended to be similar to getversion response.

var tokenList = NativeContract.ContractManagement.ListContracts(_neosystem.StoreView);
var vaildContracts = tokenList
.Where(ContractHelper.IsNep17Supported)
.OrderBy(o => o.Manifest.Name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, sorting by name is a bad idea.

{
internal static class RestErrorCodes
{
//=========================Rest Codes=========================
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right in the proposal. We should have some unification between REST and RPC codes.

src/RestServer/config.json Show resolved Hide resolved
@shargon
Copy link
Member

shargon commented May 21, 2024

Repository moved to neo, please re-open there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Need Update This pr needs to be updated.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants