notes.txt

- create config file
- consider moving all try-catch blocks to the request handler
- add signal handler for SIGINT and SIGTERM
- add option to enable use of GenQuery2
- decide how to handle distributing parallel write requests to streams
    - use in a round-robin fashion (user doesn't have to specify a specific stream)
    - require the user to pass the stream id for each request (e.g. curl ... &stream=2 ...)
- consider if parallel write operations should be handled via a queue (i.e. pipelining)
- decide if all requests should return JSON objects containing info about the success of an operation
    - decide what the structure of the JSON objects should look like
- should GET /data-objects?op=write requests return JSON objects at all?
- should specific iRODS error codes be returned to clients?

example config

{
    "port": 9000,
    "log_level": "info",

    "request_handler_threads": 5,
    "io_threads": 10,
    "irods_connection_pool_size": 15,

    "io_read_buffer_size": 20000,
    "io_write_buffer_size": 20000,

    "irods_server": {
        "host": "test",
        "port": 1247,
        "zone": "tempZone",
        "rodsadmin": {
            "username": "test",
            "password": "test"
        }
    },
}

ways to run the service:
- command line: ./irods_http_api --port 9000 --log-level info --request-handler-threads 5 --io-threads 10 ...
    - is it okay to pass username and password this way?
- point to a config file
    - ./irods_http_api --config-file /path/to/config.json
    - command line arguments override settings in config file



---
HTTP API Github URL example:
- https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28




---
multipart/form-data
- all boundaries start the directly after a CRLF
    - this is kind of Boost.Beast specific
    - the RFC states that the CRLF preceeding an encapsulation boundary must be conceptually viewed as being part of the boundary
- Content-Disposition is a required header within a part
    - there must always be at least ONE header defined explicitly (good)
- use "nc" to stand up a server and see HTTP request inputs
    - while true; do echo '' | nc -l localhost 10000; done

below is a multipart request generated by curl and captured by the http API.

    [2023-05-19 09:50:16.208] [P:1215477] [debug] [T:1215637] data_objects: Request body => [--------------------------f753f237aff8468c
    Content-Disposition: form-data; name="parallel-write-handle"

    78c36f06-97f0-4c86-9967-e6564c28e505
    --------------------------f753f237aff8468c
    Content-Disposition: form-data; name="op"

    write
    --------------------------f753f237aff8468c
    Content-Disposition: form-data; name="count"

    6
    --------------------------f753f237aff8468c
    Content-Disposition: form-data; name="bytes"
    Content-Type: application/octet-stream

    hello
    --------------------------f753f237aff8468c--
    ]
    [2023-05-19 09:50:16.208] [P:1215477] [error] [T:1215637] data_objects: Missing [op] parameter.

and here is an example captured by "nc". notice the content associated with "key3" and "key4". the parser must pay attention
to how empty strings are handled. basically, empty strings appear as lines containing a single CRLF.

    ERROR(130) 10:22AM kory@kdd-ws:~ $ while true; do echo '' | nc -l localhost 10000; done
    POST / HTTP/1.1
    Host: localhost:10000
    User-Agent: curl/7.68.0
    Accept: */*
    Content-Length: 545
    Content-Type: multipart/form-data; boundary=------------------------65fd63b92908dbad

    --------------------------65fd63b92908dbad
    Content-Disposition: form-data; name="bytes"
    Content-Type: application/octect-stream

    KORY
    --------------------------65fd63b92908dbad
    Content-Disposition: form-data; name="key1"

    value1
    --------------------------65fd63b92908dbad
    Content-Disposition: form-data; name="key2"; filename="testfile.txt"
    Content-Type: application/octet-stream

    newline



    --------------------------65fd63b92908dbad
    Content-Disposition: form-data; name="key3"


    --------------------------65fd63b92908dbad
    Content-Disposition: form-data; name="key4"


    --------------------------65fd63b92908dbad--

notice that both outputs end with a CRLF (the terminal boundary).

- which endpoints/operations should support tickets?
    - /data-objects
        - read
        - write
        - parallel_write_init
        - stat
        - touch (does not support tickets yet)
    - /query
        - execute_genquery