feat(documentation): Update documentation and standardize (#41)

* update qa construct documentation to match new format

* update doc for summarization

* update summarization userpool prop to be strongly types

* update associated test for summarization

* update documentation for lambda layer

* fix indentation

* remove unused markdown block

* various fixes in summarization construct to standardize naming

* add missing import in pattern depl

src/common/helpers/README.md

@@ -12,9 +12,9 @@
 ---
 <!--END STABILITY BANNER-->
 
-| **Reference Documentation**:| TODO|
-|:-------------|:-------------|
-<div style="height:8px"></div>
+Most of the helper functions in this folder are extracted from: https://github.com/awslabs/aws-solutions-constructs 
 
 The helper library includes the basic building blocks of the Emerging Tech construct library. It defines the core classes that are used in the rest of the library. The purpose is to standardize and accelerate development for builders.
 These methods should not be used by end customers.
+
+

src/patterns/gen-ai/aws-langchain-common-layer/README.md

@@ -12,14 +12,25 @@
 ---
 <!--END STABILITY BANNER-->
 
-| **Reference Documentation**:| <span style="font-weight: normal">TODO</span>|
-|:-------------|:-------------|
-<div style="height:8px"></div>
-
 | **Language**     | **Package**        |
 |:-------------|-----------------|
 |![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-samples/@emerging_tech_cdk_constructs`|
 
+## Table of contents
+
+- [Credits](#credits)
+- [Overview](#overview)
+- [Initializer](#initializer)
+- [Pattern Construct Props](#pattern-construct-props)
+- [Pattern Properties](#pattern-properties)
+- [Default properties](#default-properties)
+- [Troubleshooting](#troubleshooting)
+- [Architecture](#architecture)
+- [Cost](#cost)
+- [Security](#security)
+- [Supported AWS Regions](#supported-aws-regions)
+- [Quotas](#quotas)
+
 ## Credits
 
 This construct is a modified version of the following construct: https://github.com/aws-samples/aws-genai-llm-chatbot/tree/main/lib/model-interfaces/langchain 
@@ -28,12 +39,6 @@ Thanks to the original authors:
 - [Bigad Soleiman](https://www.linkedin.com/in/bigadsoleiman/)
 - [Sergey Pugachev](https://www.linkedin.com/in/spugachev/)
 
-## Precautions
-
-- Cost Management with self-hosted models on SageMaker: Be mindful of the costs associated with AWS resources, especially with SageMaker models billed by the hour.
-
-- Licensing obligations: If you choose to use any datasets or models alongside the provided samples, ensure you check the LLM code and comply with all licensing obligations attached to them.
-
 ## Overview
 
 This construct provides two AWS Lambda layers:
@@ -162,5 +167,52 @@ response = model.run(
 ## Architecture
 ![Architecture Diagram](architecture.png)
 
+## Cost
+
+You are responsible for the cost of the AWS services used while running this construct. As of this revision, the cost for running this construct with the default settings in the US East (N. Virginia) Region is approximately $X per month.
+
+We recommend creating a budget through [AWS Cost Explorer](http://aws.amazon.com/aws-cost-management/aws-cost-explorer/) to help manage costs. Prices are subject to change. For full details, refer to the pricing webpage for each AWS service used in this solution.
+
+The following table provides a sample cost breakdown for deploying this solution with the default parameters in the **US East (N. Virginia)** Region for **one month**.
+
+
+| **AWS Service**     | **Dimensions**        | **Cost [USD]** |
+|:-------------|:----------------|-----------------|
+| AWS Lambda | |  |
+| Total Deployment cost | | |
+
+> **Note**
+> The costs of API calls to third-party LLM providers are not included in these estimates. See the pricing guide of your LLM provider.
+
+## Security
+
+When you build systems on AWS infrastructure, security responsibilities are shared between you and AWS. This [shared responsibility](http://aws.amazon.com/compliance/shared-responsibility-model/) model reduces your operational burden because AWS operates, manages, and controls the components including the host operating system, virtualization layer, and physical security of the facilities in which the services operate. For more information about AWS security, visit [AWS Cloud Security](http://aws.amazon.com/security/).
+
+If choosing to interface with a third-party LLM provider (outside of Amazon Bedrock), customers must evaluate the security considerations of data leaving their AWS account. The customer is required to provide a valid API key at the time of deployment.
+
+The primary piece of data sent to the third-party LLM provider is the prompt to perform inference on. Depending on the use case, the prompt can contain the user’s input, previous interactions (for example, chat history), and document excerpts sourced from the configured knowledge base (for example, Amazon OpenSearch search result).
+
+> **Warning**
+> This construct allows you to interact with models from third party providers. Your use of the third-party Generative AI models is governed by the terms provided to you by the third-party GAI model providers when you acquired your license to use them (for example, their terms of service, license agreement, acceptable use policy, and privacy policy).
+>
+>You are responsible for ensuring that your use of the third-party GAI models comply with the terms governing them, and any laws, rules, regulations, policies, or standards that apply to you.
+>
+>You are also responsible for making your own independent assessment of the third-party GAI models that you use, including their outputs and how third-party GAI model providers use any data that might be transmitted to them based on your deployment configuration. AWS does not make any representations, warranties, or guarantees regarding the third-party GAI models, which are “Third-Party Content” under your agreement with AWS. This construct is offered to you as “AWS Content” under your agreement with AWS.
+
+## Supported AWS Regions
+
+This solution optionally uses the Amazon Bedrock and Amazon OpenSearch service, which is not currently available in all AWS Regions. You must launch this construct in an AWS Region where these services are available. For the most current availability of AWS services by Region, see the [AWS Regional Services List](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/).
+
+> **Note**
+>You need to explicity enable access to models before they are available for use in the Amazon Bedrock service. Please follow the [Amazon Bedrock User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for steps related to enabling model access.
+
+## Quotas
+
+Service quotas, also referred to as limits, are the maximum number of service resources or operations for your AWS account.
+
+Make sure you have sufficient quota for each of the services implemented in this solution. For more information, refer to [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html).
+
+To view the service quotas for all AWS services in the documentation without switching pages, view the information in the [Service endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/aws-general.pdf#aws-service-information) page in the PDF instead.
+
 ***
 &copy; Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

src/patterns/gen-ai/aws-qa-appsync-opensearch/README.md

@@ -12,20 +12,34 @@
 ---
 <!--END STABILITY BANNER-->
 
-| **Reference Documentation**:| <span style="font-weight: normal">TODO</span>|
-|:-------------|:-------------|
-<div style="height:8px"></div>
-
 | **Language**     | **Package**        |
 |:-------------|-----------------|
 |![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-samples/@emerging_tech_cdk_constructs`|
 
+## Table of contents
+
+- [Overview](#overview)
+- [Initializer](#initializer)
+- [Pattern Construct Props](#pattern-construct-props)
+- [Pattern Properties](#pattern-properties)
+- [Default properties](#default-properties)
+- [Troubleshooting](#troubleshooting)
+- [Architecture](#architecture)
+- [Cost](#cost)
+- [Security](#security)
+- [Supported AWS Regions](#supported-aws-regions)
+- [Quotas](#quotas)
+
 ## Overview
 
 This construct provides a question answering workflow (RAG + long context window) using Amazon Bedrock and a provisioned Amazon OpenSearch cluster. 
 - If a document is provided as an input to the Appsync query, the AWS Lambda function will first verify the length of the document. If the document size is above the max number of tokens for the selected model, the Lambda will query the knowledge base (similarity search) and filter by document name. This assumes that the chunks of texts stored in the knowledge base have the document name as metadata. Otherwise, the content of the document is provided to the LLM as part of the context.
 - If no document is provided as input, the Lambda will perform a similarity search against the entire knowledge base.
 
+The construct uses Amazon Bedrock as the Large Language Model provider. amazon.titan-embed-text-v1 is used as the embeddings model to query the knowledge base (provisioned Amazon OpenSearch cluster), and anthropic.claude-v2 for question answering. Make sure both models are enabled in your account. Please follow the [Amazon Bedrock User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for steps related to enabling model access.
+
+The input document must be stored in the input Amazon Simple Storage Service bucket in text format (.txt). Another construct is available to ingest and process files to text format and store them in a knowledge base: [aws-rag-appsync-stepfn-opensearch](../aws-rag-appsync-stepfn-opensearch/README.md).
+
 This construct builds a Lambda function from a Docker image, thus you need to have [Docker desktop](https://www.docker.com/products/docker-desktop/) running on your machine.
 
 Here is a minimal deployable pattern definition:
@@ -34,22 +48,86 @@ Typescript
 ``` typescript
 import { Construct } from 'constructs';
 import { Stack, StackProps } from 'aws-cdk-lib';
+import * as os from 'aws-cdk-lib/aws-opensearchservice';
+import * as cognito from 'aws-cdk-lib/aws-cognito';
 import { QaAppsyncOpensearch, QaAppsyncOpensearchProps } from '@aws-samples/aws-emerging-tech-constructs';
 
-new QaAppsyncOpensearch(this, 'new-construct', constructProps);
+// get an existing OpenSearch provisioned cluster
+const osDomain = os.Domain.fromDomainAttributes(this, 'osdomain', {
+    domainArn: 'arn:aws:es:us-east-1:XXXXXX',
+    domainEndpoint: 'https://XXXXX.us-east-1.es.amazonaws.com'
+});
+
+// get an existing userpool 
+const cognitoPoolId = 'us-east-1_XXXXX';
+const userPoolLoaded = cognito.UserPool.fromUserPoolId(this, 'myuserpool', cognitoPoolId);
+
+const rag_source = new emergingTech.QaAppsyncOpensearch(
+      this,
+      'QaAppsyncOpensearch',
+      {
+        existingOpensearchDomain: osDomain,
+        openSearchIndexName: 'demoindex',
+        cognitoUserPool: userPoolLoaded
+      }
+    )
+```
+
+The code below provides an example of a mutation call and associated subscription to trigger a question and get response notifications:
+
+Mutation call to trigger the question:
+
+```
+mutation MyMutation {
+  postQuestion(jobid: "123", jobstatus: "", max_docs: 10, question: "d2hhdCBpcyB0aGlzIGRvY3VtZW50IGFib3V0ID8=", verbose: true, filename: "document.txt") {
+    answer
+    jobid
+    jobstatus
+    max_docs
+    question
+    verbose
+  }
+}
 ```
 
+Where:
+- jobid: id which can be used to filter subscriptions on client side
+- jobstatus: this field will be used by the subscription to update the status of the question answering process for the file specified
+- max_docs: maximum number of documents (chunks) retrieved from the knowledge base if the Retrieveal Augmented Generation (RAG) approach is used 
+- question: question to ask as a base64 encoded string
+- verbose: boolean indicating if the Langchain chain call verbosity should be enabled or not
+- filename: optional. Name of the file stored in the input S3 bucket, in txt format.
+
+Subscription call to get notifications about the question answering process:
+
+```
+subscription MySubscription {
+  updateQAJobStatus(jobid: "123") {
+    sources
+    question
+    answer
+    jobstatus
+  }
+}
+```
+
+Where:
+- jobid: id which can be used to filter subscriptions on client side
+- answer: response to the question from the Large Language Model as a base64 encoded string
+- sources: sources from the knowledge base used as context to answer the question
+- jobstatus: status update of the question answering process for the file specified
+
 ## Initializer
 
 ```
-new QaAppsyncOpensearch(scope: Construct, id: string, props: RagApiGatewayOpensearchProps)
+new QaAppsyncOpensearch(scope: Construct, id: string, props: QaAppsyncOpensearchProps)
 ```
 
 Parameters
 
 - scope [Construct](https://docs.aws.amazon.com/cdk/api/v2/docs/constructs.Construct.html)
 - id string
-- props RagApiGatewayOpensearchProps
+- props QaAppsyncOpensearchProps
 
 ## Pattern Construct Props
 
@@ -66,7 +144,8 @@ Parameters
 | existingInputAssetsBucketObj | [s3.IBucket](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.IBucket.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing instance of S3 Bucket object, providing both this and `bucketInputsAssetsProps` will cause an error. |
 | bucketInputsAssetsProps | [s3.BucketProps](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.BucketProps.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | User provided props to override the default props for the S3 Bucket. Providing both this and `existingInputAssetsBucketObj` will cause an error. |
 | stage | string | ![Optional](https://img.shields.io/badge/optional-4169E1) | Value will be appended to resources name Service. |
-| existingMergedApi | [appsync.CfnGraphQLApi](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_appsync.CfnGraphQLApi.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing merged api instance. The merge API provode a federated schema over source API schemas.|
+| existingMergedApi | [appsync.CfnGraphQLApi](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_appsync.CfnGraphQLApi.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing merged api instance. The merge API provides a federated schema over source API schemas.|
+| observability | boolean | ![Optional](https://img.shields.io/badge/optional-4169E1) | Enables observability on all services used. Warning: associated cost with the services used. Best practice to enable by default. Defaults to true.|
 
 ## Pattern Properties
 
@@ -83,13 +162,27 @@ Parameters
 
 Out of the box implementation of the Construct without any override will set the following defaults:
 
-### AppSync
+### Authentication
+
+- Primary authentication method for the AppSync GraphQL API is Amazon Cognito User Pool.
+- Secondary authentication method for the AppSync graphQL API is IAM role.
+
+### Networking
+
+- Set up a VPC
+    - Uses existing VPC if provided, otherwise creates a new one
+- Set up a Security Group used by the AWS Lambda functions
+    - Uses existing Security Group, otherwise creates a new one
 
 ### Amazon S3 Bucket
 
 - Uses existing bucket if provided, otherwise creates a new one
 
+### Observability
 
+By default the construct will enable logging and tracing on all services which support those features. Observability can be turned off by setting the pattern property ```observability``` to false. 
+- AWS Lambda: AWS X-Ray, Amazon Cloudwatch Logs
+- AWS AppSync GraphQL api: AWS X-Ray, Amazon Cloudwatch Logs
 
 ## Troubleshooting
 
@@ -100,5 +193,55 @@ Out of the box implementation of the Construct without any override will set the
 ## Architecture
 ![Architecture Diagram](architecture.png)
 
+## Cost
+
+You are responsible for the cost of the AWS services used while running this construct. As of this revision, the cost for running this construct with the default settings in the US East (N. Virginia) Region is approximately $X per month.
+
+We recommend creating a budget through [AWS Cost Explorer](http://aws.amazon.com/aws-cost-management/aws-cost-explorer/) to help manage costs. Prices are subject to change. For full details, refer to the pricing webpage for each AWS service used in this solution.
+
+The following table provides a sample cost breakdown for deploying this solution with the default parameters in the **US East (N. Virginia)** Region for **one month**.
+
+
+| **AWS Service**     | **Dimensions**        | **Cost [USD]** |
+|:-------------|:----------------|-----------------|
+| Amazon Virtual Private Cloud |  |  |
+| AWS AppSync |  |  |
+| Amazon EventBridge |  |  |
+| AWS Lambda |  |  |
+| Amazon Simple Storage Service |  |  |
+| Amazon Bedrock |  |  |
+| Amazon Cloudwatch | | |
+| AWS X-Ray | | |
+| AWS X-Ray | | |
+| Total Deployment cost | | |
+
+The resources not created by this construct (Amazon Cognito User Pool, Amazon OpenSearch provisioned cluster, AppSync Merged API, AWS Secrets Manager secret) do not appear in the table above. You can refer to the decicated pages to get an estimate of the cost related to those services:
+- [Amazon OpenSearch Service Pricing](https://aws.amazon.com/opensearch-service/pricing/)
+- [AWS AppSync pricing (for Merged API if used)](https://aws.amazon.com/appsync/pricing/)
+- [Amazon Cognito Pricing](https://aws.amazon.com/cognito/pricing/)
+- [AWS Secrets Manager Pricing](https://aws.amazon.com/secrets-manager/pricing/)
+
+> **Note**
+>You can share the Amazon OpenSearch provisioned cluster between use cases, but this can drive up the number of queries per index and additional charge will apply.
+
+## Security
+
+When you build systems on AWS infrastructure, security responsibilities are shared between you and AWS. This [shared responsibility](http://aws.amazon.com/compliance/shared-responsibility-model/) model reduces your operational burden because AWS operates, manages, and controls the components including the host operating system, virtualization layer, and physical security of the facilities in which the services operate. For more information about AWS security, visit [AWS Cloud Security](http://aws.amazon.com/security/).
+
+## Supported AWS Regions
+
+This solution optionally uses the Amazon Bedrock and Amazon OpenSearch service, which is not currently available in all AWS Regions. You must launch this construct in an AWS Region where these services are available. For the most current availability of AWS services by Region, see the [AWS Regional Services List](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/).
+
+> **Note**
+>You need to explicity enable access to models before they are available for use in the Amazon Bedrock service. Please follow the [Amazon Bedrock User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for steps related to enabling model access.
+
+## Quotas
+
+Service quotas, also referred to as limits, are the maximum number of service resources or operations for your AWS account.
+
+Make sure you have sufficient quota for each of the services implemented in this solution. For more information, refer to [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html).
+
+To view the service quotas for all AWS services in the documentation without switching pages, view the information in the [Service endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/aws-general.pdf#aws-service-information) page in the PDF instead.
+
 ***
 &copy; Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.