s3-streamlogger version ^1.7.0 is encountering an error as Region is missing

[crissiaccubits]

The provided code using "s3-streamlogger" version ^1.7.0 is generating an error, and the issue pertains to a missing region.

"s3-streamlogger": "^1.7.0"

Code :

import { S3StreamLogger } from 's3-streamlogger';
import {  S3 } from '../config/env';

const s3Stream = new S3StreamLogger({
    ...S3,
    max_file_size: '20000000',
    name_format: `%Y-%m-%d-%H-%M-Ecosystem-Server-Logs-${hostname}.log`,
    rotate_every: '2592000000', // in milliseconds (30 days)
  });
  transportOptions = [
    new transports.Stream({
      stream: s3Stream,
    }),
  ];
  const logger = createLogger({
  level: config.level,
  format: config.jsonFormat
    ? format.combine(
        format.timestamp({
          format: 'YYYY-MM-DD HH:mm:ss',
        }),
        format.json()
      )
    : format.combine(
        format.errors({ stack: true }),
        format.timestamp({
          format: 'YYYY-MM-DD HH:mm:ss',
        }),
        alignFormat
      ),
  transports: transportOptions,
  exitOnError: false,
});

logger.stream = {
  write: function (message, encoding) {
    logger.http(message);
  },

config/env file as

  S3: {
    bucket: process.env.S3_BUCKET,
    access_key_id: process.env.S3_ACCESS_KEY_ID,
    secret_access_key: process.env.S3_SECRET_ACCESS_KEY,
    folder: process.env.S3_FOLDER,
  },
  

All these values are specified and verified by the log
Error as :

  
/usr/app/node_modules/@smithy/config-resolver/dist-cjs/regionConfig/config.js:10
       throw new Error("Region is missing");
             ^

Error: Region is missing
     at default (/usr/app/node_modules/@smithy/config-resolver/dist-cjs/regionConfig/config.js:10:15)
  at /usr/app/node_modules/@smithy/node-config-provider/dist-cjs/fromStatic.js:6:83
    at /usr/app/node_modules/@smithy/property-provider/dist-cjs/chain.js:12:39
    at coalesceProvider (/usr/app/node_modules/@smithy/property-provider/dist-cjs/memoize.js:14:24)
    at /usr/app/node_modules/@smithy/property-provider/dist-cjs/memoize.js:26:28
    at useFipsEndpoint (/usr/app/node_modules/@smithy/config-resolver/dist-cjs/regionConfig/resolveRegionConfig.js:21:74)
     at resolveParams (/usr/app/node_modules/@smithy/middleware-endpoint/dist-cjs/adaptors/getEndpointFromInstructions.js:29:40)
   at getEndpointFromInstructions (/usr/app/node_modules/@smithy/middleware-endpoint/dist-cjs/adaptors/getEndpointFromInstructions.js:7:28)
    at /usr/app/node_modules/@smithy/middleware-endpoint/dist-cjs/endpointMiddleware.js:8:26
  at /usr/app/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:7:26
Emitted 'error' event on S3StreamLogger instance at:
   at S3StreamLogger.<anonymous> (/usr/app/node_modules/s3-
   at /usr/app/node_modules/s3-streamlogger/index.js:94:20
Node.js v17.9.1

https://www.npmjs.com/package/s3-streamlogger/v/1.7.0

Please share an solution for the above issue.

[autopulated]

I think this must have been a change in moving to AWS SDK v3, sorry this should probably have been published as a breaking change, I didn't realise the SDK v3 had a breaking change in the way it requires the region like this.

You should ideally provide configuration (including the region) for the AWS SDK through one of the methods described here: https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-credentials-node.html, rather than passing them through the config as well.

Also, I notice that you're only rotating the file every 20MB or 30 days. Using smaller values here will significantly reduce your S3 data transfer costs for the logs.

[crissiaccubits]

@autopulated I have updated the version of both "s3-streamlogger": "^1.9.2" and "aws-sdk": "^2.1503.0".

Also configured the aws credentials as follows, but got the same error as Region is missing.

import { createLogger, format, transports } from 'winston';
import DailyRotateFile from 'winston-daily-rotate-file';
import os from 'os';
import { S3StreamLogger } from 's3-streamlogger';
import { S3 } from 'aws-sdk'; 

import config from '../config/logOptions';
import { NODE_ENV,S3Details } from '../config/env';

const file = config.jsonFormat ? config.jsonFile : config.textFile;
const hostname = os.hostname();

const alignFormat = format.printf(({ timestamp, level, message, stack }) => {
  if (stack) {
    return `${timestamp} : [ ${level} ] : ${message}\n${stack}`;
  }
  return `${timestamp} : [ ${level} ] : ${message}`;
});

let transportOptions;

if (NODE_ENV !== 'dev') {
  const awsCredentials = {
    accessKeyId: S3Details.access_key_id,
    secretAccessKey: S3Details.secret_access_key,
  };
  

  const s3Client = new S3({
    region: S3Details.region,
    credentials: awsCredentials,
  });

  const s3Stream = new S3StreamLogger({
    s3client: s3Client,
    bucket: S3Details.bucket, // Replace with your actual bucket name
    region: S3Details.region,
    max_file_size: '20000000',
    name_format: `%Y-%m-%d-%H-%M-Ecosystem-Server-Logs-${hostname}.log`,
    rotate_every: '2592000000', // in milliseconds (30 days)
  });

  transportOptions = [
    new transports.Stream({
      stream: s3Stream,
    }),
  ];
} else {
  transportOptions = [
    new DailyRotateFile(file),
    new transports.Console({
      format: format.combine(format.errors({ stack: true }), format.colorize(), alignFormat),
    }),
  ];
}

Error as :

usr/app/node_modules/@smithy/config-resolver/dist-cjs/regionConfig/config.js:10
        throw new Error("Region is missing");
           ^

 Error: Region is missing

[autopulated]

Version 1.9 uses AWS SDK v3 (specifically, "@aws-sdk/client-s3": "^3.353.0"). This will be used alongside any aws sdk v2 that you are importing into your project, and is not affected by it.

There is no such s3client option as the one you are passing to the S3StreamLogger constructor.

To configure the region either pass options.region, or configure the AWS SDK v3 using one of the other methods, like environment variables or the shared credentials file (https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/global-config-object.html).