Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 160 additions & 0 deletions packages/batch/API_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# Powertools for AWS Lambda (TypeScript) - Batch Processing Utility

Powertools for AWS Lambda (TypeScript) is a developer toolkit to implement Serverless [best practices and increase developer velocity](https://docs.aws.amazon.com/powertools/typescript/latest/#features).

You can use the library in both TypeScript and JavaScript code bases.

## Intro

The Batch Processing utility handles partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.

## Usage

To get started, install the library by running:

```sh
npm i @aws-lambda-powertools/batch
```

### Batch Processor

When using SQS, Kinesis Data Streams, or DynamoDB Streams as a Lambda event source, your Lambda functions are triggered with a batch of messages.

If your function fails to process any message from the batch, the entire batch returns to your queue or stream. This same batch is then retried until either condition happens first: **a)** your Lambda function returns a successful response, **b)** record reaches maximum retry attempts, or **c)** when records expire.

With this utility, batch records are processed individually – only messages that failed to be processed return to the queue or stream for a further retry.

### SQS Processor

When using SQS as a Lambda event source, you can specify the `EventType.SQS` to process the records. The response will be a `SQSBatchResponse` which contains a list of items that failed to be processed.

```ts
import {
BatchProcessorSync,
EventType,
processPartialResponseSync,
} from '@aws-lambda-powertools/batch';
import { Logger } from '@aws-lambda-powertools/logger';
import type { SQSHandler, SQSRecord } from 'aws-lambda';

const processor = new BatchProcessorSync(EventType.SQS);
const logger = new Logger();

const recordHandler = (record: SQSRecord): void => {
const payload = record.body;
if (payload) {
const item = JSON.parse(payload);
logger.info('Processed item', { item });
}
};

export const handler: SQSHandler = async (event, context) =>
processPartialResponseSync(event, recordHandler, processor, {
context,
});
```

### Kinesis Processor

When using Kinesis Data Streams as a Lambda event source, you can specify the `EventType.KinesisDataStreams` to process the records. The response will be a `KinesisStreamBatchResponse` which contains a list of items that failed to be processed.

```ts
import {
BatchProcessorSync,
EventType,
processPartialResponseSync,
} from '@aws-lambda-powertools/batch';
import { Logger } from '@aws-lambda-powertools/logger';
import type { KinesisStreamHandler, KinesisStreamRecord } from 'aws-lambda';

const processor = new BatchProcessorSync(EventType.KinesisDataStreams);
const logger = new Logger();

const recordHandler = (record: KinesisStreamRecord): void => {
logger.info('Processing record', { record: record.kinesis.data });
const payload = JSON.parse(record.kinesis.data);
logger.info('Processed item', { item: payload });
};

export const handler: KinesisStreamHandler = async (event, context) =>
processPartialResponseSync(event, recordHandler, processor, {
context,
});
```

### DynamoDB Streams Processor

When using DynamoDB Streams as a Lambda event source, you can use the `BatchProcessorSync` with the `EventType.DynamoDBStreams` to process the records. The response will be a `DynamoDBBatchResponse` which contains a list of items that failed to be processed.

```ts
import {
BatchProcessor,
EventType,
processPartialResponseSync,
} from '@aws-lambda-powertools/batch';
import { Logger } from '@aws-lambda-powertools/logger';
import type { DynamoDBRecord, DynamoDBStreamHandler } from 'aws-lambda';

const processor = new BatchProcessor(EventType.DynamoDBStreams); // (1)!
const logger = new Logger();

const recordHandler = (record: DynamoDBRecord): void => {
if (record.dynamodb && record.dynamodb.NewImage) {
logger.info('Processing record', { record: record.dynamodb.NewImage });
const message = record.dynamodb.NewImage.Message.S;
if (message) {
const payload = JSON.parse(message);
logger.info('Processed item', { item: payload });
}
}
};

export const handler: DynamoDBStreamHandler = async (event, context) =>
processPartialResponseSync(event, recordHandler, processor, {
context,
});
```

### Async processing

If your use case allows you to process multiple records at the same time without conflicting with each other, you can use the `BatchProcessor` to process records asynchronously. This will create an array of promises that will be resolved once all records have been processed.

```ts
import {
BatchProcessor,
EventType,
processPartialResponse,
} from '@aws-lambda-powertools/batch';
import type { SQSHandler, SQSRecord } from 'aws-lambda';

const processor = new BatchProcessor(EventType.SQS);

const recordHandler = async (record: SQSRecord): Promise<number> => {
const res = await fetch('https://httpbin.org/anything', {
body: JSON.stringify({ message: record.body }),
});

return res.status;
};

export const handler: SQSHandler = async (event, context) =>
await processPartialResponse(event, recordHandler, processor, {
context,
});
```

Check the [docs](https://docs.aws.amazon.com/powertools/typescript/latest/features/batch/) for more examples.

## Contribute

If you are interested in contributing to this project, please refer to our [Contributing Guidelines](https://github.com/aws-powertools/powertools-lambda-typescript/blob/main/CONTRIBUTING.md).

## Roadmap

The roadmap of Powertools for AWS Lambda (TypeScript) is driven by customers’ demand.
Help us prioritize upcoming functionalities or utilities by [upvoting existing RFCs and feature requests](https://github.com/aws-powertools/powertools-lambda-typescript/issues), or [creating new ones](https://github.com/aws-powertools/powertools-lambda-typescript/issues/new/choose), in this GitHub repository.

## Connect

- **Powertools for AWS Lambda on Discord**: `#typescript` - **[Invite link](https://discord.gg/B8zZKbbyET)**
- **Email**: <[email protected]>
2 changes: 1 addition & 1 deletion packages/batch/typedoc.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"extends": ["../../typedoc.base.json"],
"entryPoints": ["./src/index.ts", "./src/parser.ts", "./src/types.ts"],
"readme": "README.md"
"readme": "./API_README.md"
}
99 changes: 99 additions & 0 deletions packages/commons/API_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Powertools for AWS Lambda (TypeScript) <!-- omit in toc -->

Powertools for AWS Lambda (TypeScript) is a developer toolkit to implement Serverless [best practices and increase developer velocity](https://docs.aws.amazon.com/powertools/typescript/latest/#features).

You can use the library in both TypeScript and JavaScript code bases.

## Intro

The Commons package contains a set of utilities that are shared across one or more Powertools for AWS Lambda (TypeScript) utilities. Some of these utilities can also be used independently in your AWS Lambda functions.

## Usage

To get started, install the utility by running:

```sh
npm i @aws-lambda-powertools/commons
```

### Type utils

When working with different objects and values, you may want to do runtime type checks. The utility comes with a set of type utilities that you can use to check the type of an object or value.

```typescript
import { isRecord } from '@aws-lambda-powertools/commons/typeUtils';
import { isString } from '@aws-lambda-powertools/commons/typeUtils';
import { isTruthy } from '@aws-lambda-powertools/commons/typeUtils';


const value = { key: 'value' };
if (isRecord(value)) {
// value is a record
}

const stringValue = 'string';
if (isString(stringValue)) {
// stringValue is a string
}

const truthyValue = 'true';
if (isTruthy(truthyValue)) {
// truthyValue is truthy
}
```

You can find a full list of type utilities available [in the API docs](https://docs.aws.amazon.com/powertools/typescript/latest/api/modules/_aws-lambda-powertools_commons.typeUtils.html). Many of these utilities also double as type guards, which you can use to narrow down the type of an object or value.

### Base64 utils

When working with Base64-encoded data, you can use the `fromBase64` utilities to quickly decode data and convert it to a `Uint8Array`.

```typescript

import { fromBase64 } from '@aws-lambda-powertools/commons/utils/base64';

const encodedValue = 'aGVsbG8gd29ybGQ=';

const decoded = fromBase64(encodedValue);
// new Uint8Array([ 97, 71, 86, 115, 98, 71, 56, 103, 100, 50, 57, 121, 98, 71, 81, 61 ]);
```

### JSON type utils

In some cases, you may want to define a type for a JSON object or value. The utility comes with a set of types that you can use to define your JSON objects.

```typescript
import type { JSONValue, JSONObject, JSONArray } from '@aws-lambda-powertools/commons';
```

### Lambda interface

When using object-oriented patterns to define your Lambda handlers, you can use the `LambdaHandler` interface to define the shape of your handler methods.

```typescript
import type { Context } from 'aws-lambda';
import type { LambdaInterface } from '@aws-lambda-powertools/commons/types';

class Lambda implements LambdaInterface {
public handler = async (event: unknown, context: Context) => {
// Your handler code here
}
}

const handlerClass = new Lambda();
export const handler = lambda.handler.bind(lambda);
```

## Contribute

If you are interested in contributing to this project, please refer to our [Contributing Guidelines](https://github.com/aws-powertools/powertools-lambda-typescript/blob/main/CONTRIBUTING.md).

## Roadmap

The roadmap of Powertools for AWS Lambda (TypeScript) is driven by customers’ demand.
Help us prioritize upcoming functionalities or utilities by [upvoting existing RFCs and feature requests](https://github.com/aws-powertools/powertools-lambda-typescript/issues), or [creating new ones](https://github.com/aws-powertools/powertools-lambda-typescript/issues/new/choose), in this GitHub repository.

## Connect

- **Powertools for AWS Lambda on Discord**: `#typescript` - **[Invite link](https://discord.gg/B8zZKbbyET)**
- **Email**: <[email protected]>
2 changes: 1 addition & 1 deletion packages/commons/typedoc.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@
"./src/fromBase64.ts",
"./src/LRUCache.ts"
],
"readme": "./README.md"
"readme": "./API_README.md"
}
Loading