Transformerの学習済みモデルをLambdaにデプロイする(SAM)

$ sam init

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Multi-step workflow
        3 - Serverless API
        4 - Scheduled task
        5 - Standalone function
        6 - Data processing
        7 - Infrastructure event management
        8 - Machine Learning
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: n

Which runtime would you like to use?
        1 - dotnet6
        2 - dotnet5.0
        3 - dotnetcore3.1
        4 - go1.x
        5 - graalvm.java11 (provided.al2)
        6 - graalvm.java17 (provided.al2)
        7 - java11
        8 - java8.al2
        9 - java8
        10 - nodejs16.x
        11 - nodejs14.x
        12 - nodejs12.x
        13 - python3.9
        14 - python3.8
        15 - python3.7
        16 - python3.6
        17 - ruby2.7
        18 - rust (provided.al2)
Runtime: 13       　　　　　　　　　　　  ※

What package type would you like to use?
        1 - Zip
        2 - Image
Package type: 2

Based on your selections, the only dependency manager available is pip.
We will proceed copying the template using pip.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: 

Project name [sam-app]: text2text

Cloning from https://github.com/aws/aws-sam-cli-app-templates (process may take a moment)

    -----------------------
    Generating application:
    -----------------------
    Name: sample
    Base Image: amazon/python3.9-base
    Architectures: x86_64
    Dependency Manager: pip
    Output Directory: .

    Next steps can be found in the README file at ./sample/README.md
    

    Commands you can use next
    =========================
    [*] Create pipeline: cd sample && sam pipeline init --bootstrap
    [*] Validate SAM template: sam validate
    [*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
    

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 300
    MemorySize: 3008

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      PackageType: Image
      Architectures:
        - x86_64
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /text2text  ※
            Method: POST      ※
    Metadata:

  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/text2text/"  ※

requests
-f https://download.pytorch.org/whl/torch_stable.html
torch==1.8.0+cpu
transformers==4.20.1
sentencepiece

FROM public.ecr.aws/lambda/python:3.9

COPY app.py requirements.txt ./
COPY model01 /opt/ml/model               //追加

RUN python3.9 -m pip install -r requirements.txt -t .

# Command can be overwritten by providing a different command in the template directly.
CMD ["app.lambda_handler"]

from email.quoprimime import body_check
import json
import torch
from transformers import T5ForConditionalGeneration, T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("/opt/ml/model")
trained_model = T5ForConditionalGeneration.from_pretrained("/opt/ml/model")

def lambda_handler(event, context):

    raw_string = r'{}'.format(event['body'])
    body = json.loads(raw_string)
    data = body['text']

    MAX_SOURCE_LENGTH = 24   # 入力される記事本文の最大トークン数
    MAX_TARGET_LENGTH = 24  # 生成されるタイトルの最大トークン数

    # 推論モード設定
    trained_model.eval()

    # 前処理とトークナイズを行う

    inputs = [data]
    batch = tokenizer.batch_encode_plus(
        inputs, max_length=MAX_SOURCE_LENGTH, truncation=True, 
        padding="longest", return_tensors="pt")

    input_ids = batch['input_ids']
    input_mask = batch['attention_mask']

    outputs = trained_model.generate(
        input_ids=input_ids, attention_mask=input_mask, 
        min_length=8,
        temperature=1.0,          # 生成にランダム性を入れる温度パラメータ
        num_beams=5,             # ビームサーチの探索幅
        diversity_penalty=1.0,    # 生成結果の多様性を生み出すためのペナルティ
        num_beam_groups=5,       # ビームサーチのグループ数
        num_return_sequences=3,  # 生成する文の数
        repetition_penalty=1.5,   # 同じ文の繰り返し（モード崩壊）へのペナルティ
    )

    # 生成されたトークン列を文字列に変換する
    generated_titles = [tokenizer.decode(ids, skip_special_tokens=True, 
                                        clean_up_tokenization_spaces=False) 
                        for ids in outputs]
    
　　return {
        "statusCode": 200,
        "body": json.dumps({"generated_titles": generated_titles}),
    }

        Stack Name [text2text]: 
        AWS Region [ap-northeast-1]: 
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [Y/n]: 
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]: 
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]: 
        HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
        Save arguments to configuration file [Y/n]: 
        SAM configuration file [samconfig.toml]: 
        SAM configuration environment [default]: