Not able to use models permitted under Services on AWS AI Playground

Artificial Intelligence
#1

I am not able to use models permitted under Services on AWS AI Playground | KodeKloud

I am getting this error in my localhost terminal:

return new AuthenticationError(status, error, message, headers);
             ^


AuthenticationError: 401 User: arn:aws:iam::822233328238:user/kk_labs_user_504951 is not authorized to perform: bedrock-mantle:CreateInference on resource: arn:aws:bedrock-mantle:us-east-1:822233328238:project/default because no identity-based policy allows the bedrock-mantle:CreateInference action
    at Function.generate (/Users/t25/Downloads/genai/node_modules/openai/src/core/error.ts:76:14)
    at OpenAI.makeStatusError (/Users/t25/Downloads/genai/node_modules/openai/src/client.ts:494:28)
    at OpenAI.makeRequest (/Users/t25/Downloads/genai/node_modules/openai/src/client.ts:745:24)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async <anonymous> (/Users/taran/Downloads/genai/src/index.ts:11:18) {
  status: 401,
  headers: Headers {},
  requestID: 'req_lskvufom64ftqfqvz7zef37mieeqgdxnuls3w77jtqm4k7xfitva',
  error: {
    code: 'access_denied',
    message: 'User: arn:aws:iam::822233328238:user/kk_labs_user_504951 is not authorized to perform: bedrock-mantle:CreateInference on resource: arn:aws:bedrock-mantle:us-east-1:822233328238:project/default because no identity-based policy allows the bedrock-mantle:CreateInference action',
    param: null,
    type: 'permission_denied_error'
  },
  code: 'access_denied',
  param: null,
  type: 'permission_denied_error'
}

This was my index.ts file content:

I was just trying to see how llm responds

import dotenv from 'dotenv';
import OpenAI from 'openai';

dotenv.config();
const client = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
    baseURL: process.env.OPENAI_BASE_URL,
});
const response = await client.chat.completions.create({
    model: 'meta.llama3-2-3b-instruct-v1:0',
    messages: [
        {
            role: 'user',
            content:
                'In a single sentence, why should someone choose KodeKloud over other platforms to learn DevOps?',
        },
    ],
});
console.log(response.choices[0]?.message.content);

image
image1920×2221 181 KB

AWS AI Playground | KodeKloud mentions:

  • Models Permitted in Bedrock:
    ‍
    1. ‍[Llama 3.2 1B Instruct]
    2. ‍[Llama 3.2 3B Instruct]
    3. [Titan Embeddings G1 - Text]
#2

We’re aware of the issue; AWS has changed the API used, and we’ll need to enable it for the QuickStart examples to work.

#3

Thanks for taking a look Rob.
Any estimated time for completion for this update?

(Can’t wait to use Titan embedding model as well)

#4

Probably soon. I’d expect that the models will be limited; not sure if you’ll get “titan”. Writing correct scoping code for the models takes some time, which is why it isn’t done already.

#5

Hi Rob,
Any update on this?

#6

I’m still waiting for resolution on this particular error. For now, I’d recommend not relying on this feature – QuickStart needs a new set of permissions, and engineering is still trying to figure what they want so support, if any of it.

What does work now is our KodeKey feature: you can run models using the same OpenAI python API you’re using here, and it will work for you. Just generate a key using this playground instead.

#7

What will work in Bedrock is the “playground” feature, which allows testing models with simple prompts. But it appears that our engineering group does not want to grant the 12 hour short-term API keys that the QuickStart feature requires.

#8

Got a reply from the engineer – the feature you’re using requires granting API key access, which we can’t limit to a playground session. So we’ve updated the notes in the launch page to explain. Thanks for reporting this.